Domain-Specific Accelerator Design & Profiling for Deep Learning Applications From Circuits to Architecture

The field of computer architecture is exiting an era of predictable gains, and entering an era of rapid change. For decades, the trends of Dennard scaling and Moore’s law improved energy-delay product [Gonzalez96] while allowing for ever-higher numbers of transistors to be integrated on a single chip. These trends set the pace for the entire hardware industry, and ultimately drove the economics of computation. Year over year, computer users could count on hardware that was faster, and oftentimes cheaper. If an application didn’t work well on existing hardware, there was a decent chance that next year’s refresh would bring about a processor that was up to the task. From the 2 MHz Intel 8008 in 1974, to the 3.8 GHz Pentium 4 Prescott in 2005, CPU clock speed increased nearly 2000X over three decades. (Dennard scaling enables smaller, faster logic to fit within existing power and thermal envelopes. [Dennard74])