A High-performance Hardware Accelerator using a Fusion Approach of Convolution and Pooling

Convolutional neural networks (CNNs) have shown excellent effect in many fields. However, their increasingly depth and scales make it difficult to apply them in resource constraint scenarios, such as Internet of Things (IOT) and mobile devices. Therefore, by exploiting the data extraction of the pooling layers in CNNs, a Fusion Approach of Convolution and Pooling (FACP) was proposed to eliminate the redundant operations inter convolution and pooling (i.e., average-pooling and max-pooling) layers. In addition, a high-performance and efficient hardware accelerator with an FACP component was designed to deploy our proposed FACP. Our designed accelerator was implemented on the Xilinx XCVU9P platform and run at 300MHz clock frequency. Under the evaluation of AlexNet, our hardware accelerator can achieve 962.45 performance and 0.94 DSP efficiency. Compared with previous works, the performance and DSP efficiency can achieve up to 5.57× and 3.47×, respectively.