FPGA-Based Sparsity-Aware CNN Accelerator for Noise-Resilient Edge-Level Image Recognition

This paper presents a novel sparsity-aware CNN accelerator supporting the edge-level image recognition even for noisy images. In the proposed accelerator, we characterize the class of input noises by utilizing the FFT-based on-the-fly noise classifier. The proposed convolution engine then accesses the external memory to load the dedicated network that provides the accurate inference processing for the detected noise class. To save the energy consumed by the external DRAM accesses, in addition, we present the filter-level pruning algorithm with the memory-reduced indexing scheme, which can reduces the processing latency by utilizing the indexing method. To verify the effectiveness of the proposed methods, the proposed CNN accelerator is implemented in the commercialized FPGA-based platform, achieving the processing rate of 57.6GOPS at the speed of 100MHz. Utilizing the dedicated network for each noise type, the prototype accelerator reduces the energy consumption by 62% compared to the conventional network with a similar recognition accuracy, which is suitable for the intelligent edge-level devices subjected to the various noises in practice.