Empirical Analysis of Full-System Approximation on Non-Spiking and Spiking Neural Networks.

Spiking Neural Networks (SNNs) are a promising alternative to conventional neural networks. In particular, Spikeby-Spike (SbS) neural networks offer exceptional noise robustness and reduced complexity. The error resilience of SbS provides a unique opportunity to improve its hardware efficiency with minimal loss of accuracy. Approximate processing is a mechanism that exploits the error resilience of applications to improve hardware cost and efficiency. Most of the previous works have mainly focused on quantization or approximation of a single subsystem, ignoring the potential improvements of simultaneous communication and computation approximation and quantization. This paper, for the first time, investigates the robustness of Convolutional Neural Network (CNN) and SbS inference for a combination of quantization and approximate communication and computation. We have shown that the full system approximation and quantization results in a better accuracy-cost trade-off rather than applying approximation or quantization individually. Furthermore, we observed that the SbS is more robust than the CNN in most cases, and can even improve the accuracy by more than 0.6% compared to an exactly unquantized network.