Softsign Function Hardware Implementation Using Piecewise Linear Approximation

Softsign is a widely used activation function in recurrent neural networks. However, no special attention has been paid to the hardware implementation of Softsign function. In this paper, we propose a hardware architecture, which is based on the piecewise linear (PWL) approximation, for the hardware implementation of Softsign function. The main advantage of the proposed hardware architecture is that we develop a low-power high-accuracy approximate multiplier for the PWL approximation. Implementation results show that the proposed hardware architecture can save 55.1 % power consumption with only 4.2% increase in mean absolute error.