A High Speed SoftMax VLSI Architecture Based on Basic-Split

Due to simple to calculate and derivate, SoftMax is widely used in neural network computing as an activation function. But how to guarantee the accuracy and speed while minimizing the consumption of hardware resources is the most critical issue in the current SoftMax hardware design. The available Polynomial Fitting and CORDIC are contradictory in resource occupancy and accuracy. To meet the need of SoftMax, this paper proposes a new method named Basic-Split Calculation. Taking advantage of the math feature of exponential function, the Basic-Split Calculation method splits the exponentiation calculation of the SoftMax into several specific basics which is implemented by look-up table. Based on this strategy, the complex exponential calculation process is implemented by look-up table procedure and multiply operation. It simplify the hardware complexity and logic propagation delay remarkably. Experiment results show that the actual error rate is 10 ⁻⁸ magnitude. And, synthesis report based on SMIC65nm technology library show that, the operation frequency can be up to 1GHz. Compared with the existing design, Basis-Split Calculation method has a lot of advantages, such as faster calculation speed, higher accuracy, larger throughput.