CEST: Computation-Efficient N:M Sparse Training for Deep Neural Networks

2023 Design, Automation & Test in Europe Conference & Exhibition (DATE)(2023)

引用 0|浏览0
N:M fine-grained structured sparsity has attracted attention due to its practical sparsity ratio and hardware-friendly pattern. However, the potential to accelerate N:m sparse deep neural network (DNN) training has not been fully exploited, and there is a lack of efficient hardware supporting N:M sparse training. To tackle these challenges, this paper presents a computation-efficient scheme for N:M sparse DNN training, called CEST. A bidirectional weight pruning method, dubbed BDWP, is firstly proposed to significantly reduce the computational cost while maintaining model accuracy. A sparse accelerator, namely SAT, is further developed to neatly support both the regular dense operations and N:M sparse operations. Experimental results show CEST significantly improves the training throughput by $1.89-12.49\times$ and the energy efficiency by $1.86-2.76\times$ .
AI 理解论文