CEST: Computation-Efficient N:M Sparse Training for Deep Neural Networks

DATE(2023)

引用 0|浏览16
暂无评分
摘要
N:M fine-grained structured sparsity has attracted attention due to its practical sparsity ratio and hardware-friendly pattern. However, the potential to accelerate N:M sparse deep neural network (DNN) training has not been fully exploited, and there is a lack of efficient hardware supporting N:M sparse training. To tackle these challenges, this paper presents a computation-efficient scheme for N:M sparse DNN training, called CEST. A bidirectional weight pruning method, dubbed BDWP, is firstly proposed to significantly reduce the computational cost while maintaining model accuracy. A sparse accelerator, namely SAT, is further developed to neatly support both the regular dense operations and N:M sparse operations. Experimental results show CEST significantly improves the training throughput by 1.89 - 12.49x and the energy efficiency by 1.86 - 2.76x.
更多
查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络