Group $\mathrm{L}_{1/2}$ Regularization for Filter Pruning of Convolutional Neural Networks

Group regularization method has been widely used for structured pruning of convolutional neural networks (CNNs). While the group lasso (GL) regularization can impose group level sparsity on redundant filters, it cannot impose strong sparsity on the redundant weights in the retained filters. This leads to the low weight sparsity of the retained filters and the inability to prune the retained filters with few redundant nonzero weights. In this paper, the group $\mathbf{L}_{\mathbf{1}/\mathbf{2}}$ regularization $(\mathbf{GL}_{\mathbf{1}/\mathbf{2}})$ is introduced into the loss function for pruning not only the redundant filters but also the redundant weights of the retained filters. The numerical experiments show that $\mathbf{GL}_{\mathbf{1}/\mathbf{2}}$ can obtain better pruning performance with comparable to even better accuracy performance than GL. In addition, the empirical analysis in terms of regularization loss shows that $\mathbf{GL}_{\mathbf{1}/\mathbf{2}}$ possesses more powerful punishment ability than GL.