A Main/Subsidiary Network Framework for Simplifying Binary Neural Network

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)(2018)

引用 34|浏览94
暂无评分
摘要
To reduce memory footprint and run-time latency, techniques such as neural network pruning and binarization have been explored separately. However, it is unclear how to combine the best of the two worlds to get extremely small and efficient models. In this paper, we, for the first time, define the filter-level pruning problem for binary neural networks, which cannot be solved by simply migrating existing structural pruning methods for full-precision models. A novel learning-based approach is proposed to prune filters in our main/subsidiary network framework, where the main network is responsible for learning representative features to optimize the prediction performance, and the subsidiary component works as a filter selector on the main network. To avoid gradient mismatch when training the subsidiary component, we propose a layer-wise and bottom-up scheme. We also provide the theoretical and experimental comparison between our learning-based and greedy rule-based methods. Finally, we empirically demonstrate the effectiveness of our approach applied on several binary models, including binarized NIN, VGG-11, and ResNet-18, on various image classification datasets.
更多
查看译文
关键词
Deep Learning,Deep Learning
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络