Conditional Information Gain Networks
2018 24th International Conference on Pattern Recognition (ICPR)(2018)
摘要
Deep neural network models owe their representational power to the high number of learnable parameters. It is often infeasible to run these largely parametrized deep models in limited resource environments, like mobile phones. Network models employing conditional computing are able to reduce computational requirements while achieving high representational power, with their ability to model hierarchies. We propose Conditional Information Gain Networks, which allow the feed forward deep neural networks to execute conditionally, skipping parts of the model based on the sample and the decision mechanisms inserted in the architecture. These decision mechanisms are trained using cost functions based on differentiable Information Gain, inspired by the training procedures of decision trees. These information gain based decision mechanisms are differentiable and can be trained end-to-end using a unified framework with a general cost function, covering both classification and decision losses. We test the effectiveness of the proposed method on MNIST and recently introduced Fashion MNIST datasets and show that our information gain based conditional execution approach can achieve better or comparable classification results using significantly fewer parameters, compared to standard convolutional neural network baselines.
更多查看译文
关键词
neural network models,conditional computing,model hierarchies,feed forward deep neural networks,decision mechanisms,differentiable Information Gain,decision losses,conditional execution approach,representational power,conditional information gain networks,convolutional neural network baselines,classification loses,cost function,mobile phones
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络