Feature selection for orthogonal broad learning system based on mutual information.

The broad learning system (BLS) is a recently proposed neural network model. Different from the deep neural network model, the BLS has a flatted structure. Therefore, it can be trained efficiently and enjoyed much popularity on many applications. In this paper, we investigate the BLS from the perspective of information theory. By applying the newly proposed matrix-based Renyi's alpha-entropy, we analyze the mutual information between the feature nodes and the output, and the results indicate that the contribution of different nodes vary from each other. Therefore, we propose a feature selection algorithm for orthogonal broad learning system based on mutual information. In the proposed algorithm, the mapping weights are chosen from a orthogonal base set to insure that the extracted features are independent to each other. In addition, each node is valued based on the mutual information between the feature and the output, and a feature selection algorithm is given. We conducted extensive experiments on challenging datasets of classification and regression to demonstrate the effectiveness of the proposed algorithm.