A multi-objective evolutionary algorithm for feature selection based on mutual information with a new redundancy measure.

A new feature redundancy measurement based on mutual information was proposed.A multi-objective evolutionary algorithm for feature selection was presented.Pareto optimality was used to evaluate candidate feature subsets and find compact feature subsets.Experiments showed that our algorithm could select compact and discriminative feature subsets. Feature selection is an important task in data mining and pattern recognition, especially for high-dimensional data. It aims to select a compact feature subset with the maximal discriminative capability. The discriminability of a feature subset requires that selected features have a high relevance to class labels, whereas the compactness demands a low redundancy within the selected feature subset. This paper defines a new feature redundancy measurement capable of accurately estimating mutual information between features with respect to the target class (MIFS-CR). Based on a relevance measure and this new redundancy measure, a multi-objective evolutionary algorithm with class-dependent redundancy for feature selection (MECY-FS) is presented. The MECY-FS algorithm employs the Pareto optimality to evaluate candidate feature subsets and finds compact feature subsets with both the maximal relevance and the minimal redundancy. Experiments on benchmark datasets are conducted to validate the effectiveness of the new redundancy measure, and the MECY-FS algorithm is verified to be able to generate compact feature subsets with a high predictive capability.