Diversity improvement in homogeneous ensemble feature selection: a case study of its impact on classification performance

Despite ensemble learning has recently been usefully applied in feature selection (FS) models, there are some issues such as the diversity and its effects on model performance that need to be more investigated. Diversity is a crucial property in the success of ensemble FS models, so that ignoring it and trying to improve only accuracy makes the model suffer from “diminishing returns”. This led us in this paper to focus on enhancing the diversity paradigm in the homogeneous ensemble feature selection problem via applying a partitioning approach named recursive balanced partitioning (RBP) that deliberately divides the instances into several different partitions. Besides, a new diversity measurement in ensemble FS and a new aggregation criterion named min-mean by taking “minimum” and “mean” criteria is proposed. Experimental results on twelve datasets illustrated that the proposed RBP efficaciously outperforms the traditional random partitioning as a baseline in terms of diversity achievement. Furthermore, examining the impact of diversity on classification accuracy through a case study revealed that the proposed partitioning approach provides more classification accuracy than the baselines; moreover, it was demonstrated that there is an almost positive relationship between diversity and accuracy. These findings can lead to further understanding of the effectiveness of diversity in an ensemble learning pattern.