Resampling Techniques for Materials Informatics: Limitations in Crystal Point Groups Classification

Imbalanced data sets in materials informatics are pervasive and pose a challenge to the development of classification models. This work investigates crystal point group prediction as an example of an imbalanced classification problem in materials informatics. Multiple resampling and classification techniques were considered. The findings suggest that the most influential variable of the resampling algorithms is the one controlling the number of samples to omit (undersample) or synthetically generate (over-sample), as expected. The effect of balancing is to enhance the classification performance of the minority class at the cost of reducing the correct predictions of the majority class. Moreover, ideal balancing, where the classes are precisely balanced, is not optimum. Alternatively, partial balancing should be performed. In this study, the ideal ratio of the minority to majority class was found to be around two-thirds. The biggest improvement in the classification was for the random undersampling technique with k-nearest neighbors and random forest.