A replica analysis of under-bagging

A sharp asymptotics of the under-bagging (UB) method, which is a popular ensemble learning method for training classifiers from an imbalanced data, is derived and used to compare with several other standard methods for learning from imbalanced data, in the scenario where a linear classifier is trained from a binary mixture data. The methods compared include the under-sampling (US) method, which trains a model using a single realization of the subsampled dataset, and the simple weighting (SW) method, which trains a model with a weighted loss on the entire data. It is shown that the performance of UB is improved by increasing the size of the majority class, even if the class imbalance can be large, especially when the size of the minority class is small. This is in contrast to US, whose performance does not change as the size of the majority class increases, and SW, whose performance decreases as the imbalance increases. These results are different from the case of the naive bagging in training generalized linear models without considering the structure of class imbalance, indicating the intrinsic difference between the ensembling and the direct regularization on the parameters.