Using Automatic Programming to Improve Gradient Boosting for Classification.

In this paper, we present our new and automatically tuned gradient boosting software, Classifium GB, which beats its closest competitor, H2O, for all datasets that we ran. The primary reason that we found it easy to develop Classifium GB is that we employed meta machine learning, based on evolution, to automatically program its most important parts. Gradient boosting is often the most accurate classification algorithm for tabular data and quite popular in machine learning competitions. However, its practical use has been hampered by the need to skilfully tune many hyperparameters in order to achieve the best accuracy. Classifium GB contains novel regularization methods and has automatic tuning of all regularization parameters. We show that Classifium GB gives better accuracy than another automatically tuned algorithm, H2O, and often also outperforms manually tuned algorithms such as XGBoost, LightGBM and CatBoost even if the tuning of these is done with exceptional care and uses huge computational resources. Thus, our new Classifium GB algorithm should rapidly become the preferred choice for practically any tabular dataset. It is quite easy to use and even say Random Forest or C5.0 require more skilled users. The primary disadvantage is longer run time.