Model Performance Prediction: A Meta-Learning Approach for Concept Drift Detection.

Data stream applications in highly dynamic environments often face concept drift problems, a phenomenon in which the statistical properties of the variables change over time, which can degrade the performance of Machine Learning models. This work presents a new model monitoring tool through the use of Meta Learning. The algorithm was conceived for data streams with concept drift and large target arrival delay. Additionally, a new set of Meta Features is proposed based on the use of unsupervised concept drift metrics. Unlike related Meta Learning approaches, a regressor was used at the meta level to predict the predictive performance of the base model. These predictions can be used to generate concept drift alerts before the arriving objects are labelled. Experimental results show that the proposed approach obtains, on average, a classification error reduction of 12.8%, when compared to the traditional Meta Learning approaches, and 38%, when compared to the baseline, the last known performance, in predicting the performance of the base model.