Dynamic ensemble pruning algorithms fusing meta-learning with heuristic parameter optimization for time series prediction

The ensemble learning paradigm demonstrates powerful advantages in solving Time Series Prediction (TSP) problems. But static ensemble methods do not always play a positive role in predictive performance. Dynamic Ensemble Pruning (DEP) methods aim to select an optimal subensemble for each test sample. By fusing the Meta-learning paradigm with the Heuristic optimization algorithm, this paper develops a hybrid Dynamic Ensemble Pruning (HeuMetaDEP) framework for TSP. The MetaDEP subalgorithm designed based on Meta-Learning (ML) trains a meta-predictor, which can judge whether each base leaner in the ensemble has enough ability to predict the query sample, to dynamically acquire the proper subensemble for each sample. Based on different aspects of the generalization ability of the base learner, seven sets of meta-characteristics are constructed for the training of the meta-predictor. And the Heuristic Optimization Algorithm (HOA) is used to filter the meta-characteristics and select the optimal parameters for a specific TSP problem. In this paper, we apply three HOAs, including the Genetic Algorithm (GA), Particle Swarm Optimization (PSO) algorithm, and Artificial Fish Swarm Algorithm (AFSA) to form a set of three specific algorithms, collectively abbreviated as the HeuMetaDEP algorithms. An experimental evaluation conducted on eight benchmark time series datasets shows that the proposed HeuMe-taDEP algorithms are significantly superior on more than half of the datasets, and achieve an average RMSE drop of 12.49% and an average MAE drop of 12.73% on the eight datasets, compared with the five naive algorithms. In addition, the three groups of extended experiments demonstrate that the proposed algorithms have better generalization performance compared with other advanced DEP algorithms, except under the small-sample condition.