Random Oracles for Regression Ensembles.

This paper considers the use of Random Oracles in Ensembles for regression tasks. A Random Oracle model (Kuncheva and Rodriguez, 2007) consists of a pair of models and a fixed randomly created "oracle" (in the case of the Linear Random Oracle, it is a hyperplane that divides the dataset in two during training and, once the ensemble is trained, decides which model to use). They can be used as the base model for any ensemble method. Previously, they have been used for classification. Here, the use of Random Oracles for regression is studied using 61 datasets, Regression Trees as base models and several ensemble methods: Bagging, Random Subspaces, AdaBoost.R2 and Iterated Bagging. For all the considered methods and variants, ensembles with Random Oracles are better than the corresponding version without the Oracles.