Stochastic online decisioning hyper-heuristic for high dimensional optimization

Most existing heuristic optimizers are found to be restricted to problems of moderate dimensionality, and their performance suffers when solving high-dimensional or large-scale optimization tasks. In this paper, we transform the high-dimensional optimization into online decision making problems and propose a stochastic online decisioning hyper-heuristic framework, by considering multi-armed bandits with temporal reward estimation as our essential backbone. The multi-armed bandit problem simulates an agent which tries to balance exploration and exploitation simultaneously. Specifically, we introduce 1) a sliding time window to assign temporal credit for differing heuristics, and 2) boltzmann exploration for balancing the exploration-exploitation tradeoff. The proposed method is well suited for real-world applications, with flexible compatibility for versatile cost definitions, easy interfaces for heuristics as well as fewer hyper-parameters for consistent generalization performance. Experimental studies on the benchmarks results verify the efficacy and significance of the proposed framework, i.e., when considering three differing heuristics, our method reported consistently competitive performance on benchmark problems with a dimensionality up to 10,000.