Traditional And Hybrid Derivative-Free Optimization Approaches For Black Box Functions

Picking a suitable optimization solver for any optimization problem is quite challenging and has been the subject of many studies and much debate. This is due in part to each solver having its own inherent strengths and weaknesses. For example, one approach may be global but have slow local convergence properties, while another may have fast local convergence but is unable to globally search the entire feasible region. In order to take advantage of the benefits of more than one solver and to overcome any shortcomings, two or more methods may be combined, forming a hybrid. Hybrid optimization is a popular approach in the combinatorial optimization community, where metaheuristics (such as genetic algorithms, tabu search, ant colony, variable neighborhood search, etc.) are combined to improve robustness and blend the distinct strengths of different approaches. More recently, metaheuristics have been combined with deterministic methods to form hybrids that simultaneously perform global and local searches. In this Chapter, we will examine the hybridization of derivative-free methods to address black box, simulation-based optimization problems. In these applications, the optimization is guided solely by function values (i.e. not by derivative information), and the function values require the output of a computational model. Specifically, we will focus on improving derivative-free sampling methods through hybridization. We will review derivative-free optimization methods, discuss possible hybrids, describe intelligent hybrid approaches that properly utilize both methods, and give an examples of the successful application of hybrid optimization to a problem from the hydrological sciences.