Modular mechanisms for Bayesian optimization

The design of methods for Bayesian optimization involves a great number of choices that are often implicit in the overall algorithm design. In this work we argue for a modular approach to Bayesian optimization and present a Python implementation, pybo, that allows us to easily vary these choices. In particular this includes selection of the acquisition function, kernel, and hyperpriors as well as less-discussed components such as the recommendation and initialization strategies. Ultimately this approach provides us a straightforward mechanism to examine the effect of each choice both individually and in combination.