Adaptive Noisy Optimization

In this paper, adaptive noisy optimization on variants of the noisy sphere model is considered, i.e. optimization in which the same algorithm is able to adapt to several frameworks, including some for which no bound has never been derived. Incidentally, bounds derived by (16) for noise quickly decreasing to zero around the optimum are extended to the more general case of a positively lower-bounded noise thanks to a careful use of Bernstein bounds (using empirical estimates of the variance) instead of Chernoff-like variants.