Technical Note—Nonstationary Stochastic Optimization Under Lp,q-Variation Measures

AbstractMost existing literature in stochastic optimization assumes that the underlying cost function does not change over time. However, in practice, cost functions may be nonstationary and change along the time horizon. In “Nonstationary Stochastic Optimization Under L_{p,q}-Variation Measures,” X. Chen, Y. Wang, and Y.-X. Wang study nonstationary sequential stochastic optimization problems and consider a general L_{p,q}-variation functional to quantify the nonstationarity of underlying cost functions. The L_{p,q}-variation functional generalizes a previously considered variation constraint and captures local temporal and spatial changes of cost functions. The matching regret upper and lower bounds are provided. The regret bound shows interesting phenomena under this general variation functional, such as the curse of dimensionality, which shares a similar spirit as in nonparametric statistics.We consider a nonstationary sequential stochastic optimization problem in which the underlying cost functions change over time under a variation budget constraint. We propose an Lp,q-variation functional to quantify the change, which yields less variation for dynamic function sequences whose changes are constrained to short time periods or small subsets of input domain. Under the Lp,q-variation constraint, we derive both upper and matching lower regret bounds for smooth and strongly convex function sequences, which generalize previously published results [Besbes O, Gur Y, Zeevi A (2015) Non-stationary stochastic optimization. Oper. Res. 63(5):1227–1244]. Furthermore, we provide an upper bound for general convex function sequences with noisy gradient feedback, which matches the optimal rate as p → ∞. Our results reveal some interesting phenomena under this general variation functional, such as the curse of dimensionality of the function domain. The key technical novelties in our analysis include affinity lemmas that characterize the distance of the minimizers of two convex functions with bounded Lp difference and a cubic spline–based construction that attains matching lower bounds.