Fast Incremental Method for Nonconvex Optimization
arXiv: Optimization and Control, Volume abs/1603.06159, 2016.
We analyze a fast incremental aggregated gradient method for optimizing nonconvex problems of the form $min_x sum_i f_i(x)$. Specifically, we analyze the SAGA algorithm within an Incremental First-order Oracle framework, and show that it converges to a stationary point provably faster than both gradient descent and stochastic gradient des...More
Full Text (Upload PDF)