# A sub-sampled tensor method for nonconvex optimization

IMA JOURNAL OF NUMERICAL ANALYSIS（2023）

Abstract

A significant theoretical advantage of high-order optimization methods is their superior convergence guarantees. For instance, third-order regularized methods reach an (epsilon(l), epsilon(2), epsilon(3))third-order critical point in at most 0(max(epsilon(4/3)(1), epsilon(-2)(2), epsilon(-4)(3))) iterations. However, the cost of computing high-order derivatives is prohibitively expensive in real applications, including for instance many real-world machine learning tasks. In order to address this problem, we present a sub-sampled optimization method that uses a thirdorder regularized model to find local minima of smooth and potentially nonconvex objective functions with a finite-sum structure. This algorithm uses sub-sampled derivatives instead of exact quantities and is guaranteed to converge to a third-order critical point. Our analysis relies on a novel tensor concentration inequality for sums of tensors of any order that makes explicit use of the finite-sum structure of the objective function.

MoreTranslated text

Key words

nonlinear optimization,nonconvex optimization,unconstrained optimization,cubic regularization,trust-region methods

AI Read Science

Must-Reading Tree

Example

Generate MRT to find the research sequence of this paper

Chat Paper

Summary is being generated by the instructions you defined