Approximate Newton Methods
JOURNAL OF MACHINE LEARNING RESEARCH(2021)
摘要
Many machine learning models involve solving optimization problems. Thus, it is important to address a large-scale optimization problem in big data applications. Recently, subsampled Newton methods have emerged to attract much attention due to their efficiency at each iteration, rectified a weakness in the ordinary Newton method of suffering a high cost in each iteration while commanding a high convergence rate. Other efficient stochastic second order methods have been also proposed. However, the convergence properties of these methods are still not well understood. There are also several important gaps between the current convergence theory and the empirical performance in real applications. In this paper, we aim to fill these gaps. We propose a unifying framework to analyze both local and global convergence properties of second order methods. Accordingly, we present our theoretical results which match the empirical performance in real applications well.
更多查看译文
关键词
Approximate Newton, Stochastic Second-order, Hessian Approximation
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络