Optimization Methods for a Class of Rosenbrock Functions

This paper gives an in-depth review of the most common iterative methods for unconstrained optimization using two functions that belong to a class of Rosenbrock functions as a performance test. This study covers the Steepest Gradient Descent Method, the Newton-Raphson Method, and the Fletcher-Reeves Conjugate Gradient method. In addition, four different step-size selecting methods the including fixed-step-size, variable step-size, quadratic-fit, and golden section method were considered. Due to the computational nature of solving minimization problems, testing the algorithms is an essential part of this paper. Therefore, an extensive set of numerical test results is also provided to present an insightful and a comprehensive comparison of the reviewed algorithms. This study highlights the differences and the trade-offs involved in comparing these algorithms.