Investigating the Performance of Steepest Descent, Newton-Raphson, and Fletcher-Reeves Approaches in Unconstrained Minimization Problems

Different optimization approaches are explored to solve unconstrained minimization problems numerically in this paper. A classic optimization problem i.e., the Rosenbrock function is considered a benchmark to compare the performance of multiple algorithms including steepest descent, Newton-Raphson, and Fletcher-Reeves conjugate gradient. The performance of four methods including fixed step size, variable step size, quadratic fit, and golden section search region elimination with the steepest descent algorithm is investigated also. The required number of iterations to converge the minimum set point while solving problems is considered an evaluation metric. The converging trajectory of the algorithms is presented using level curves.