A Local Search for Numerical Optimisation Based on Covariance Matrix Diagonalisation.

Pattern Search is a family of optimisation algorithms that improve upon an initial solution by performing moves along the directions of a basis of vectors. In its original definition Pattern Search moves along the directions of each variable. Amongst its advantages, the algorithm does not require any knowledge of derivatives or analytical expression of the function to optimise. However, the performance of Pattern Search is heavily problem dependent since the search directions can be very effective on some problems and lead to poor performance on others. The present article proposes a novel enhancement of Pattern Search that explores the space by using problem-dependent search directions. Some points are sampled within the basin of attraction and the diagonalisation of the covariance matrix associated with the distribution of these points is performed. The proposed Covariance Pattern Search improves upon an initial point by varying it along the directions identified by the eigenvectors of the covariance matrix.