A Hybrid Learning Algorithm For The Optimization Of Convolutional Neural Network

The stochastic gradient descend (SGD) is a prevalence algorithm used to optimize Convolutional Neural Network (CNN) by many researchers. However, it has several disadvantages such as occurring in local optimum and vanishing gradient problems that need to be overcome or optimized. In this paper, we propose a hybrid learning algorithm which aims to tackle the above mentioned drawbacks by integrating the methods of particle swarm optimization (PSO) and SGD. To take advantage of the excellent global search capability of PSO, we introduce the velocity update formula which is combined with the gradient descend to overcome the shortcomings. In addition, due to the cooperation of the particles, the proposed algorithm helps the convolutional neural network dampen overfitting and obtain better results. The German traffic sign recognition (GTSRB) benchmark is employed as dataset to evaluate the performance and experimental results demonstrate that proposed method outperforms the standard SGD and conjugate gradient (CG) based approaches.