Dynamics Of On-Line Gradient Descent Learning For Multilayer Neural Networks

We consider the problem of on-line gradient descent learning forgeneral two-layer neural networks. An analytic solution is presentedand used to investigate the role of the learning rate in controllingthe evolution and convergence of the learning process.Learning in layered neural networks refers to the modification of internal parametersfJg which specify the strength of the interneuron couplings, so as to bring the mapfJimplemented by the network as close as possible to a desired...