Constructive Cascade Learning Algorithm For Fully Connected Networks

The Fully Connected Cascade Networks (FCCN) were originally proposed along with the Cascade Correlation (CasCor) learning algorithm that having three main advantages over the Multilayer Perceptron (MLP): the structure of the network could be determined dynamically; they were more powerful for complex feature representation; the training was efficient by optimizing newly added neuron only in every stage. However, at the same time, they were criticized that the freezing strategy usually resulted in an overlarge network with the architecture much deeper than necessary. To overcome the disadvantage, in this paper, a new hybrid constructive learning (HCL) algorithm is proposed to build a FCCN as compact as possible. The proposed HCL algorithm is compared with the CasCor algorithm and some other algorithms on several popular regression benchmarks.