Hierarchical Convolutional Neural Network via Hierarchical Cluster Validity Based Visual Tree Learning

In multi-category classification task, some categories have strong inter-categories similarity, while others do not. Therefore, it is unreasonable to treat all these categories equally. One possible way is to organize all categories into a hierarchical structure and train a hierarchical classifier based on it. The general convolutional neural networks (CNN) can be seen as a flat classifier on hierarchical feature representations. Therefore, it is natural to combine the hierarchical structure and deep neural networks. However, for hierarchical classification, one open issue is how to build a reasonable hierarchical structure which characterizes the inter-relations between categories. An effective approach is to utilize hierarchical clustering to build a visual tree structure, but the critical issue is how to determine the number of clusters in hierarchical clustering. In this paper, a hierarchical cluster validity index (HCVI) is developed for supporting visual tree learning. Before clustering of each level begins, we will measure the impact of different numbers of clusters on visual tree building and select the most suitable number of clusters. Based on this visual tree, a hierarchical convolutional neural network (HCNN) can be trained for achieving more discriminative capability. Our experimental results have demonstrated that the proposed hierarchical cluster validity index (HCVI) can guide the building of a more reasonable visual tree structure, so that the hierarchical convolutional neural network can achieve better results on classification accuracy.