Subset based deep learning for RGB-D object recognition

RGB-D camera can easily record both color and depth images and previous works have proved that combining them together could dramatically improve the RGB-D based object recognition accuracy. In this paper, a new method based on a subset approach was introduced to learn higher level features from the raw data. The raw RGB and depth images were divided into several subsets according to their shapes and colors, guaranteeing that any two different objects in each subset are nearly not similar. Then a RGB-Subset-Sparse auto-encoder was trained to extract features from RGB images and a Depth-Subset-Sparse auto-encoder was trained to extract features from depth images for each subset. Then the learned features were transmitted to recursive neural networks (RNNs) to reduce the dimensionality of the features and learn robust hierarchical feature representations. The feature representations learned from RGB images and depth images were concatenated as the final features and then sent to a softmax classifier for classification. The proposed method is evaluated on three benchmark RGB-D datasets, RGB-D dataset of Lai et al., 2D3D dataset of Browatzki et al. and Aharon dataset of Aharon et al. Compared with other methods, ours achieves state-of-the-art performance on the first two datasets. Furthermore, to validate the generalization of our subset approach, we also do some extra experiments of applying the subsets approach to several previous works, these accuracies improved significantly.