Category-Sensitive Incremental Learning for Image-Based 3D Shape Reconstruction

Recovering the three-dimensional shape of an object from a two-dimensional image is an important research topic in computer vision. Traditional methods use stereo vision or inter-image matching to obtain geometric information about the object, but they require more than one image as input and are more demanding. Recently, the CNN-based approach enables reconstruction using only a single image. However, they rely on limited categories of objects in large-scale datasets, which leads to limitations in their scope of application. In this paper, we propose an incremental 3D reconstruction method. When new interested categories are labeled and provided, we can finetune the network to meet new needs while retaining old knowledge. To achieve these requirements, we introduce the category-wise and instance-wise contrastive loss and the energy-based classification loss. They help the network distinguish between different categories, especially when faced with new ones, and the uniqueness and variability of the predictions generated for different instances. Extensive experiments demonstrate the soundness and feasibility of our approach. We hope our work can attract further research.