Online Adaptive Integration of Observation and Inpainting for Diminished Reality with Online Surface Reconstruction

Diminished reality (DR) is one of the frameworks in augmented reality, which visually makes obstructive objects transparent and shows their background. The techniques in DR can be divided into observation-based and inpainting-based methods. The observation-based method completes the region of interest (ROI) with the color and shape observed as the camera moves, while the inpainting-based method completes the region by inferring it from the surrounding pixels. In this paper, we propose a method that adaptively integrates the advantages of both methods with online surface reconstruction. The inpainting-based method completes the ROI at the beginning, while the observation-based method gradually completes the re-gion using the reconstruction with RGB-D SLAM as the camera moves. In this way, our proposed method adaptively utilizes both the inpainting-based and observation-based methods according to the camera movement online. The evaluation with various scenes was conducted to demonstrate the effectiveness of our proposed integration method by comparing it with each method.