Depth map continuous super-resolution with local implicit guidance function

Depth super-resolution (SR) is an effective approach to compensate the resolution gap between the relative low resolution of current depth sensing devices and the increasing demands of high-resolution content in many nowadays applications and future scenarios. However, most convolutional neural networks (CNNs) for depth SR are designed for particular integer SR ratios ( e.g., 2x, 4x, or 8x) or their combinations, which limits its practical applications that usually demands fractional SR ratios. This paper proposes a depth map continuous SR (DCSR) framework that is able to achieve resolution adaptation at arbitrary SR ratios. Specifically, we proposed an encoder with a dual-stream architecture. Using depth map and the guided color image as input, each stream interacts with the other by a series of adaptive resize blocks (ARBs) to retain essential details and resize stabilization across features of various resolutions. In the decoder, depth information at arbitrary missing locations is fitted as the weighted prediction of neural implicit functions (NIFs) from latent codes containing relative coordinate, target pixel size, and fused depth-color features. Extensive experiments demonstrate that the proposed DCSR method provides high quality depth SR results at arbitrary ratios, which is a promising merit for practical applications. Code will be released upon the publication of this work.