TSDFFilter: content-aware communication planning for remote 3D reconstruction

We present a novel solution, TSDFFilter, for remote 3D reconstruction to relieve the high bandwidth requirement problem. Our approach is designed for scenarios where agents are used to collect data using an RGB-D camera and then transmit the information over the regular network to a high-performance server, where a global, dense, and volumetric model of a real-world scene is reconstructed. Our approach uses a content-aware communication planning framework in which agents can prune the gathered RGBD information according to the transmission policy generated by the server. To generate the transmission policy, we introduce a confidence value to estimate how much each RGB-D pixel contributes to the reconstruction quality, and present an algorithm to find the confidence value. As a result, agents can transmit less RGB-D information without blindly compromising the reconstruction quality as the key-frame method and down-sampling method do. We implement our TSDFFilter framework to achieve real-time agent-assisted 3D reconstruction. Extensive evaluations show that comparing with the key-frame and down-sampling methods, our TSDFFilter framework can reduce the bandwidth requirement by up to 36% with similar reconstruction Chamfer distance, and reduce the reconstruction Chamfer distance by up to 78% with similar bandwidth requirement.