FlyFusion: Realtime Dynamic Scene Reconstruction Using a Flying Depth Camera

While dynamic scene reconstruction has made revolutionary progress from the earliest setup using a mass of static cameras in studio environment to the latest egocentric or hand-held moving camera based schemes, it is still restricted by the recording volume, user comfortability, human labor and expertise. In this paper, a novel solution is proposed through a real-time and robust dynamic fusion scheme using a single flying depth camera, denoted as FlyFusion. By proposing a novel topology compactness strategy for effectively regularizing the complex topology changes, and the Geometry And Motion Energy (GAME) metric for guiding the viewpoint optimization in the volumetric space, FlyFusion succeeds to enable intelligent viewpoint selection based on the immediate dynamic reconstruction result. The merit of FlyFusion lies in its concurrent robustness, efficiency, and adaptation in producing fused and denoised 3D geometry and motions of a moving target interacting with different non-rigid objects in a large space.