Pointflownet: Learning Representations For Rigid Motion Estimation From Point Clouds

Despite significantprogress in image-based 3D scene flow estimation, the performance of such approaches has not yet reached the fidelity requiredby many applications. Simultaneously, these applicationsare often not restrictedto image-basedestimation: laserscannersprovide a popular alternativeto traditionalcameras,for example in the context of self-driving cars, as they directly yield a 3D point cloud. In this paper we propose to estimate 3D motionfrom such unstructuredpoint clouds using a deep neural network. In a singleforwardpass, our model jointly predicts 3D scene flow as well as the 3D bounding box and rigid body motion of objects in the scene. While the prospectof estimating 3D scene flow from unstructuredpoint clouds ispromising, it is also a challenging task. We show that the traditionalglobal representationof rigid body motion prohibitsinference by CNNs, andpropose a translationequivariantrepresentation to circumvent this problem. For trainingour deep network, a large dataset is required.Because of this, we augment real scansfrom KITJI with virtualobjects, realisticallymodeling occlusions and simulating sensor noise. A thorough comparison with classic and learning-basedtechniques highlights the robustnessof the proposedapproach.