Realtime Extended 3D Reconstruction from Stereo for Navigation

We report some of our experiences on Leaving Flatland, an exploratory project which studies the key chal- lenges in closing the loop on autonomous perception and action in challenging terrain. A primary objective of the project is to demonstrate the acquisition and processing of robust 3D geometric model maps from stereo data and Visual Odometry techniques. The 3D geometric model is used to infer different terrain types and construct a 3D semantic model. I. INTRODUCTION Most of the navigation systems today rely on 2D repre- sentations of the world (e.g. occupancy maps), which are solely used to plan the robot's motion. With such a flat representation, the geometry of the world is lost, which means the robot can only go around possible obstacles and not over them. To increase the mobility of the robot, we propose the creation of three-dimensional models of the environment that preserve the geometric structure of the scene and more importantly all the obstacles in it. The model is built and updated online as the robot moves, and contains additional semantic labels, by associating each part of the model to a predefined class of environment such as "stairs", "flat ground", "elevated flat ground", "ramp" or "irregular obstacle".