Fusing IMU Data into SfM for Image-Based 3D Reconstruction.

Structure-from-Motion (SfM), one of the most extensively used image-based 3D reconstruction methods in Computer Graphics and Vision, suffers from mismatching of image feature points when input images are sparse, thus leading to miscalculation of camera rotation matrices as well as subsequent reconstruction errors. To address these problems, this paper reports an improved SfM reconstruction system and proposes to suppress the miscalculation of camera rotation matrices during Bundle Adjustment (BA) by forming a rotation constraint with Internal Measurement Unit (IMU) data collected from a smartphone, consequently improving the reconstruction precision and efficiency. More specifically, being refined by the Kalman filter, the IMU data are adopted to estimate the camera rotation matrix of each image captured by the smartphone. As the camera rotation matrix calculated by the IMU data is based on the sensor coordinate system, this paper employs the Lie Algebra theory to transform the reference frame of the rotation matrix from the sensor coordinate system to the camera coordinate system.