Object Oriented Structure From Motion: Can A Scribble Help?

The concept of anywhere anytime scanning of 3D objects is very appealing. One promising solution to extract structure is to rely on a monocular camera to perform, what is well-known as Structure from Motion (SfM). Despite the significant progress achieved in SfM, the structures that are obtained are still below par the quality of reconstruction obtained through laser scanning, especially when objects are kept as part of their background. This paper looks into the idea of treating points in the scene non-uniformly, in an attempt to give more weight to the objects of interest. The system presented utilizes a minimal user interaction, in the form of a scribble, to segment the pertinent objects from different views and focus the reconstruction on them, leading to what we call Object Oriented SfM (OOSfM). We test the effect of OOSfM on the reconstruction of specific objects by formulating the bundle adjustment (BA) step in three novel manners. Our proposed system is tested on several real and synthetic datasets, and results of the different formulations of BA presented are reported and compared to the conventional (vanilla) SfM pipeline results. Experiments show that keeping the background points actually improves the reconstructed objects of interest.