MobileTutAR: a Lightweight Augmented Reality Tutorial System using Spatially Situated Human Segmentation Videos

BSTRACT Augmented reality (AR) tutorial systems have a strong potential to help workers improve learning efficiency in the ongoing trend of Industry 4.0. The current state-of-the-art point cloud approach usually requires cumbersome preparation and additional computational resources only to suffer from low-resolution visual results. To overcome the above limitations, we propose MobileTutAR, a lightweight AR tutorial system that runs entirely on mobile devices providing high definition spatially situated tutorial videos. Our approach captures the tutorial images that extract a human’s body segmentation combined with a user-defined area of interest. The system then projects the tutoring content spatially in situ, to align with the human expert’s recorded position. When playing back, the learner is guided by a navigation-centered user interface to observe the segmentation video from the recorder’s original position/orientation. In this way, we deliver a high-definition AR experience without needing any cumbersome equipment, exotic computational resources, or in-depth training.