VRTAB-Map: A Configurable Immersive Teleoperation Framework with Online 3D Reconstruction

The availability of consumer virtual reality peripherals has led to a growing interest in developing immersive teleoperation interfaces that feature online 3D environment reconstruction of unknown environments. However, there are many unanswered questions around how such systems are used within deployed teleoperation missions and how operators navigate and explore the 3D data presented to them. We present VRTAB-Map, a configurable immersive teleoperation framework built using the RTAB-Map SLAM library. The framework can be configured with different sensor inputs allowing for it to be used as an experimental baseline for understanding operator behaviours under different teleoperation conditions and offering a methodology for real time immersive analytics in teleoperation missions. We analyse its performance during operation and, building upon our previous work in UAV teleoperation, showcase how it can be integrated in a realised teleoperation system featuring a UAV-UGV team. We then explore opportunities for future work that can be achieved with the presented framework including understanding human factors in teleoperation missions, analysing how variable environment representation effects teleoperation performance, and investigating how immersive interface design complements the operator's mental model of complex environments.