Immersive Interaction in Digital Factory: Metaverse in Manufacturing

Digital twins and virtual reality are pivotal technologies in the context of Industry 4.0, facilitating the design, simulation, optimization, and remote interaction with production systems. These technologies also present new prospects for developing immersive and hyper-realistic digital factories within the metaverse. This paper aims to enhance collaboration and communication in a 3D virtual world, focusing on shared reality and the metaverse's integration in digital factories. Our main contribution enables users to have full-body immersive interaction with virtual 3D assets and realistic locomotion in a 3D environment, fostering flexibility in collaboration and environment monitoring/control. To achieve this, we propose a systematic methodology for designing and implementing a digital twin-based factory with an IoT infrastructure. By replicating sensors and actuators in digital twins, real-time asset management synchronized with the physical factory is realized, empowering full-body interaction. To enable full-body interaction with virtual equipment, we employ a human motion tracking system. A proof-of-concept case study, developed using Unity3D game engine, Solace PubSub+ event streaming APIs, and Awinda Xsens wearable inertial sensors as the motion tracking system, validates our proposed methodology. The results of the case study demonstrate the successful integration of digital twins, virtual reality, and the metaverse, enabling real-time monitoring and control, full-body interaction, and immersive user experiences in the virtual environment.