Design and User Evaluation of Haptic Augmented Virtuality System for Immersive Virtual Training

This paper presents the design and user evaluation of a haptic augmented virtuality system (HAVS) to provide an immersive virtual reality (VR) training in a plant facility. We designed an encounter-type haptic display system to enable a realistic interaction between the trainee and a diverse variety of tools such as levers, wheels, and buttons of various shapes and sizes in several places, with arbitrary poses. The trainee wears a fully immersive head-mounted display (HMD) and reaches out to operate (press, or grasp and rotate) a tool that is virtually visible. A robot manipulator promptly fetches the designated real tool from a pallet system in front of the trainee for them to operate. A precise spatiotemporal collocation of virtual and real tools is necessary for such an immersive experience. Our study presents two user evaluations on the effectiveness of a previously proposed collocation method for two tasks: button-pressing and wheel-grasping. The first user evaluation showed that the temporal reaction time of the trainees in the virtual environment was approximately 1 s slower than that in the real environment. However, a subjective evaluation showed that this time difference is not considerably affected when reaching, pressing, or grasping real tools, primarily because of the similarity of positional accuracy in both the environments that were compared. The second user evaluation measured the impact of enactment on memory encoding through a virtual environment using the proposed system and to identify the training effect of the proposed HAVS. The results showed that the enactment effect was weaker in the early phase of retrieval; however, the meta memory was more accurate when the enactment was performed using the HAVS, compared with the control conditions. Therefore, the proposed HAVS may be a cost-effectively and risk-free solution in some plant operations or safety training.