Virtually the Same Experience?: Learning from User Experience Evaluation of In-vehicle Systems in VR and in the Field

In order to investigate how a VR study context influence participants' User Experience responses of an interactive system an UX evaluation of the same in-vehicle systems was conducted in the field and in virtual reality. The virtual environment featured a virtual road scene and an interactive in-car environment, paired with a physical set-up containing a table-mounted steering wheel and a touch-sensitive panel. The VR system enabled a high estimation of presence and focus, however participants voiced less affect in the virtual setting and had difficulty in separating judgments of the VR experience from the UX of the in-vehicle systems. No significant differences in UX questionnaire data were identified between VR and the field, but there were correlations between rated presence in the VR system and UX ratings, especially for reported stimulation. Based on the lessons learned, a number of methodological and technological consequences are recommended in the paper, such as the need for more dynamic movement behaviour, improved resolution of graphics of the virtual vehicle and introducing the test leader as visually present in the virtual environment.