Impact of BCI-Informed Visual Effect Adaptation in a Walking Simulator

In this paper, we explore the use of brain-computer interface (BCI)-adapted visual effects to support atmosphere in a walking simulator, and investigate its impact on player-reported immersive experience. While players were using a keyboard or joystick controller to control the basic movement of a character, their mental state was accessed by a non-invasive BCI technique called functional near-infrared spectroscopy (fNIRS) to implicitly adjust the visual effects. Specifically, when less brain activity is detected, the players' in-game vision becomes blurry and distorted, recreating the impression of losing focus. With this biological indication, we designed a BCI-controlled game, in which the vision becomes blurry and distorted when less brain activity is detected, recreating the impression of losing focus. To analyze the player's experience, we conducted a within-subjects study where participants played both a BCI-controlled and non-BCI-controlled game and completed a questionnaire after each session. We then conducted a semi-structured interview to investigate player perceptions of the impact the BCI had on their experiences. The results showed that players had slightly improved immersion in the BCI-adaptive game, with the temporal dissociation score significantly different. Players also reported the BCI-adaptive visual effects are realistic and natural, and they enjoyed using BCI as a supplemental control.