Storyplay Multimodal: A Research Tool For The Mutimodal Evaluation Of Serious Games

The many benefits of Serious Games have led to a growing interest in using them as educational tools. With this growing interest, researchers in the field of Serious Games have been increasingly recognizing the importance of evaluation for advancing this research field. In addition, traditional techniques used for evaluating software fall short when evaluating in-game interactions and reactions of Serious Games users, especially in dynamic settings such as using smartphones. Fortunately, these same devices make it possible to make more diverse data available to help researchers during the evaluation process. Although remote, unobtrusive playtesting is already being deployed for evaluating many mobile applications, there is still a lack of multimodal tools specifically targeting Serious Games as we will show in this paper. Thus, there is a need for supporting multimodal Serious Games evaluation in natural mobile settings with tailored frameworks. After providing a review over available tools and related literature, this paper will propose a mobile multimodal data gathering and pre-processing framework for Serious Games evaluation. It is being implemented as an extension to StoryPlay, a researcher tool for rapid prototyping of Serious Games created with the Serious Games authoring environment StoryTec. As the latter originally had no support for mobile or multimodal data, the main focus is on complementing logging with multimodal data to facilitate the linking of affective and cognitive engagement to interaction and gameplay events. The goal is supporting (reducing time, effort and expenses of and enhancing the results of) observational user-based evaluation of mobile serious games. Main requirements for the framework arise from the literature and software review and are then translated into design decisions for capturing, synchronizing, reconstructing and pre-processing multimodal game session data for efficient navigation and evaluation. Although the framework is built for (mobile) games exported using StoryTec, the concept can be generally applied for the evaluation of mobile scene-based educational games and can serve as a basis for advancing the research on Serious Games evaluation.