Interaction Modalities and Children's Learning in Multisensory Environments: Challenges and Trade-offs

Allowing children to engage in technologically enabled embodied interaction activities has the potential to enhance learning and play. This work leverages the capabilities provided by multisensory environments (MSEs) to address the underlying research question: What are the benefits, challenges, and trade-offs between the various interaction modalities in the context of educational MSEs for children?. To answer this question, we designed and deployed MOVES, a MSE-enabler that goes beyond the previous "hardwired" technologies and affords different interaction modalities. We conducted an in-situ field study with 175 children aged 6-10, who engaged with MOVES and the five interaction modalities. We captured children's experiences (perceptions and actual use) through action logs, data collected from the various sensors (e.g., physiological data from wristbands, skeletal data from motion sensors), and pictorial-based self-reports. The results provide the differences between the various interaction modalities and design considerations aimed at facilitating children's learning experiences within an MSE.