An HMM-based Real-time Intervention Methodology for a Social Robot Supporting Learning

To make social robots effective in education, they need to be autonomous both in terms of assessing the student's engagement state as well as intervening effectively in soft real-time when necessary. Hidden Markov Model (HMM) is an interpretable machine learning technique for modeling temporal data that is commonly used post-hoc to analyse latent learning processes. In this paper, we contribute by proposing an HMM-based intervention methodology for assessing and classifying the state of the student as either productive or unproductive in soft real-time. The system identifies and tracks states and patterns not conducive to learning, and a robot intervention is triggered whenever a too-high non-productive engagement is detected. In a pilot study with 22 children, we evaluate this methodology in terms of both 1) the effectiveness of the interventions on the students' learning gains and on behaviors found conducive to learning, and 2) the students' perception of the robotic interventions. Results suggest that the robot interventions have a positive effect on the post-test scores relative to the baseline robot, although there isn't a significant difference in the learning gains. Moreover, interventions that try to induce reflective behaviors are most effective in inducing the required learning behavior, followed by communication-inducing interventions. Lastly, students' perception of intervention usefulness does not reflect their actual effectiveness.