Tackling the design and evaluation of a theater-based intelligent system to monitor audience experience in virtual public speaking settings

COVID-19 has brought about a sharp increase in the use of videoconferencing tools. In education, this complicates the monitoring of student experience, which is essential to perform adequate classroom management. Researchers have designed tools to aid teachers within on-site settings, but they focus only on student engagement and are not suitable for virtual environments.In this paper, we present our system’s architecture and evaluation. First, we adapted a theater-based framework to measuring audience experience beyond engagement in online settings. Secondly, we designed a proof-of-concept computer vision system and a companion video conferencing tool to automatically measure audience experience in the classroom and present near real-time feedback. We also describe the experiment we conducted to obtain a dataset to test our system and present the results. Although the predictive accuracy of our proof-of-concept system is limited, it opens several directions for future research.