Multimodal Teacher Dashboards: Challenges and Opportunities of Enhancing Teacher Insights Through a Case Study.

Teacher dashboards provide insights on students' progress through visualizations and scores derived from data generated during teaching and learning activities (e.g., response times and task correctness) to improve teaching. Despite the potential usefulness of enhancing teacher dashboards, and the respective teaching practices, with rich information regarding students' cognitive and affective states (e.g., cognitive load), few studies on teacher dashboards have considered such information. In this study, we drew on contemporary developments of multimodal (MM) learning analytics and designed an MM teacher dashboard with a notification system. The proposed system: 1) receives data from various sensors; 2) computes relevant cognitive and affective measurements; 3) visualizes the resulting measurements in a clean customizable interface; and 4) notifies instructors during moments of interest, so they may determine an appropriate method to support struggling students. To evaluate our MM teacher dashboard, we first collected multimodal data (MMD), performance data, and video recordings of students' interactions during an in situ study where 26 students engaged with a motion-based learning task. Then, we used our MM teacher dashboard to present the collected MMD and video recordings to 20 experienced teachers and educational researchers and collected qualitative data regarding respondents' insights on the advantages and challenges of visualizing students' MMD. Results showed that teachers found an MM teacher dashboard enhanced with a notification system, useful to complement their pedagogical practices. We offer empirically founded guidelines for design and integration of an MM teacher dashboard with notification systems, aimed to enhance teachers' understanding of students' learning states (e.g., real-time awareness of students' stress).