CNN for Facial Emotion Recognition in Online Learning Platforms to Identify Learner Engagement

Technology provides ample opportunities for students to develop academically and gain easy access to education through online learning systems, especially in pandemic situations where in-class arrangements are restricted. However, one of the most challenging aspects of online learning is being aware of and supportive of students' emotional needs. This is because emotions play a crucial role in predicting student learning levels. In a traditional in-class setup, educators can easily recognize a student's emotions and adjust their teaching methods accordingly to match the students' learning levels. However, the same is much more challenging in an online environment as the available online meeting tools do not include emotion detection as part of their product due to various factors, such as technical limitations and the complexity of accurately detecting emotions through an online interface. Therefore, this work proposes a realtime student emotion detection system in an online education environment to change the delivery methods accordingly. First, we identify the five most influential emotions in an education environment: boredom, anxiety, enjoyment, anger, and surprise, based on the analysis of the literature. Then, we propose a CNN-based facial emotion recognition system based on these five emotions. To design and test the model, we designed a dataset based on images captured in a real university online learning setup and merged it with publicly available DAiSEE and FER 2013 datasets. Finally, we evaluate the model's performance based on classification accuracy, precision, recall, and other standard metrics. The model shows 96.70% accuracy in learner categorization, which is a significant increase in performance compared to the existing literature.