Ensuring Reliability of Educational Data Mining Detectors for Diverse Populations of Learners

In recent years, classroom use of educational software for tutoring and assessment has increased considerably. Aside from the benefits to learners, these systems provide extensive data on student learning processes. In recent years, researchers in educational data mining (EDM) have utilized this data in order to develop models of a range of constructs which have been historically difficult to assess at scale, including learner engagement, emotion, knowledge, and preparation for future learning. These models, sometimes termed “automated detectors,” can be embedded into adaptive educational software or used to analyze the contexts where learning and engagement are enhanced.Using these models creates the opportunity to better understand and, in turn, improve learning and engagement, which may particularly benefit underserved populations for whom the current generation of learning interventions are less effective. However, this will only be true if the models are accurate for the populations for who use the software. Unfortunately, insufficient attention has been placed on the reliability of these models when applied to populations–eg populations in different countries or among different demographics within-country–that are culturally distinct from those which the model was trained upon.