Using Empirical Learning Curve Analysis to Inform Design in an Educational Game

Having insights into players' learning has important implications for design in an educational game. Empirical learning curve analysis is an approach from intelligent tutoring systems literature for measuring student learning within a system in terms of the skills involved. The approach can be used to evaluate how well different hypothesized models of required skills fit to actual student performance data from the game. This information can be used to highlight whether players need more practice with specific concepts, how the game's progression might be altered, and whether the game is succeeding at its educational objectives. In this paper we apply empirical learning curve analysis to Beanstalk, an educational game designed to teach young children the concept of balance. We show that the process is able to give insight into the detailed skills and concepts (or knowledge components) that players are learning, and give implications for level (re)design by highlighting a previously unforeseen shortcut strategy.