Investigating the causal relationships between badges and learning outcomes in SQL-Tutor

The practice of adding game elements to non-gaming educational environments has gained much popularity. Gamification has been shown in some studies to enhance engagement, motivation and learning outcomes in technology-supported learning environments. Although gamification research has matured, there are some shortcomings such as inconsistency in applying gamification theories and frameworks and evaluating multiple game mechanics simultaneously. Moreover, there is little research on applying gamification to Intelligent Tutoring Systems (ITS). This paper investigates the causal effects of gamification on learning in SQL-Tutor, a mature ITS teaching students how to phrase queries in SQL. Having conducted a study under realistic conditions, we present a quantitative analysis of the performance of 77 undergraduate students enrolled in a database course. There are three main findings of our study: (1) gamification affects student learning by mediating the time-on-task; (2) students’ background knowledge does not influence time-on-task unless students achieve badges; and (3) students’ interest in topic (motivational construct) moderates the relationship between badges and time-on-task, but does not improve learning outcomes directly.