Reducing the State Space of Programming Problems through Data-Driven Feature Detection

The large state space of programming problems makes providing adaptive support in intelligent tutoring systems (ITSs) difficult. Reducing the state space size could allow for more interpretable analysis of student progress as well as easier integration of data-driven support. Using data collected from a CS0 course, we present a procedure for defining a small but meaningful programming state space based on the presence or absence of features of correct solution code. We present a procedure to create these features using a panel of human experts, as well as a data-driven method to derive them automatically. We compare the expert and data-driven features, the resulting state spaces, and how students progress through them. We show that both approaches dramatically reduce the state-space compared to traditional code-states and that the data-driven features have high overlap with the expert features. We conclude by discussing how this feature-state space provides a useful platform for integrating data-driven support methods into ITSs.