On the Evidence for a Learning Hierarchy in Data Structures Exams.

Several previous research studies have found a relationship between the ability of novices to trace and explain code, and the ability to write code. Harrington and Cheng refer to that relationship as the Learning Hierarchy. However, almost all of those studies examined students at the end of their first semester of learning to program (i.e. CS1). This paper is only the third paper to describe a study of explain in plain English questions on students at the end of an introductory data structures course. The preceding two papers reached contradictory conclusions. Corney et al. presented results consistent with the Learning Hierarchy identified in the CS1 studies. However, Harrington and Cheng presented results for data structures students suggesting that the hierarchy reversed by the time students had progressed to the level of learning about data structures; that is, tracing and explaining were skills that followed writing. In our study of data structures students, we present results that are consistent with the Learning Hierarchy derived from the CS1 students. We believe that the reversal identified by Harrington and Cheng can occur, but only as a consequence of a mismatch in the relative difficulty of tracing, explaining and writing questions.