Evaluation of Parsons Problems with Menu-Based Self-Explanation Prompts in a Mobile Python Tutor

The overarching goal of our project is to design effective learning activities for PyKinetic, a smartphone Python tutor. In this paper, we present a study using a variant of Parsons problems we designed for PyKinetic. Parsons problems contain randomized code which needs to be re-ordered to produce the desired effect. In our variant of Parsons problems, students were asked to complete the missing part(s) of some lines of code (LOCs), and rearrange the LOCs to match the problem description. In addition, we added menu-based Self-Explanation (SE) prompts. Students were asked to self-explain concepts related to incomplete LOCs they solved. Our hypotheses were: (H1) PyKinetic would be successful in supporting learning; (H2) menu-based SE prompts would result in further learning benefits; (H3) students with low prior knowledge (LP) would learn more from our Parsons problems in comparison to those with high prior knowledge (HP). We found that the participants’ scores on the post-test improved, thus showing evidence of learning in PyKinetic. The experimental group participants, who had SE prompts, showed improved learning in comparison to the control group. Further analyses revealed that LP students improved more than HP students and the improvement is even more pronounced for LP learners who self-explained. The contributions of our work are a) pedagogically-guided design of Parsons problems with SE prompts used on smartphones, b) showing that our Parsons problems are effective in supporting learning and c) our Parsons problems with SE prompts are especially effective for students with low prior knowledge.