Where exactly are the difficulties in reasoning logically about code? experimentation with an online system.

CS students can typically reason about what a piece of code does on specific inputs. While this is a useful starting point, graduates must also be able to logically analyze, comprehend, and predict the behavior of their code in more general terms, no matter what the inputs are. Results of data collection and analysis from an online educational system show it can help to pinpoint the difficulties in doing this for individual students and groups, and to partition the groups in terms of their difficulties so that instructional interventions may be better targeted. Unlike traditional debugging, this online system helps reveal difficulties in reasoning in more general terms because it is equipped with a verification engine.