Applying Item Response Theory to Evaluate Instruments of Introductory Programming Skills Measurement

This Research-to-practice Full Paper presents an exploratory and preliminary investigation on the reliability and validity of instruments for measuring introductory programming skills. Our data set consists of the performance of 30 students who participated in the experiment of a Brazilian university. We provide participants with instructional material, practical problems and solutions based on different experimental conditions. The results suggest that the instruments have a good internal consistency index and items with excellent psychometric properties. In addition, some initial evidence to suggest practicing each of the four programming skills. We found that participants who received practice in all four skills obtained a better estimate of study material and Assessment, especially for the most advanced knowledge of programming introduction.