Data-Driven Decision-making in DPT Curricula Part II: Course-Level Analysis

Background and Purpose. In a physical therapist entry-level education program, there is need for continuous monitoring of student progress toward degree completion and forecasts of potential roadblocks. We propose a method by which a course instructor can provide reasonable estimates of final course performance in real time, so that the student and instructor together can make data-driven decisions regarding next steps. Our primary goal was to apply this method to a course that had a high correlation of successful performance to first time pass rate on the National Physical Therapy Examination exam. Our secondary goal was to replicate this methodology in additional classes to further determine utility of this method. Method/Model. We have developed a methodology, using a simple algebraic framework, based on individual assessment grades (quizzes or tests) in any particular course, which can provide a student with a final grade prediction within two or four points to encourage conversation with the student and guide the student early in the semester. Description and Evaluation. To validate this approach, a retrospective analysis of course grades in one course across five Doctor of Physical Therapy (DPT) cohorts was performed and the technique was replicated using additional courses at the graduate and undergraduate levels. Outcomes. By Quiz 2, the final grade is predictable for 82 ± 13% of the students to within a 2-point margin and for 90 ± 9% of students to within a 4-point margin. Thus, with only 9.5% of the total grade determined and 83% of the time remaining in the semester, average prediction utility was greater than 80%. Prediction utility varied over time, and by margin, but is generally near 80% throughout the semester in the narrow margin (2 points; coefficient of variation = 0.13 ± 0.04) and greater than 90% in the wide margin (4 points; coefficient of variation = 0.05 ± 0.03). Discussion and Conclusion. We show that course performance can be predicted with high utility and with maximal time for intervention. We provide an evidence-based approach to guide the tandem investment in success, as shared between student and school. We believe that regular monitoring of course performance as described here may provide increased opportunity to intervene with remediation activities and foster better student success within a course, enhancing the probability of successful and timely program completion.