Beyond multiple choice exams: using computerized lexical analysis to understand students' conceptual reasoning in STEM disciplines

Constructed response questions - in which students must use their own language in order to explain a phenomenon - create more meaningful opportunities for instructors to identify their students' learning obstacles than multiple choice questions. However, the realities of typical large-enrollment undergraduate classes restrict the options faculty have for moving towards more learner-focused instruction. We are exploring the use of computerized lexical analysis of students' writing in large enrollment undergraduate biology and geology courses. We have created libraries that categorize student responses with 90% accuracy. These categories can be used to predict expert ratings of student responses with accuracy approaching inter-rater reliability among expert raters. These techniques also provide insight into students' use of analogical thinking, a fundamental part of scientific modeling. These techniques have potential for improving assessment practices across STEM disciplines.