Marking: Visual Grading with Highlighting Errors and Annotating Missing Bits
arxiv(2024)
摘要
In this paper, we introduce "Marking", a novel grading task that enhances
automated grading systems by performing an in-depth analysis of student
responses and providing students with visual highlights. Unlike traditional
systems that provide binary scores, "marking" identifies and categorizes
segments of the student response as correct, incorrect, or irrelevant and
detects omissions from gold answers. We introduce a new dataset meticulously
curated by Subject Matter Experts specifically for this task. We frame
"Marking" as an extension of the Natural Language Inference (NLI) task, which
is extensively explored in the field of Natural Language Processing. The gold
answer and the student response play the roles of premise and hypothesis in
NLI, respectively. We subsequently train language models to identify
entailment, contradiction, and neutrality from student response, akin to NLI,
and with the added dimension of identifying omissions from gold answers. Our
experimental setup involves the use of transformer models, specifically BERT
and RoBERTa, and an intelligent training step using the e-SNLI dataset. We
present extensive baseline results highlighting the complexity of the "Marking"
task, which sets a clear trajectory for the upcoming study. Our work not only
opens up new avenues for research in AI-powered educational assessment tools,
but also provides a valuable benchmark for the AI in education community to
engage with and improve upon in the future. The code and dataset can be found
at https://github.com/luffycodes/marking.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要