The student becomes the master: Outperforming GPT3 on Scientific Factual Error Correction.
EMNLP 2023(2023)
摘要
Due to the prohibitively high cost of creating
error correction datasets, most Factual Claim
Correction methods rely on a powerful verification model to guide the correction process.
This leads to a significant drop in performance
in domains like Scientific Claim Correction,
where good verification models do not always
exist. In this work we introduce SciFix, a
claim correction system that does not require
a verifier but is able to outperform existing
methods by a considerable margin — achieving correction accuracy of 84% on the SciFact
dataset, 77% on SciFact-Open and 72.75% on
the CovidFact dataset, compared to next best
accuracies of 7.6%, 5% and 15% on the same
datasets respectively. Our method leverages the
power of prompting with LLMs during training to create a richly annotated dataset that can
be used for fully supervised training and regularization. We additionally use a claim-aware
decoding procedure to improve the quality of
corrected claims. Our method outperforms the
very LLM that was used to generate the annotated dataset — with FewShot Prompting on
GPT3.5 achieving 58%, 61% and 64% on the
respective datasets, a consistently lower correction accuracy, despite using nearly 800 times
as many parameters as our model.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要