Robust and Efficient Claim Retrieval for Online Fact-Checking Applications

Abstract Understanding the veracity of statements is important when consuming information on the Web. Whereas fact-checking sites have provided a large corpus of already verified claims, matching a given utterance to already fact-checked claims remains a challenging task. Verified claim retrieval has been approached through a variety of different methods, among them approaches relying on supervised neural models. Whereas such models tend to perform strongly, they require significant training effort and their robustness towards unseen data distributions may vary heavily. Also, prior works demonstrate the capability of unsupervised models to provide state-of-the-art performance.In this paper, we assess established claim retrieval benchmark datasets and experimentally evaluate and compare different state-of-the-art supervised and unsupervised methods with regard to performance, but also computational effort and run time. We show that unsupervised approaches outperform supervised ones with respect to robustness. While the best state-of-the-art method relies on supervised deep neural networks, its high computational costs make it difficult to use in online fact-checking applications. The best unsupervised method reaches a similar performance and meets efficiency requirements of online application scenarios due to low hardware requirements. Our experiments verify that, due to the nature of the task and data, the choice of pre-trained language models is more important than fine-tuning and that training supervised models on the target data may not be cost-efficient in online claim retrieval applications.