A Systematic Review of Multimodal Approaches to Online Misinformation Detection

During the COVID-19 pandemic, the spread of pandemic-related misinformation on social media has had a significantly adverse impact on society. The sources of such misinformation usually use not only well-tailored text but also eye-catching images to establish their credibility. In this paper, we present an overview of current efforts on the task of detecting online COVID-19 conspiracy theory and misinformation. We perform a review of multimedia misinformation datasets related to the topic and an exploratory study on the state-of-the-art approaches towards these tasks. These approaches fuse textual analysis with modeling of images, propagation graphs, user reputation and fact-checking to build a comprehensive multimodal understanding of online misinformation. Our analysis indicates that using modalities in addition to text has a significant improvement on the performance of detecting misinformation, and out of the modalities presented, modeling user reputation and graph with social data are the most effective approaches. We conclude that a dataset that unifies all modalities is needed, and we present several promising directions for future research.