Detecting incongruent news headlines with auxiliary textual information

News readers are left with their initial impression gained from its headline, whereas a headline’s purpose is to attract the reader’s attention to the news content. Incongruent news headlines can easily mislead readers with their clickbait headlines, which have become pervasive in online. Thus, considerable attention has been paid to detect incongruent news headlines before they reach to the readers, however there is still a lack of large-scale dataset which is restricted to title and body text. Accordingly, in this study, we released Incongruent News Headline Dataset, which has been collected and written by one of the largest news media outlets in South Korea. The generated dataset contains additional textual information, such as subtitles, image captions, and other auxiliary information. We proposed a method that effectively detects incongruent news headlines by capturing the complex lexical and contextual textual relationships between a headline and its body using an attention mechanism. The proposed model outperforms the existing models, and we investigated how it fully utilizes all the possible features.