Answering knowledge-based visual questions via the exploration of Question Purpose

•Explore the purposes of visual questions and use the purposes for answer prediction.•Estimate the correctness of each candidate answer by the consistency between candidate answers and the Question Purpose.•Exploit the knowledge facts accordant with the Question Purpose to improve predictive results.