Image Retrieval with Textual Label Similarity Features

AbstractThis article presents a knowledge-based solution for retrieving English descriptions of images. We analyse the errors made by a baseline system that relies on term frequency, and we find that the task requires deeper semantic representation. Our solution is to perform incremental, task-driven development of an ontology. Ontological features are then applied in a machine-learning algorithm for ranking candidate image descriptions. This work demonstrates the advantage of combining knowledge-based and statistical approaches for text retrieval, and it establishes the important result that an empirically tuned task-specific ontology performs better than a domain-general resource like WordNet, even on previously unseen examples. Copyright © 2015 John Wiley & Sons, Ltd.