Automatic Image Annotation Using Quantization Reweighting Function and Graph Neural Networks

This paper investigates the issues in image annotation, which automatically assigns appropriate tags to a given image describing its content the best. Due to the introduction of deep learning methods and the use of graph neural networks (GNNs), automatic image annotation has made significant progress in recent years. An image may have multiple tags associated with it, and a tag may appear in several images within the dataset; therefore, it is inefficient to study each tag individually. Some studies have attempted to model the dependencies between tags using vocabulary to improve the performance of automatic image annotation. However, it remains unclear how to create an appropriate vocabulary graph. We propose to construct this graph by modeling the relationship between tags. In the tag graph, edges are reweighted based on cosine similarity and a quantization function. To represent each node in the graph, we use two methods of word embedding. We then use graph neural networks to extract graph features. From the graph and image features, we obtain our output vector (set of class probabilities). The proposed approach is evaluated using precision, recall, F-1, and N+ performance measures on two public benchmark datasets (Corel5k, and ESP Game). Results of experiments show that our method is superior to current state-of-the-art methods. On Corel5k, we achieved the best performance with N+ and recall, the second-best performance with F-1. The second-best performance with N+ and precision and the best F-1 are also achieved on ESP Game.