TagEz: Flickr Tag Recommendation

User tagging of multimedia has emerged as the pre- mier organizational tool for large sets of rapidly grow- ing information. We present a tag prediction system for images on Flickr which combines both linguistic and vision features. We describe methods for building language models of tags on Flickr, similar in spirit to traditional language modeling in the NLP community. We evaluate our system against held-out Flickr data, and achieve competitive performance. Secondly, we describe a Greasemonkey script which a user can download that seamlessly adds our tag predic- tion system to the normal Flickr tag interface. Using AJAX technology, this allows us to asynchronously run our image analysis (which takes on average less than 2 seconds per novel image), returning our list in near real-time to the user. Lastly, we present an empirical evaluation of the TagEz system using standard Information Retrieval metrics. These results show that the language compo- nent outperforms the vision component, and that their combination actually underperforms just the language component. We discuss a method for using held out data in a lower bound evaluation to avoid the labors of manual annotation.