Talking about other people: an endless range of possibilities.

Image description datasets, such as Flickr30K and MS COCO, show a high degree of variation in the ways that crowd- workers talk about the world. Although this gives us a rich and diverse collection of data to work with, it also introduces un- certainty about how the world should be described. This paper shows the extent of this uncertainty in the PEOPLE domain. We present a taxonomy of different ways to talk about other people. This taxonomy serves as a reference point to think about how other people should be described, and can be used to classify and compute statis- tics about labels applied to people.