The Impact of Colour Difference and Colour Codability on Reference Production.

The Impact of Colour Difference and Colour Codability on Reference Production Jette Viethen (h.a.e.viethen@uvt.nl) Martijn Goudbeek (m.b.goudbeek@uvt.nl) Emiel Krahmer (e.j.krahmer@uvt.nl) Tilburg center for Cognition and Communication (TiCC) Tilburg University The Netherlands Abstract It has often been observed that colour is a highly preferred at- tribute for use in distinguishing descriptions, that is, referring expressions with the purpose of identifying an object within a visual scene. However, most of these observations were based on visual displays containing only colours that were maximally different in hue and for which the language of experimentation possessed basic colour terms. The experiment described in this paper investigates the question whether people’s preference for colour is reduced if the colour of the target referent is similar to that of the distractors. Because colours that look similar are often also harder to distinguish linguistically, we also exam- ine the impact of the codability of the used colour values. The results of our experiment show that, while people are indeed less likely to use colour when the colours in the display are similar, this effect is entirely due to the difficulty in naming similar colours. When the colours of target and distractors are similar but can be named using different basic colour terms, no reduction in colour use was observed. Keywords: reference production, language production, colour Introduction Referring expressions are an essential part of communication. Whenever people engage in any type of discourse they use re- ferring expressions to encode the entities that they are talking or writing about. Sometimes it suffices to use a pronoun to let the addressee know what is meant, but often a distinguishing description, a noun phrase differentiating the target referent from all other visually available distractor objects, is neces- sary. The production of such distinguishing descriptions has been a central theme for researchers both in psycholinguis- tic and in computational research on reference production. One particular question of interest is which attributes should be chosen for realisation in a distinguishing description, the problem of semantic content selection. One of the most often made observations in psycholinguis- tic research regarding the choice of attributes for distinguish- ing descriptions is that people seem to favour colour over almost all other attributes when describing a target referent with the aim of identification (cf. Pechmann, 1989; Belke & Meyer, 2002; Sedivy, 2003; Brown-Schmidt & Tanenhaus, 2006; Arts, Maes, Noordman, & Jansen, 2011). This includes frequent redundant use of colour; cases in which the referring expression would be equally as distinguishing if colour was not mentioned. In some cases, people even use colour when all objects in a scene are of the same colour (Koolen, Goud- beek, & Krahmer, 2012). However, as far as we know, all of this research was based on stimulus material using prototypical primary colours with clearly defined basic colour terms. In this paper, we inves- tigate the question of whether people’s preference for using the colour attribute diminishes or remains the same when the colour values in a visual scene are more similar to each other, and when no different basic colour terms exist for them. Various researchers have argued that colour is preferred over, for example, size, in reference production, because it expresses absolute rather than relative information. In partic- ular, Pechmann (1989) found in an early eye-tracking study that people usually begin to verbalise a description before they have fully scanned the scene. He found that a third of the descriptions in his data that contained both size and colour did not follow standard word order by mentioning colour be- fore size (e.g., the blue small car). 1 He also noted that the first-mentioned attribute in overspecified descriptions was al- most always colour, which often was ultimately not useful for the task of distinguishing the target referent from the vi- sual context. He argued that both these observations might be due to the fact that colour is more easily cognisable than the other distinguishing features in his experiment because it can be perceived without having to compare the target referent to the other objects in the scene. Belke and Meyer (2002) found similar overspecification effects for colour and size as Pechmann. They additionally provided eye-tracking evidence from a same–different judge- ment task for an account which credits this effect to differ- ences in the way absolute and relative attributes are processed at a perceptual level. Based on experiments using the Stroop paradigm, Naor-Raz, Tarr, and Kersten (2003) even argued that an object’s colour is an intrinsic component of the visual representation retained in long-term memory. Another prominent source of evidence for people’s prefer- ence for colour comes from corpus studies on purpose-built collections of referring expressions. The furniture section of the TUNA Corpus is a collection of human-produced distin- guishing descriptions for furniture items differing in type, colour, size and orientation. In this corpus, colour is used redundantly more than three times as often as the other at- 1 The standard word order is in this case identical for English and Dutch, the language of Pechmann’s experiment.