Using Real-World Reference to Improve Spoken Language Understanding

Humans understand spoken language in a continuous manner, incorporating complex semantic and contex- tual constraints at all levels of language processing on a word-by-word basis, but the standard paradigm for computational processing of language remains sentence-at-a-time, and does not demonstrate the tight integration of interpretations at various levels of pro- cessing that humans do. We introduce the fruit carts task domain, which has been specifically designed to elicit language that requires this sort of continuous un- derstanding. A system architecture that incrementally incorporates feedback from a real-world reference res- olution module into the parser is presented as a ma- jor step towards a continuous understanding system. A preliminary proof in principle shows that real-world knowledge can help resolve certain parsing ambiguities, thus improving accuracy, and that the efficiency of the parser, as measured by the number of constituents built, improves by upwards of 30% on certain example sen- tences with multiple attachment ambiguities. A 26% ef- ficiency improvement was achieved for a dialogue tran- script taken from those collected for the fruit carts task domain. We also argue that real-world reference infor- mation can help resolve ambiguities in speech recogni- tion.