Active Representations for Language Acquisition and Use

This abstract describes recent work done within the NTL project at UC Berkeley and ICSI and is intended as a companion submission to the one entitled "Struc- tured Connnectionist Modeling of Word Learning". Our presentation will detail some recent work on de- veloping structured connectionist representations that support a range of linguistic and cognitive tasks. Our earlier work (Bailey 97, Narayanan 97) pro- duced a unified representation of verbs and the actions and events they describe. Our model was partly in- spired by results in high-level cortical motor control schemas (Steruberg et ai 78, Berustein 67), leading to refer to our verb model as x-schemas. X-schemas are parameterized routines with internal state that execute when invoked. Our computational model is based on extensions to the Petri net formalism. The most rel- evant features of Petri nets for our purposes are their ability to model events and states in a distributed sys- tem and cleanly capture sequentiality, concurrency and event-based asynchronous control. Our extensions to the basic Petri net formalism include typed arcs, hi- erarchicai control, durative transitions, parameteriza- tion, typed (individual) tokens and stochasticity. The x-schema representation has been used as an inductive bias in a system that models cross-linguistic acquisi- tion of hand action terms (Bailey 97). We have also implemented a structured connectionist version of x- schemas in SHRUTI (Shastri et al 97). The transla- tion was fairly straightforward and efficient, suggesting that x-schemas are a reasonable computational frame- work for developing and analyzing our cognitive and linguistic models. In recent work, we have extended the representation to model entire domains where the same mechanism can be used for acting and reasoning about actions in a dynamic environment. The basic idea is simple. We assume that people can execute x-schemas with respect to structures that are not linked to the body, the here and the now. In this case, x-schema actions are not carried out directly, but instead trigger simulations of what they would do in the imagined situation. We model the physical world as other x-schemas that have i/o links to the x-schema representing the planned ac- tion. There is some biological evidence to support this view (Rizzolatti et al 96, Jeannerod 97, Tanji 94) that planning, recognition and imagination share a common representational substrate. The x-schema simulation framework developed here has proven useful in modeling metaphoric reasoning about event descriptions in abstract domains such as international economics. A crucial aspect of the im- plemented model is its capability to exploit domain knowledge of spatial motion and manipulation (imple- mented as x-schema simulations) for real-time context- sensitive simulative inference. Details can be found in (Narayanan 1997). Additionally, vexing linguis- tic problems of aspectuai composition seem to lend themselves to simple analyses in terms of the context- sensitive interaction between verb-specific x-schemas and a CONTROLLER x-schema that captures impor- tant regularities in the evolution of events. The re- sulting x-schemas can be elaborated and constrained by such factors as tense, temporal modifiers, nominals and pragmatic context, providing a rich representation that supports simulative inference in language under- standing. The current status of our work on aspect can be found in a paper entitled "A Dynamic model of As- pectual Composition". Both papers can be obtained from "http://www.icsi.berkeley.edu/NTL".