Synchronous Colored Petri Net Based Modeling and Video Analysis of Conversational Head-Gestures for Training Social Robots

Social robotics is the future of help for elderly care, health care, child education, and space exploration. Social robots will share a common space with humans and exhibit human-like interaction capabilities such as facial-expression analysis, speech understanding, gesture analysis and generation. Currently, gesture generation and understanding are limited to the domain of cognitive psychology because of the lack of computationally meaningful classification of gestures, limited work on integrated gesture generation, and limited integration of gesture analysis, speech understanding and gesture generation. There is limited work on training humanoids to learn and adapt new gestures by observing, analyzing, mimicking and learning. This paper focuses on training humanoids to learn human head-gestures by observation. We propose a new synchronous colored Petri net model and video analysis of head-gestures to identify combinations of synchronized head-motions, combining a sequence of head-motions to make Petri nets for each gesture, similarity-based graph matching to recognize the gestures, and generate archived gestures using a declarative model of gesture generation for human verification. This paper describes the proposed model of head-gestures, implementation methodology and video analysis algorithms to identify synchronized head-motions corresponding to nonemotional conversational head-gestures. The methodology is based upon analyzing time-stamped sequences of motion, stillness and silence on Facial feature-points based video analysis. The experimental result shows that the methodology provides a high-level of accuracy and robustness in detecting places, transitions and synchronization needed for constructing the proposed synchronous colored Petri net based model for recognizing conversational head-gestures.