Point Target Detection for Multimodal Communication.

International Conference on Human-Computer Interaction(2024)

Cited 0|Views0
No score
The future of multimodal communication between humans and AIs will rest on AI’s ability to recognize and interpret non-linguistic cues, such as gestures. In the context of shared collaborative tasks, a central gesture is deixis, or pointing, used to indicate objects and referents in context. In this paper, we extend our previously-developed methods for gesture recognition and apply them to a collaborative task dataset where objects are frequently indicated using deixis. We apply gesture detection to deictic gestures in the task context and use a “pointing frustum” to retrieve objects that are the likely targets of deixis. We perform a series of experiments to assess both the quality of gesture detection and optimal values for the radii of the conical frustum, and discuss the application of target detection using pointing to multimodal collaborative tasks between humans and computers.
Translated text
AI Read Science
Must-Reading Tree
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined