An exploratory study of multimodal interaction modeling based on neural computation

Multimodal interaction serves an important role in human-computer interaction. In this paper we propose a multimodal interaction model based on the latest cognitive research findings. The proposed model combines two proven neural computations, and helps to reveal the enhancement or depression influence of multimodal presentation upon the corresponding interaction task performance. A set of experiments is designed and conducted within the constraints of the model, which demonstrates the observed performance enhancement and depression effects. Our exploration and the experimental results help to further solve the question about how tactile feedback signal contribute the multimodal interaction efficiency which could provide guidelines for designing the tactile feedback in multimodal interaction.