Human perception of haptic-to-video and haptic-to-audio skew in multimedia applications

The purpose of this research is to assess the sensitivity of humans to perceive asynchrony among media signals coming from a computer application. Particularly we examine haptic-to-video and haptic-to-audio skew. For this purpose we have designed an experimental setup, where users are exposed to a basic multimedia presentation resembling a ping-pong game. For every collision between a ball and a racket, the user is able to perceive auditory, visual, and haptic cues about the collision event. We artificially introduce negative and positive delay to the auditory and visual cues with respect to the haptic stream. We subjectively evaluate the perception of inter-stream asynchrony perceived by the users using two types of haptic devices. The statistical results of our evaluation show perception rates of around 100 ms regardless of modality and type of device.