Learning to Integrate an Artificial Sensory Device: From Decision Fusion to Sensory Integration

The present study examines how artificial tactile stimulation from a novel non-invasive sensory device is learned and integrated with information from another sensory system. Participants were trained to identify the direction of visual dot motion stimuli with a low, medium, and high signal-to-noise ratio. In bimodal trials, this visual direction information was paired with reliable symbolic tactile information. Over several blocks of training, discrimination performance in unimodal tactile test trials improved, indicating that participants were able to associate the visual and tactile information and thus learned the meaning of the symbolic tactile cues. Formal analysis of the results in bimodal trials showed that the information from both modalities was integrated according to two different integration policies. Initially, participants seemed to rely on a linear decision integration policy based on the metacognitive experience of confidence. In later learning phases, however, our results are consistent with a Bayesian integration policy, that is, optimal integration of sensory information. Thus, the present study demonstrates that humans are capable of learning and integrating an artificial sensory device delivering symbolic tactile information. This finding connects the field of multisensory integration research to the development of sensory substitution systems.