Human see, human do: comparing visual and motor representations of hand gestures

Our hands are the primary means for interacting with our surroundings. As such, they are supported by a plethora of relevant representations in the brain, most notable are the somatosensory and motor representations in sensorimotor cortex and visual representations within occipitotemporal cortex, respectively. Here, we compared the representational structure when observing and executing hand gestures within and across visual and sensorimotor cortices using 3T functional MRI and 8-channel electromyography in human participants (n=60). To characterize both visual and motor features of hand representation, participants performed a visuomotor task that required them to either execute a specific hand gesture (8 gestures: open, close, pinch, tripod, one finger, two finger, three fingers, four fingers) or to observe a first-person video of a biological or robotic hand perform the same gesture. First, when visualizing the univariate activity for the contrast: actions vs. observations, we found an expected preference for actions in sensorimotor cortex. However, within OTC, we found separate regions that prefer hand actions (anterior portion) and hand observation (posterior portion). Next, using representational similarity analysis, we quantified the multivariate representational structure of observed and executed hand gestures in both regions. We found that OTC has more separable representations for hand actions and observations compared to sensorimotor cortex. When quantifying just observations, we found distinct representational structure between observations within OTC only, which were similar for observations of biological or robotic hands, and limited similarity in the structure of observations between regions. Surprisingly, when quantifying just hand actions, we found similar distances between actions in OTC as sensorimotor cortex and a strong correlation when comparing the representational structure between the two regions. Collectively, these results reveal fine-grained representational structure about hands in OTC that is similar for both observation and action and suggest a systematic visuomotor organization within OTC.