Analysis of Social Performance and Action Units during Social Skills Training: Focus Group Study with Autism Spectrum Disorder and Schizophrenia (Preprint)

BACKGROUND Social communication is a crucial factor influencing human social life. Quantifying the degree of difficulty faced in social communication is necessary for understanding developmental and neurological disorders and for creating systems used in automatic symptom screening and assistive methods such as social skills training. Social skills training by human trainers is a well-established method. Previous social skills training used a modified roleplay test to evaluate human social communication skills. However, there are no widely accepted evaluation criteria or social behavioral markers to quantify social performance during social skills training. OBJECTIVE This paper has two objectives. First, we propose applying a social performance rating scale to social skills training data to measure social communication skills. We construct a Japanese version of a social performance rating scale already developed in English and French. Second, we attempt to quantify action units during social skills training to people suffering from autism spectrum disorder or schizophrenia. METHODS We use videos of interactions between trainers and adults with autism spectrum disorder (N=16) or schizophrenia (N=15) and control participants (N=19) during social skills training sessions. Two raters apply the proposed scale to annotate the collected data. We investigate the differences between roleplay tasks and participant groups (autism spectrum disorder, schizophrenia, and control). Furthermore, the intensity of action units on the OpenFace toolkit is measured in terms of mean and standard deviation during SST roleplaying. RESULTS We find significantly greater gaze scores in adults with autism spectrum disorder than in adults with schizophrenia. Differences are also found between the ratings of different tasks in the adults with schizophrenia and the control participants. Action units numbered AU6 and AU12 are significantly deactivated in people with schizophrenia compared with the control group. Moreover, AU2 is significantly activated in people with autism spectrum disorder compared with the other groups. CONCLUSIONS The results suggest that the social performance rating scale can be a useful tool for assessing social communication skills in different cultures and different pathologies when used with the modified roleplay test. Furthermore, facial expressions could provide effective social and behavioral markers to characterize psychometric properties. Possible future directions include using the social performance rating scale for assessing social behavior during interaction with a virtual agent.