Multi-Modal Emotion Classification in Virtual Reality Using Reinforced Self-Training

Affective computing focuses on recognizing emotions using a combination of psychology, computer science, and biomedical engineering. With virtual reality (VR) becoming more widely accessible, affective comput-ing has become increasingly important for supporting social interactions on online virtual platforms. How-ever, accurately estimating a person's emotional state in VR is challenging because it differs from real-world conditions, such as the unavailability of facial expres-sions. This research proposes a self-training method that uses unlabeled data and a reinforcement learn-ing approach to select and label data more accurately. Experiments on a dataset of dialogues of VR players show that the proposed method achieved an accuracy of over 80% on dominance and arousal labels and out-performed previous techniques in the few-shot classi-fication of emotions based on physiological signals.