ERBFV-Net: An Emotion Recognizer Based on Face Video for MAHNOB-HCI

Emotion recognition has attracted the attention of the research community, due to its wide application in different fields, such as medicine and autonomous driving. At present, most of the emotion recognition experiments are carried out on the datasets that actors actively express emotions, but in the passive aroused emotion experiments, the effect of these methods is not ideal, and the expressions cannot be recognized well. This paper proposes an emotion recognition network ERBFV-Net composed of Action Units and MKRBlock (Multi Kernel Res-Block). For the facial key point AUs, this method uses OpenFace to extract feature on the video, and pass the signal through the AUBTS Model (AUs-Based Time Sequential Model). For the video itself, the rectified face image is directly used, and the frame image in the video is sent to the multilayer network. Finally, the two signals are fused and classified. ERBFV-Net achieved an accuracy of 64.74% in the evaluation of the emotional arousal experimental dataset MAHNOB-HCI, classifying nine emotions. The results show that these two channels of information carry relevant information for detecting the user's aroused emotion, and their combination can improve the final detection accuracy.