The Impact of Face Mask and Emotion on Automatic Speech Recognition (ASR) and Speech Emotion Recognition (SER)

Meetings are common in the workplace, and it's critical to understand employees' feelings during a meeting since emotions influence how individuals talk and behave. Furthermore, due to Covid-19, individuals are required to wear a face mask in meetings, and there is not much research on how wearing a face mask impact the underlying emotion through speech. Although past research has shown that emotions and face masks have some form of impact on ASR, respectively, as of the time of writing, there are no findings on which attribute (emotion or mask) impacts the ASR system more and contributes more significantly to the speech processing error. Experiments were conducted, which aligned with the previous research that the face masks have a relatively small impact on ASR System performance at the sentence level. In addition, emotion affects speech significantly, especially sadness. Therefore, emotion contributes more significantly to speech processing error than a face mask. The experiments proved that the augmentation technique is useful as the F1 Score improved, with the greatest improvement being a 10% increase. The experiment also proved that using a face mask does not have much impact on emotion, making SER a feasible solution for detecting the underlying emotion through speech.