CogEmoNet: A Cognitive-Feature-Augmented Driver Emotion Recognition Model for Smart Cockpit

Driver’s emotion recognition is vital to improving driving safety, comfort, and acceptance of intelligent vehicles. This article presents a cognitive-feature-augmented driver emotion detection method that is based on emotional cognitive process theory and deep networks. Different from the traditional methods, both the driver’s facial expression and cognitive process characteristics (age, gender, and driving age) were used as the inputs of the proposed model. Convolutional techniques were adopted to construct the model for driver’s emotion detection simultaneously considering the driver’s facial expression and cognitive process characteristics. A driver’s emotion data collection was carried out to validate the performance of the proposed method. The collected dataset consists of 40 drivers’ frontal facial videos, their cognitive process characteristics, and self-reported assessments of driver emotions. Another two deep networks were also used to compare recognition performance. The results prove that the proposed method can achieve well detection results for different databases on the discrete emotion model and dimensional emotion model, respectively.