Multi-modal Emotion Analysis for Chatbots

Developing chatbots that can recognize the emotions of users is a challenging problem of artificial intelligence. In order to build such a system, we need to define the emotion taxonomy to cover human-like feelings. Consequently, we need to prepare a large scale training data by using the defined emotion taxonomy. In this paper, we investigate methods of representing emotions and applying them in a deep neural network model that classifies the user’s emotion into many dimensions. We also take into account auditory signals of spoken language in addition to contextual information for classifying the emotions of users. Furthermore, we tackle the compositional negation of utterances which may cause misinterpretation of the emotion in the opposite direction. Our experiment shows that our model improves the performance of baseline models significantly.