Computational representation and analysis of emotion dynamics

Human emotions are dynamic in nature. The intensity with which they are felt changes over time, and they have a natural timescale of expression, from onset to decay. Further, emotions shade from one to another, and many feelings are built up of blends of pure emotions. In order to represent this complex reality visually, a variety of models of the space of emotions have been introduced in the psychology literature. In this paper we present a computational approach to transforming facial expressions of emotions into positions in one of these models, the activation-evaluation space. This enables the study of emotion dynamics from facial expressions, including the transitions between emotion states and changes in emotion intensity through time, based on a set of shape models for different emotions. We consider different ways to build these shape models, and then show how to represent each frame of emotion in the activation-evaluation space, and analyse sequences of emotions from video by following temporal trajectories in that space. We demonstrate the approach on a standard dataset and compare it to human annotations, with promising results.