Valence and Arousal Estimation In-The-Wild with Tensor Methods

While it is relatively easy and natural for humans to detect and interpret non-verbal cues, it is a hard task for computer systems. Automatic recognition of emotions has been the subject of extensive studies in the past decade, but despite the various methods that have been implemented, the problem remains challenging. In particular, most existing works focus on predicting a set discrete stereotypical emotion categories. We are instead interested in predicting continuous values of valence and arousal, which are able to model accurately a broader range of spontaneous emotions. Moreover, as opposed to much of the prior work that focused on controlled (laboratory) conditions, we are interested in analysis in naturalist (in-the-wild) conditions. To do so, we propose to leverage the structure in the data using tensor methods. In addition to preserving the structure, these have the potential to also reduce the total number of parameters in the models, thus improving the computational performance. We first consider a model with analytic solution in the form of a Tucker Tensor Regression. We then investigate a deep, gradient based method, namely Tensor Regression Networks. We perform thorough experiments on two publicly available databases, AFEW-VA and SEWA, for facial affect estimation in-the-wild, in terms of valence and arousal levels. Experimental results demonstrate that tensor-based methods successfully leverage the structure in the data and on average outperform baseline methods.