How do you smile? Towards a comprehensive smile analysis system

To better understand the expression of human smile, there have been considerable studies about automatic smile detection. Despite all the research, few attention is paid to analyze a smile in a comprehensive way. In this paper, a smile analysis system is presented to detailedly measure a person's smile, which consists of three main modules: smile detection, smile intensity estimation and spontaneous versus posed (SVP) smile recognition. Firstly, our recent proposed feature, Self-Similarity of Gradients (GSS), is employed to detect smiling facial images in unconstrained scenarios. Secondly, the smile intensity is estimated in terms of different facial regions rather than merely the mouth region, which is also applied in the temporal phase segmentation of a smile. Finally, in SVP smile recognition module, a discriminative learning model (DLM) is proposed based on a local spatial-temporal feature, which devotes to obtaining most robust and discriminative patterns of interest. The first two modules are the bases of the last, preparing a deeper understanding of a smile. Experiments on benchmark databases are carried out and compared with the state-of-the-art methods respectively, which validate the advantages of our approach of SVP smile recognition. Moreover, a comprehensive analysis of human smile is given for the first time to the best of our knowledge, which could pave the way for computers that better assess the emotional states of their users and provide useful and important information in helping the research of psychology and behavior science. © 2017