Facial Expression Recognition With Confidence Guided Refined Horizontal Pyramid Network

Facial expression recognition has become one of the most studied applications in computer vision and human-computer interaction. Part-level features constitute one the most appealing breakthrough to offer fine-grained information and have been intensively studied consequently. The main prerequisite is accurate location of facial parts with additional cues, which is turned to be another difficulty. Rather than directly locating parts, this paper proposes a confidence guided Refined Horizontal Pyramid Network (RHPN) to fully exploit various partial information of a given facial image. It can learn discriminative features with multiple facial granularities. Inconsistencies within each granularity are efficiently limited. Specifically, we first design a Horizontal Pyramid Network (HPN) for classification using the uniform partial feature representations at different horizontal pyramid scales. It successfully enhances the discriminative capabilities of various facial parts. Then, a refinement mechanism is added to HPN due to the fact that the uniform partition inevitably incurs outliers in each part and introduces unreasonable similarity between different parts. It re-assigns these outliers to the parts they are closest to, resulting in the refined parts with enhanced within-stripe consistency. Due to the lack of explicit supervisory information, we design an induced training strategy additionally for efficiently training of RHPN. For avoiding the finagle of prediction based on the separate feature vector, we prefer the confidence guided prediction as the final classification result. Experiments verify the effectiveness of our network for increasing the performance on facial expression recognition. More importantly, it surpasses the state-of-the-art by a large margin on the lab-controlled Oulu-CASIA dataset and the real world RAF-DB dataset.