3D convolution network and Siamese-attention mechanism for expression recognition

Researches on Expression recognition focus on common subject-independent task, while cross-database evaluation is rare and lack of universal protocol. The key challenge for both tasks is to extract features that effectively describe the pattern of expression. In this paper, we present a variable length 3D convolution network that is able to output variable length features. Additionally, we proposed a Siamese 3D convolution network that utilize the“neutral, intermediate, peak” frames from another subject to provide attention weights for the extracted features. Furthermore, we proposed a method to extract fixed length landmark features from expression sequence as auxiliary for convolution network. At last, we try to recommend a universal protocol for cross-database evaluation. Experiments on both subject-independent task and cross-database evaluation show that our network not only achieves comprehensive better performance than previous methods, but also have better generalization ability due to the attention mechanism.