Identity-invariant representation and transformer-style relation for micro-expression recognition

Micro-expression recognition (MER) is a challenging task due to the subtle changes of facial muscle movements in a short duration. These muscle movements, which are generalized as action units (AUs), have close correlations with micro-expressions (MEs). On the other hand, due to the limited and imbalanced training data, most existing MER works are apt to learn facial identities instead of MEs as the intrinsic representations. In this paper, we propose a novel MER method by identity-invariant representation learning and transformer-style relational modeling. Specifically, we propose to disentangle the identity information from the input via an adversarial training strategy. Considering the coherent relationships between AUs and MEs, we further employ AU recognition as an auxiliary task to learn AU representations with ME information captured. Moreover, we introduce a transformer to achieve MER by modeling the correlations among AUs. MER and AU recognition are jointly trained, in which the two correlated tasks can contribute to each other. Extensive experiments show that our method (i) obtains competitive performance over the state-of-the-art MER methods on the CASME II and SAMM benchmarks, and (ii) also works well for macro-expression recognition.