Statistical Learning of Facial Expressions Improves Realism of Animated Avatar Faces

A high realism of avatars is beneficial for VR experiences such as avatar-mediated communication and embodiment. Previous work, however, suggests that the usage of realistic virtual faces can lead to unexpected and undesired effects including phenomena like the uncanny valley. This work investigates the role of photographic and behavioral realism of avatars with animated facial expressions on perceived realism and congruence ratings. More specifically, we examine ratings of photographic and behavioral realism and their mismatch in differently created avatar faces. Furthermore, we utilize these avatars to investigate the effect of behavioral realism on perceived congruence between a physical person's video-recorded expressions and their imitations by the avatar. We compared two types of avatars, both with four identities which were created from the same facial photographs. The first type of avatars contains expressions that were designed by an artistic expert. The second type contains expressions that were statistically learned from a 3D facial expression database. Our results show that the avatars containing learned facial expressions were rated more photographically and behaviorally realistic and posses a lower mismatch between the two dimensions. They were also perceived more congruent to the video-recorded physical person's expressions. We discuss our findings and the potential benefit of avatars with learned facial expressions for experiences in VR and future research on enfacement.