Recovering variations in facial albedo from low resolution images.

•We propose a face image enhancement framework which can jointly estimate facial albedo and perform face super-resolution. It is more effective to simultaneously solve these two tasks, which both aim to recover the facial albedo or texture from low quality images.•The proposed framework can be modeled as a non-convex optimization problem. We propose an efficient alternating optimization strategy which interleaves removing intrinsic facial variations and performing super resolution.•Existing albedo estimation methods can only deal with single sources of intrinsic facial image variation, such as illumination variation. In contrast, our framework can model more diverse sources of facial image variation.•Experiments demonstrate that the proposed method can also significantly improve the performance of face recognition and clustering when given very low resolution images with various facial variations.