Unsupervised moiré pattern removal for recaptured screen images

Moiré patterns, mainly due to the aliasing between the grids of display devices and camera sensors, often occur in recaptured screen images (videos). The spatial variety in densities and scales makes it difficult to remove moiré patterns with hand-crafted priors. Prevalent supervised-learning based methods require huge amount of paired images, which are difficult to capture and align. This paper proposes an unsupervised Generative Adversarial Network for Moiré Removal (MR-GAN), which is the first attempt at unsupervised learning based moiré removal. The recaptured images are corrected against vignetting artifacts to make MR-GAN focus on correcting brightness and contrast distortions while removing moiré patterns. We propose two complementary discriminator groups to effectively distinguish moiré patterns and image features at both large scales and small scales. A group of self-supervised loss functions, including cycle consistent loss, identity loss, cosine similarity loss, and content leakage loss, are designed to train effective generators. Experimental results demonstrate that our method outperforms state-of-the-art demoiréing methods on a large set of test images. Our code and dataset are released in the link: https://github.com/JerryLeolfl/pytorch-MRGAN-master.