Correl-Net: Defect Segmentation in Old Films Using Correlation Networks

The old film restoration process involves many operations, one of which is the ability to identify defects that altered the film. This operation can be formulated as a binary segmentation problem and solved using state-of-the-art segmentation networks such as DeepLab v3+ or NAS-FPN. While being very powerful at describing the spatial characteristics of defects, these methods fail to take into account the fact that defects are also temporal anomalies. We therefore propose an architecture that builds on the correlation layer introduced in FlowNet to compensate for motion and eliminate potential false positives, features that look like defects but can be tracked over multiple images and are actually part of the scene. We also introduce a self-supervised pre-training process of the network, which precedes a fine-tuning phase to specifically adapt the detector to each film. Results show that our architecture, while being more compact and less resource-consuming than state-of-the-art methods, achieves higher precision and recall.