Switchable CNN-Based Same-Resolution and Super-Resolution In-Loop Restoration for Next Generation Video Codecs.

We present a common framework for in-loop same-resolution and super-resolution restoration for incorporation into a next-generation video codec. Building on the in-loop filtering pipeline in the AV1 video codec from the Alliance for Open Media (AOM), we first enhance it to support symmetric spatial down-up scaling with better down and upscaling filters, followed by adding switchable frame-level CNNs to restore the reconstructed frames. Furthermore, the architectures for the CNNs used are constrained to be very simple with a relatively small number of parameters and multiply-add operations per decoded pixel to make them practically feasible in hardware. Preliminary results are presented on test sets being used by AOM for their ongoing next-generation video codec (AV2) development effort.