No-reference omnidirectional video quality assessment based on generative adversarial networks

Omnidirectional video quality assessment (OVQA) helps to evaluate the viewers’ visual experience and promotes the development of omnidirectional video. The perceived quality of omnidirectional video is affected not only by the video content and distortion, but also by the viewing directions of viewer’s preference. At present, there are some quality assessment methods for omnidirectional video, but most of them are full-reference (FR). Compared with the FR method, the no-reference (NR) method becomes more difficult due to the lack of the reference video. In this paper, a NR OVQA based on generative adversarial networks (GAN) is proposed, which is composed of the reference video generator and the quality score predictor. Generally, a reference image/video is distorted by some distortion types, and each distortion type has some distortion levels. To the best of our knowledge, there are some NR methods using GAN to generate the reference images/videos for quality assessment. In these methods, the reference images/videos is generated by GAN when the distorted images/videos, which are from a distortion type but with different distortion level, are input into the GAN. In order to achieve an accurate quality assessment, the generated reference images/videos, which are from a distortion type but with different distortion level, are expected to have as similar quality as possible with each other. However, the distorted images/videos are independent for GAN, and the GAN will generate a little different reference images/videos for these distorted images/videos. This issue is not considered in the existing GAN-based methods. To solve this issue, we introduced a level loss in OVQA. For the quality score predictor, as a further contribution of this paper, the viewing direction of the omnidirectional video is incorporated to guide the quality and weight regression. The publicly available dataset is used to evaluate the proposed method. The experimental results indicate the effectiveness of the proposed method.