360° Video Coding Based on Projection Format Adaptation and Spherical Neighboring Relationship

This paper describes a 360° video coding scheme submitted in response to the joint call for proposal on video compression for capability beyond HEVC issued by ITU-T SG16 Q.6 (VCEG) and ISO/IEC JTC1/SC29/WG11 (MPEG) in October 2017. The proposed coding scheme uses projection format adaptation and spherical neighboring relationship to improve coding efficiency. A hybrid angular cubemap projection format is used to adapt the sampling within each face. The faces are packed in an adaptive manner, and the frame packing configuration is updated at regular intervals. Geometry padding of reference photographs is used to improve inter prediction. For intra-/inter-prediction and in-loop filters, processes are modified to avoid utilizing “wrong” neighbors in the frame packed photograph. Finally, post-filtering is used to reduce artifacts at face discontinuities. Experimental results are presented to demonstrate the superior compression efficiency achieved by the proposed 360° video coding scheme. Based on the end-to-end weighted-to-spherically uniform PSNR metric, it achieves average bit rate savings of 33.9% and 13.5% over the HM and joint exploration model anchors, respectively, encoded in the padded equirectangular projection format.