Rate-Distortion Video Coding And Uncertainties: To Be Blindly Chasing Marginal Improvement Or To Be Greener

The last decade has witnessed the rapid development of video encoding and video quality assessment. However, each new generation of codecs has come with a significant increase in computational complexity, especially to exploit their full potential. To ease the exponential growth in computational load, a greener video encoding scheme that consumes less power should be considered. The improvement of encoding efficiency is driven by Rate-Distortion Optimization (RDO), where the goal is to minimize the coding distortion under a target coding rate. As distortions are quantified by quality metrics, whether the applied quality metric is capable of judging accurately the perceived quality is vital for selecting encoding recipes, etc. In most cases, more complex codecs are developed seeking for any enhancement of quality scores predicted by a ad hoc quality metric, e.g., 1 dB by PSNR or 1/100 of the SSIM scale. Despite some rules of thumb, whether such improvement is worth a significant increase in power consumption is questionable, as the resolution of most quality metrics with respect to human judgement accuracy is often above the measured improvement. In this work, we propose a simple model to quantify the uncertainty/resolution of a quality metric, where confidence intervals (CI) of the quality scores predicted by the metric are computed with respect to existing observed ground truth (e.g. human observer opinion). As a possible use case, if the CI of two encoding recipes overlap, the greener one could be selected. Extensive experiments have been conducted on several datasets tailored to different purposes, and the uncertainties of mainstream quality metrics are reported. Perspectives of the trade-off between complexity and efficiency of encoding techniques are provided.