Perceptual-based HEVC Intra Coding Optimization using Deep Convolution Networks

In this paper, we propose a novel perceptual-based intra coding optimization algorithm for the High Efficiency Video Coding (HEVC) using deep convolution networks (DCNs). According to the saliency map, the algorithm can intelligently adjust bit rate allocation between the salient and non-salient regions of the video. The proposed strategy mainly consists of two techniques, saliency map extraction, and intelligent bit rate allocation. First, we train a DCN model to generate the saliency map that highlights semantically salient regions. Compared with the texture-based region of interest (ROI) extraction techniques, our model is more consistent with the human visual system (HVS). Second, based on the saliency map, a modified rate-distortion optimization (RDO) method is designed to adaptively adjust bit rate allocation. As a result, the quality of the salient regions will be improved by allocating more bits while allocating fewer bit rates for the non-salient regions. The experimental results demonstrate that our approach can deal with multiple types of video to enhance the visual experience. For conventional videos, the proposed method achieves 0.64-dB PSNR improvement for the salient regions and saves 3.02% bit rate on average compared with HM16.7. Moreover, for conversational videos, the proposed method can significantly reduce the bit rate by 8.65% without dropping the quality of important regions.