Fast intramode decision using residual variance in AVC/H.264

In this paper, we propose a fast intramode decision method in MPEG-4 AVC/H.264 based on the intraprediction cost. The rate-distortion function in MPEG-4 AVC/H.264 exhaustively computes the distortion and bit rate of every mode type, which results in high computational complexity. The target of a good, fast mode decision method is to reduce computational complexity with minimal loss of rate-distortion performance. The proposed algorithm is based on prediction cost estimation using the variances of input pixels and those of residual values to determine the minimal number of mode types for a given macroblock. The test results show that the total encoding time is reduced by 77.4%, on average, compared to that of the JM 13.2 reference encoder. (C) 2011 Society of Photo-Optical Instrumentation Engineers (SPIE). [DOI: 10.1117/1.3646749]