Occupancy-Map-Based Rate Distortion Optimization and Partition for Video-Based Point Cloud Compression

AbstractThe video-based point cloud compression (V-PCC) is the state-of-the-art dynamic point cloud compression method developed by the Moving Pictures Experts Group (MPEG). It projects the point cloud patch by patch to its bounding box and organizes all the patches into a video to utilize the efficient video coding framework. However, the unoccupied pixels among different patches will lead to inefficiency of the video compression. First, the unoccupied pixels are treated equal to the occupied pixels in the rate distortion optimization process. However, the unoccupied pixels are useless for the reconstructed quality of the point cloud. Second, the edges of the occupied and unoccupied pixels can divide a coding unit into arbitrary shapes. Consequently, they are not well-characterized by the regular partitions supported in the video coding framework. Therefore, we propose using occupancy-map-based rate distortion optimization and partition to deal with these two problems. First, the occupancy map is used as a mask to ignore the distortions of the unoccupied pixels when calculating the rate distortion cost of a specified block. This strategy is applied to both intra prediction, inter prediction, and the sample adaptive offset to boost the performance. Second, we propose an occupancy-map-based partition to divide the occupied pixels from different patches and the unoccupied pixels into different prediction units (PUs). The motions of occupied PUs are then predicted using the auxiliary information. The unoccupied PU is finally padded using the approach that generates the geometry frames. The proposed algorithms are implemented in the V-PCC and High Efficiency Video Coding reference software. The experimental results show that the algorithms can individually contribute significant performance improvements compared with the V-PCC. Additionally, their combination can achieve even more bitrate savings than the sum of the individual algorithms.