Real-time lens distortion algorithm on embedded GPU systems

BSTRACT The lens distortion is essential for displaying VR contents on a head-mounted display (HMD) with a distorted display surface. We propose a novel lens distortion algorithm on an embedded GPU system. To minimize the memory access overhead, we propose a compressed form of a lookup table. We also utilize the integrated memory architecture of the edge GPU system (e.g., NVIDIA’s Jetson devices) to reduce data communication overhead between host and device. As a result, our method shows up to 1.72-times higher performance than prior lookup table-based lens distortion approaches while it consumes up to 28.93% less power. Finally, our algorithm achieved real-time performance for high-resolution images on edge GPU systems (e.g., 94 FPS for 8K image on Jetson NX). These results demonstrate the benefits of our approach from the perspectives of both performance and energy.