Aerial 360-Degree Video Delivery for Immersive First Person View UAV Navigation

Adaptive transmission of conventional video from a UAV to the ground has been researched for various applications, but the research topic of 360° video transmission from a UAV for the specific application of first-person view (FPV) based navigation is still nascent. In this work, we present adaptive 360° video compression and streaming methods to optimize the perceptual quality of experience of a pilot, who navigates the UAV in real time by viewing this immersive FPV feed, which is sent wirelessly from the UAV to the pilot. This adaptation of the 360° FPV feed is performed in response to the wireless channel conditions and the pilot’s viewport, wherein each 360° frame is split into two regions of variable size, one meant to be within the pilot’s viewport and the other outside. Each region is encoded using different H. 265 quantization parameters (QP) and modulation orders. We model the scenario realistically by generating probability distributions of the variation in frame size and quality with QP, for aerial 360° videos. These models are expressed using a two-term exponential function, whose parameters are also provided. This model achieves lower prediction errors than the single-term exponential and power law functions. Simulations on a set of aerial 360-degree videos demonstrate that the adaptive approach achieves 9.73 dB (21.77 %) greater QoE than a baseline approach that utilizes throughput-based adaptive bit rate algorithm (ABR) to tune QP per GoP, and a 5G new radio adaptive modulation scheme (AMS) to tune modulation order: Additionally, we present a deep reinforcement learning approach to adapt FPV, which achieves an expected pilot QoE just 2.07 dB lower than the adaptive approach, while being significantly faster and requiring no prior knowledge of the environment.