360-Video Navigation for 360-Multimedia Delivery Systems: Research Challenges and Opportunities

With the emergence of new 360-degree cameras, ambisonic microphones, and VR/AR display devices, more diverse multi-modal content has become available, and with it the demand for the capability of streaming 360-degree videos to enhance users? 360-multimedia experience on mobile devices such as mobile phones and head-mounted displays. The big issue for the mobile 360-multimedia delivery systems is the huge resource demand on the underlying networks and devices to deliver 360-multimedia content with high quality of experience. In this talk, we will discuss the research challenges of 360-degree video delivery systems such as the large bandwidth, low latency, users? disorientation, and cyber-sickness, and opportunities to solve these challenges including rate adaptation algorithms of tiles videos, view prediction algorithms, content navigation, enhancement of DASH streaming for 360-videos, and control of Quality of Experience (QoE) [1]. We will briefly dive into more details of the concept of navigation graphs for 360-degree videos and present the opportunity of navigation graphs to organize 360-video content that can help in viewing navigation, caching and improvements of QoE [2]. We will show how navigation graphs are serving as models for viewing behaviors in the temporal and spatial domains, and can assist with view predictions, bandwidth, and latency control. Our experimental results are encouraging [3] and support the intuition that if we can encapsulate viewing patterns of 360-degree videos into navigation graphs at multiple levels of contextual details, we will be able to stream "need-to-see" 360-content to wireless HMD devices in timely manner within bandwidth-constrained environments, and enhance viewing quality experience of 360-degree videos in augmented reality applications.