Real-time mobile video communications

Among numerous emerging video services, real-time video communication on personal portable computing units (such as personal digital assistants (PDAs), cell phones, etc.) attracts a surge of research interest. The fundamental challenge for the wireless real-time video service is to access the quality of service (QoS) sensitive video content over a network with little QoS provision using the handheld device featuring thin computing resource and limited battery energy. Specifically, one of the most generic problems is to guarantee that the compressed video data survive even the most hostile wireless environment. In addition, the characteristics of the application should be considered to achieve the system optimization. In this dissertation, a robust and practical real-time video system is outlined by proposing a set of application layer techniques not dependent on the underlying layers. First, a novel error resilient technique, namely progressive group of picture (PGOP), is presented to replace the intra-frame (I-frame) in the low-bandwidth error prone environment. We discuss its advantages over the I-frame based group of picture (GOP) structure in varied networking scenarios, including non-feedback and feedback based networks. For the non-feedback network, we emphasize the performance gain of PGOP under different encoding modes—such as single-layer and multiple-layer coding. For the feedback-based network, we fit PGOP into an end-to-end quality monitoring system and optimally adapt PGOP to achieve high perceptual decoding quality. We also propose schemes to solve the paradox between the thin resource of the handheld device and the heavy system load necessitated by the applications. In particular, we propose an interactive graphics object streaming architecture, by which the computation intensive part—the graphics rendering—is moved from the client side to the server side. The video coding speed is boosted by replacing the conventional motion picture expert group (MPEG) standardized motion search with a direct motion vector mapping algorithm. To reduce the energy consumed by the video playback without profoundly jeopardizing the video quality, we analyze the characteristics of the display unit—Liquid Crystal Display (LCD), and propose a quality adapted backlight scaling strategy (QABS).