Real-Time Video-Based Relighting Technology for Moving Object

This paper proposes a real-time relighting technology for tangible teleconference system. The images of remote users in video conference are relit by the captured environment image so that the user feels like being in the same space. This paper exploits the visual hull data for the 3D shape information of users and extracts the normal vectors of user's 3D surface. The normal vectors are calculated from polygonal mesh structure of depth information map. The environment image is generated by the mirror ball probe images which capture the ambient lighting conditions around users. The lighting sources in the environment image are optimally sampled on the basis of 3D geometric structure and surface reflection characteristics of objects. This paper implements the image-based relighting algorithms by fast computation and programming optimization. According to the simulations, the proposed algorithms implement image-based relighting for teleconference system in real-time. The proposed system enables the users to immerse video conference just as they are in the same space.