Parameterized Animation Compression

Synthetic images can be parameterized by more than time or viewpoint. We generalize image-based rendering by exploiting texture-mapping graphics hardware to decompress ray-traced animations. The animations are parameterized by two or more arbitrary variables allowing view/lighting changes or relative motion of objects. Starting with a field of ray-traced images and a description of the shading models, camera parameters, and scene geometry, we encode the parameterized animation as a set of per-object parameterized textures. We present a novel method to infer texture maps from the ray-tracer's segmented imagery that provide the best match when applied by graphics hardware. The parameterized textures are encoded as a multidimensional Laplacian pyramid on fixed size blocks of parameter space. This scheme captures the great coherence in parameterized animations and, unlike previous work, decodes directly into texture maps that load into hardware with a few, simple image operations. We introduce adaptive dimension splitting in the Laplacian pyramid to take advantage of differences in coherence across different parameter dimensions and separate diffuse and specular lighting layers to further improve compression. We describe the run-time system and show high-quality results at compression ratios of 200 to 800 with interactive play back on current consumer graphics cards. The central problem of computer graphics is real-time rendering of physically-illuminated, dynamic environments. Though the computation needed is far beyond current capability, specialized graphics hardware that renders texture- mapped polygons continues to get cheaper and faster. We exploit this hardware to decompress animations computed and compiled offline. Our imagery exhibits the full gamut of stochastic ray tracing effects, including indirect lighting with reflections, refractions, and shadows. For synthetic scenes, the time and viewpoint parameters of the plenoptic function (Ade91,McM95) can be generalized. We are free to parameterize the radiance field based on time, position of lights or viewpoint, surface reflectance properties, object positions, or any other degrees of freedom in the scene, resulting in an arbitrary- dimensional parameterized animation. Our goal is maximum compression of the parameterized animation that maintains satisfactory quality and decodes in real time. Once the encoding is downloaded over a network, the decoder can take advantage of specialized hardware and high bandwidth to the graphics system allowing a user to explore the parameter space. High compression reduces downloading time over the network and conserves server and client storage. Our approach infers and compresses parameter-dependent texture maps for individual objects rather than combined views of the entire scene. To infer a texture map means to find one which when applied to a hardware- rendered geometric object matches the offline-rendered image. Encoding a separate texture map for each object better captures its coherence across the parameter space independently of where in the image it appears. Object silhouettes are correctly rendered from actual geometry and suffer fewer compression artifacts. In addition, the viewpoint can move from the original parameter samples without revealing geometric disocclusions. Figure 1 illustrates our system. Ray-traced images at each point in the parameter space are fed to the compiler together with the scene geometry, lighting models, and viewing parameters. The compiler targets any desired type of graphics hardware and infers texture resolution, texture domain mapping, and texture samples for each object over the parameter space to produce as good a match as possible on that hardware to the "gold-standard" images. Per- object texture maps are then compressed using a novel, multi-dimensional compression scheme. The interactive runtime consists of two parts operating simultaneously: a texture decompression engine and a traditional hardware- accelerated rendering engine.