Neural Fields for Visual Computing: SIGGRAPH 2023 Course

Neural fields---popularized by NeRFs ('neural radiance field')---seem to be everywhere in the popular press (e.g., Corridor Crew) for applications such as shape and image synthesis and human avatars. But, beyond writing research papers and fancy demos, what benefits might they bring to the broad SIGGRAPH community - to artists, game developers, or graphics engineers - through their inherent properties? Neural fields let us solve computer graphics problems by representing physical properties of scenes or objects across space and time using coordinate-based neural networks. Their key properties as continuous and compressed representations of shape and appearance are especially useful in tasks that reconstruct scenes from real-world images for content creation. These properties are so compelling that new graphics hardware architectures are being proposed to accelerate their use, and the acronym NeRF has now become generic. In sum, neural fields represent a wider inflection point within computer graphics, and people beyond researchers should know why this is. Over half a day, this course aims to provide an overview of neural fields techniques for visual computing, an understanding of the mathematical and computational properties that determine their practical uses, and examples of how we can use that understanding to solve many kinds of problems. We will identify the common components of neural field methods: their representations, their forward maps as differentiable renderers, the neural network architectures, their ability to generalize to different scenes and objects, and their ability to manipulate representations. The course features an invited industry speaker (Alex Yu of Luma AI) who will share how neural fields are used in practice, providing the audience with insights into how the latest developments can make practical tools for media production.