latentSplat: Autoencoding Variational Gaussians for Fast Generalizable 3D Reconstruction
arxiv(2024)
摘要
We present latentSplat, a method to predict semantic Gaussians in a 3D latent
space that can be splatted and decoded by a light-weight generative 2D
architecture. Existing methods for generalizable 3D reconstruction either do
not enable fast inference of high resolution novel views due to slow volume
rendering, or are limited to interpolation of close input views, even in
simpler settings with a single central object, where 360-degree generalization
is possible. In this work, we combine a regression-based approach with a
generative model, moving towards both of these capabilities within the same
method, trained purely on readily available real video data. The core of our
method are variational 3D Gaussians, a representation that efficiently encodes
varying uncertainty within a latent space consisting of 3D feature Gaussians.
From these Gaussians, specific instances can be sampled and rendered via
efficient Gaussian splatting and a fast, generative decoder network. We show
that latentSplat outperforms previous works in reconstruction quality and
generalization, while being fast and scalable to high-resolution data.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要