WonderWorld: Interactive 3D Scene Generation from a Single Image

We present WonderWorld, a novel framework for interactive 3D sceneextrapolation that enables users to explore and shape virtual environmentsbased on a single input image and user-specified text. While significantimprovements have been made to the visual quality of scene generation, existingmethods are run offline, taking tens of minutes to hours to generate a scene.By leveraging Fast Gaussian Surfels and a guided diffusion-based depthestimation method, WonderWorld generates geometrically consistent extrapolationwhile significantly reducing computational time. Our framework generatesconnected and diverse 3D scenes in less than 10 seconds on a single A6000 GPU,enabling real-time user interaction and exploration. We demonstrate thepotential of WonderWorld for applications in virtual reality, gaming, andcreative design, where users can quickly generate and navigate immersive,potentially infinite virtual worlds from a single image. Our approachrepresents a significant advancement in interactive 3D scene generation,opening up new possibilities for user-driven content creation and explorationin virtual environments. We will release full code and software forreproducibility. Project website: https://WonderWorld-2024.github.io/