L3G-World generates consistent worlds in real time
Original title: L3G-World: Real-Time Infinite World Generation with Latent 3D Gaussian Memory
Latent 3D Gaussian memory grows with scene extent, not stream length, and the authors report 9 FPS generation.
The paper introduces L3G-World, an interactive world model that keeps a Latent 3D Gaussian Memory so spatial consistency does not require memory that grows with video length. A memory updater and a video diffusion transformer alternate autoregressively; each Gaussian stores a VAE latent so the scene renders into the DiT latent space without RGB re-encoding. The authors report state-of-the-art quality and long-horizon consistency, with up to 38.6 times fewer 3D primitives, 3.2 times faster memory updates, and 359 times faster rendering than the most efficient prior 3D-memory baseline on each metric, enabling 9 FPS.
- Memory grows with newly observed scene extent rather than generated stream length.
- Gaussians store VAE latents and skip RGB re-encoding into the DiT.
- Authors report up to 38.6 times fewer primitives and 359 times faster rendering.
- They report real-time generation at 9 FPS.
From the original article
1 min read at huggingface.co
Interactive world models generate visual content conditioned on user actions, yet maintaining spatial consistency over long-horizon exploration in real time remains challenging. Existing approaches rely on either temporal context or explicit 3D memory, both of which scale poorly because memory grows with the length of the generated stream.
By Jinhyeok Choi, JoungBin Lee, Jongmin Lee · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.