GAE: Learning a Geometry-Native Latent Space for 3D-Consistent World Generation
GAE learns a compact geometry-native latent that improves 3D-consistent video and world generation.
The geometry-native autoencoder (GAE) reparameterizes a geometry foundation model's features into a compact latent jointly decodable to appearance, depth, cameras, and point maps. Holding the generator and training protocol fixed, replacing the latent with GAE cuts FVD by 12.7% on RealEstate10K and 23.1% on DL3DV. Camera-trajectory error is halved on RealEstate10K. The authors argue the latent space, not only the generator, determines cross-view 3D coherence and can link perception and generation.
- Latent jointly decodes to appearance, depth, cameras, and point maps.
- FVD falls 12.7% on RealEstate10K and 23.1% on DL3DV.
- Camera-trajectory error is halved on RealEstate10K.
- A conditional flow uses the shared latent for multiple generation tasks.
From the original article
1 min read at huggingface.co
We present a compact geometry-native latent space as a shared foundation for perception and generation. Visual generators can produce photorealistic frames without preserving a consistent 3D scene.
By Jiahao Lu, Minghao Yin, Wenbo Hu · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.