AlayaVista: Streaming World Modeling from Panoramic States to Perspective Video
AlayaVista is a camera-controllable streaming video world model that decouples panoramic scene evolution from perspective synthesis, trained on a 1,318-hour 4K dataset.
AlayaVista builds a 360-degree scene prior from a single perspective image, evolves it as a camera-conditioned panoramic latent state, and maps it to perspective video via a latent viewport renderer plus a perspective refiner. Chunk-autoregressive generation and few-step distillation enable efficient streaming. The authors introduce MUGEN, a real-world panoramic video dataset with 1,318 hours of at-least-4K video and rich semantic and geometric annotations.
- Decouples panoramic world evolution from perspective observation synthesis for low-latency streaming.
- Panorama expansion model plus camera-conditioned latent state; viewport renderer and refiner produce output video.
- Chunk-autoregressive generation and distillation into few-step processes support streaming.
- MUGEN dataset: 1,318 hours of 4K+ panoramic video with semantic and geometric annotations.
From the original article
1 min read at huggingface.co
Interactive video world models must maintain broad scene context under camera motion while producing high-fidelity observations with low latency.
By Jiaming Tan, Mingliang Zhai, Zhen Li · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.