SPW-Nav: A Streaming Panoramic World Model for Language-Guided Navigation
SPW-Nav streams one minute of real-time 2K 360-degree video from language movement instructions.
SPW-Nav is a streaming panoramic world model that converts movement instructions into camera motion and generates one minute of 2K 360-degree video in real time from a single panorama. Spherical rotation decoupling, pose-aligned conditioning, and a multi-term memory with a few-step generator keep long streams coherent as instructions change. The authors also build SPW-NavSet, panoramic videos with camera trajectories and verified instructions. Language-driven SPW-Nav outperforms prior panoramic generators on camera-following accuracy and video quality and supports on-the-fly instruction switching.
- Streams one minute of real-time 2K 360-degree video from one panorama
- Interprets language instructions as spherical camera motion
- Releases SPW-NavSet with trajectories and verified instructions
- Beats prior panoramic generators and supports live instruction switching
- Products
- SPW-NavSet
- AI models
- SPW-Nav
From the original article
1 min read at huggingface.co
Language-guided panoramic video generation benefits various downstream applications, such as interactive 3D scene exploration, virtual reality experiences, and embodied agent training. Existing panoramic generators follow predefined trajectories, and interactive world models act through low-level actions in perspective views.
By Yunheng Liu, Ziqi Cai, Siqi Yang · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.