World-in-Context unifies view, action, and camera video controls
Original title: World-in-Context: Interactive Video World Modeling with In-Context Conditioning
A 5B model matches 22B EchoWM navigation at 81.5 versus 81.6 and switches views 93.8% of the time.
World-in-Context places view, action, and camera controls as explicit tokens in one self-attention space instead of separate conditioning paths. Explicit view tokens separate conflicting dynamics and enable view switching learned from single-view data. Interaction-Transport RoPE carries accumulated control transitions to preserve temporal continuity. On WBench, the 5B model matches 22B EchoWM navigation and far exceeds other 5B baselines on perspective switching.
- World-in-Context-5B scores 81.5 on navigation versus 81.6 for 22B EchoWM.
- Perspective-switching success is 93.8% versus 18.8% for existing 5B baselines.
- IT-RoPE parameterizes temporal attention with accumulated control transitions.
- Products
- World-in-ContextWBench
- AI models
- World-in-Context-5BEchoWM
From the original article
1 min read at huggingface.co
Interactive video world models require coordinated control over viewing perspectives, character actions, and camera motion. Existing methods typically leave view modes implicit in text captions, conflate action with camera motion, and inject controls through fragmented architectural pathways that capture interdependencies only indirectly.
By Harold Haodong Chen, Rui Peng, Rongjin Guo · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.