World Observer: Joint Actor-Observer Generation for Persistent World Modeling
World Observer jointly generates an actor view and panoramic observers so video world models preserve out-of-view object dynamics.
Video world models are actor-centric and lose object state once something leaves the camera. World Observer jointly generates a perspective actor and one or more panoramic observers that keep selected regions visually evolving. Actor and observers are grounded by warping a shared panoramic source, and an Observer Sink of high-resolution references restores appearance on re-entry. A new world-space benchmark on real and synthetic scenes shows stronger out-of-view dynamics while remaining competitive on fidelity, camera control, and 3D adherence.
- Decouples acting from observing via freely placed panoramic observers.
- Shared panoramic warping gives explicit geometric correspondence.
- Observer Sink restores fine appearance when objects re-enter.
- New world-space metrics cover real and synthetic scenes.
From the original article
1 min read at huggingface.co
How can a world model continuously observe regions beyond the actor's current view? Video world models simulate how an environment evolves from an agent's actions, yet remain actor-centric. Once an object leaves the actor's view, they lose direct evidence of its evolution, often failing to preserve its state and dynamics upon re-entry.
By Hyunwook Choi, Dahyun Chung, Hyunsung Kim · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.