Can World and Action Modeling Benefit Each Other under Unified Causal Training?
OneWM shows unified causal training lets world modeling and action learning improve each other.
OneWM is a unified causal video-action framework that uses actions both as conditions for world prediction and as targets for policy learning. On LIBERO, LIBERO-Plus, and RoboCasa365, learning visual consequences of specified actions improved policy performance and generalization across scene changes. Action learning also improved multi-view world-prediction fidelity under the same architecture, and real-world policy experiments supported the same result.
- OneWM trains world prediction and action generation in one causal model.
- Action-conditioned dynamics improved policies on LIBERO and RoboCasa365.
- Learning actions also improved multi-view world-prediction fidelity.
- Real-world robot experiments supported the same reciprocal benefit.
From the original article
1 min read at huggingface.co
World and action modeling are closely related, yet their reciprocal benefits under unified causal training remain unclear. We introduce OneWM, a simple framework for unified causal training that treats actions as both conditions for world prediction and targets for policy learning within a shared video-action architecture.
By Yue Su, Bokai Lin, Xinkai Wang · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.