RynnWorld-Latent conditions a world model on universal latent actions
Original title: RynnWorld-Latent: A Cross-Embodiment World Model Conditioned on Universal Latent Actions
Pretraining spans 23 datasets and about 553.5 million frames, then adapts to target robots without a shared action space.
RynnWorld-Latent is a cross-embodiment video world model that conditions on 3D-grounded latent actions from RynnLAM so human videos, robot demonstrations, and simulation can be pretrained together. It adds global action tokens and an identity-initialized local pathway on a pretrained video transformer. Digital teleoperation and latent-action synthesis both improve real-world policy success over real-demonstration-only training in the evaluated settings, and synthetic mid-training helps on LIBERO and LIBERO-Plus.
Unconfirmed: Benchmark and robot results are the authors' reported experiments, not independently verified.
- Pretraining uses approximately 553.5 million frames from 23 datasets.
- Video prediction improves on public benchmarks and two real-robot platforms.
- Both generation routes beat real-demonstration-only training in the evaluated settings.
- Synthetic mid-training raises policy success on LIBERO and LIBERO-Plus.
- Products
- RynnLAMLIBEROLIBERO-Plus
- AI models
- RynnWorld-Latent
From the original article
1 min read at huggingface.co
Learning controllable world dynamics from heterogeneous interaction videos is challenging because robot embodiments use incompatible action spaces, while human videos lack native control labels. We present RynnWorld-Latent, a cross-embodiment video world model that uses the 3D-grounded latent actions introduced by RynnLAM as a shared conditioning interface.
By Haoyu Zhao, Yinzhou Tang, Zixuan Wang · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.