SimForcing: Distilling Simulation Motion Priors into Real-Domain Robot World Models
SimForcing distills simulation motion into robot world models and improves LIBERO policy success.
SimForcing transfers motion knowledge from a simulation teacher into a real-domain robot world model via latent-motion distillation and multi-block simulation conditioning with condition dropout. Classifier-free guidance balances internalized motion priors against simulation latents, and the student produces both simulation conditions and real videos without a second world model at inference. On Bridge it reports the best PSNR, SSIM, LPIPS, and FVD among compared methods without external embodied pretraining, with further results on InternData-A1. Using the trained world model to initialize a vision-language-action policy improves success on LIBERO.
- Distills simulation motion priors into real-domain robot world models.
- Uses latent-motion distillation plus multi-block simulation conditioning.
- Student generates simulation conditions and real videos at inference.
- Best PSNR, SSIM, LPIPS, and FVD on Bridge without embodied pretraining.
- Initializing a VLA model with the world model improves LIBERO success.
- Products
- LIBEROBridgeInternData-A1
- AI models
- SimForcing
From the original article
1 min read at huggingface.co
Action-conditioned robot world models must respond precisely to robot trajectories while preserving realistic visual dynamics, yet learning both from heterogeneous robot videos remains challenging. Simulation offers structured motion supervision, but appearance differences hinder direct transfer, and inaccurate simulation predictions can misguide real-video generation.
By Xiaodong Wang, Tianle Li, Chuanxin Song · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.