Think Like a World Model, Act Like a VLA: Distilling World-Model Representations into Compact Robot Policies
A 0.8B robot policy distilled from cached world-model features hits 97.9% on LIBERO without extra test-time compute.
The paper adds a feature-alignment term so a Vision-Language-Action policy matches cached features from a frozen world model without generating futures at decision time. No teacher is loaded in training after caching, the projector is discarded, and the deployed policy matches the undistilled baseline. A 0.8B student reaches 97.9 percent on LIBERO and improves RoboCasa-GR1 from 48.2 percent to 50.5 percent, with transfer to single-arm and bimanual real robots. It runs in about 32 ms and 1.86 GB on an RTX 5090, and gains persist across scale, backbone, alignment layer, and teacher.
- One feature-alignment loss uses features cached once from a frozen world model.
- The deployed policy is identical and loads no teacher at inference.
- A 0.8B student reaches 97.9 percent on LIBERO.
- RoboCasa-GR1 rises from 48.2 percent to 50.5 percent, with real-robot results.
- Inference is about 32 ms and 1.86 GB on an RTX 5090.
- Vendors
- NVIDIA
- Products
- LIBERORoboCasa-GR1
From the original article
1 min read at huggingface.co
Vision-Language-Action (VLA) models map observations to actions with no objective that accounts for how the world responds, so their robustness is bounded primarily by data coverage. World models carry precisely that missing objective and are better grounded for it, yet rolling the future forward costs seconds per decision and rules them out of the control loop. We show the two can be separated.
By Trung Dao, Sankalp Yamsani, Jaden Park · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.