WorldBridge Boosts Task Generalization in World Action Models
Original title: WorldBridge: Bridging Video and Action for Task Generalization in World Action Models
WorldBridge separates video and action experts to achieve state-of-the-art results on RoboTwin 2.0 and other benchmarks.
Achieving broad task generalization in embodied robot learning remains challenging. The paper studies video generalization for predicting visual evolution of new tasks and action generalization for translating those predictions. On RoboTwin 2.0, using only in-distribution supervision yields 25.07% success on out-of-distribution tasks. Adding videos without actions raises it to 55.90% and one action-labeled demo per task to 70.70%. WorldBridge uses separate video and action experts with structured attention and joint flow matching.
- 25.07% success with in-distribution supervision
- 55.90% with videos only
- 70.70% with one action demo per task
- 96.23% on RoboTwin 2.0 full data setting
- 80.9% on clean to random setting
From the original article
1 min read at huggingface.co
Achieving the broad task generalization of embodied models in physical environments remains a central challenge in robot learning. We study this challenge in World Action Models (WAMs) through two complementary capabilities: video generalization, which predicts the visual evolution of new tasks, and action generalization, which translates these predictions into executable behavior through a video-to-action bridge.
By Jun Cen, Yubo Guo, Hao Li · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.