DeltaWAM: Delta World Action Models for Bimanual Manipulation
DeltaWAM predicts visual deltas and robot actions, raising RoboTwin success while cutting training and inference cost.
World-action models transfer video-generator priors to robot control but waste compute predicting largely unchanged future frames. DeltaWAM jointly predicts visual deltas and actions with dense-anchor, sparse-delta, and action streams, and Streaming Delta Memory updates cached anchors from compact observed deltas. On RoboTwin, DeltaWAM with SDM lifts average success over Fast-WAM from 81.3% to 85.4% clean and from 75.8% to 83.9% under visual randomization. The three architectures cut training FLOPs by 17.78-23.77%, while SDM cuts one-step inference latency 36.57% and FLOPs 31.55%.
- Predicts visual deltas and actions instead of dense future frames
- Streaming Delta Memory updates cached anchors with compact observed deltas
- RoboTwin success rises to 85.4% clean and 83.9% under randomization
- Training FLOPs drop 17.78-23.77%; one-step inference FLOPs drop 31.55%
- Organizations
- AIGeeksGroup
From the original article
1 min read at huggingface.co
World-action models (WAMs) transfer visual and motion priors from pretrained video generators to robot control by jointly modeling visual dynamics and actions. Existing WAMs, however, predict dense future frames during training, repeatedly modeling largely unchanged content and coupling action-conditioned dynamics to nuisance appearance variations.
By Han Yan, Zishang Xiang, Haokai Jiang · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.