DeltaWAM: Delta World Action Models for Bimanual Manipulation
DeltaWAM predicts visual deltas and robot actions, raising RoboTwin success while cutting training and inference cost.
World-action models transfer video-generator priors to robot control but waste compute predicting largely unchanged future frames. DeltaWAM jointly predicts visual deltas and actions with dense-anchor, sparse-delta, and action streams, and Streaming Delta Memory updates cached anchors from compact observed deltas. On RoboTwin, DeltaWAM with SDM lifts average success over Fast-WAM from 81.3% to 85.4% clean and from 75.8% to 83.9% under visual randomization. The three architectures cut training FLOPs by 17.78-23.77%, while SDM cuts one-step inference latency 36.57% and FLOPs 31.55%.
- Predicts visual deltas and actions instead of dense future frames
- Streaming Delta Memory updates cached anchors with compact observed deltas
- RoboTwin success rises to 85.4% clean and 83.9% under randomization
- Training FLOPs drop 17.78-23.77%; one-step inference FLOPs drop 31.55%
Full article173 words · extracted from huggingface.co · click to collapse
World-action models (WAMs) transfer visual and motion priors from pretrained video generators to robot control by jointly modeling visual dynamics and actions. Existing WAMs, however, predict dense future frames during training, repeatedly modeling largely unchanged content and coupling action-conditioned dynamics to nuisance appearance variations. At inference, processing each complete observation with the heavy video expert bottlenecks few-step action generation. Accordingly, we propose DeltaWAM, which jointly predicts visual deltas and actions using dense-anchor, sparse-delta, and action streams, with three architectures that differ in representation and computation sharing. We further develop Streaming Delta Memory (SDM), which updates cached anchor context with compact observed deltas, reducing heavy video-expert processing. On RoboTwin, DeltaWAM with SDM improves average success over Fast-WAM from 81.3% to 85.4% in the clean setting and from 75.8% to 83.9% under visual randomization. The three architectures reduce training FLOPs by 17.78-23.77%, while SDM reduces one-step inference latency and FLOPs by 36.57% and 31.55%, respectively; real-world evaluations further show the highest overall success rate and normalized progress among the evaluated policies. Code: https://github.com/AIGeeksGroup/DeltaWAM. Website: https://aigeeksgroup.github.io/DeltaWAM.
Text extracted automatically; images, tables and formatting may be missing. Original: https://huggingface.co/papers/2609.28811