DexTacWAM: A Visuo-Tactile World-Action Model for Dexterous Manipulation
DexTacWAM injects fingertip touch into a video world model, scoring 70.6 versus 38.0 on dexterous tasks.
DexTacWAM is a visuo-tactile world-action model that encodes each fingertip, compresses those features with a finger- and pose-aware module, and injects the tactile latent into a video diffusion world model. On six contact-rich dexterous tasks using a 22-DoF bimanual platform, it averages 70.6 versus 38.0 for the strongest baseline. Ablations show modeling contact evolution, not tactile conditioning alone, drives the gain, and continual vision-to-touch learning extends a pretrained video model with roughly 100 demonstrations per task.
- Each fingertip is encoded and compressed into a tactile latent.
- Average score is 70.6 versus 38.0 for the strongest baseline.
- Removing tactile world modeling drops a four-task mean from 74.7 to 26.6.
- About 100 demonstrations per task adapt a frozen vision VAE to touch.
- The compressor keeps 89.4% contact recall and speeds training 2.26x.
- AI models
- DexTacWAM
From the original article
1 min read at arxiv.orgDexterous manipulation depends on contact dynamics that are often only partially observable from vision. Recent World-Action Models (WAMs) couple predictive video world modeling with action generation, but remain largely vision-centric and therefore cannot directly model these contact dynamics.
By Haoran Yuan · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.