DexForesight lifts dexterous policies with implicit spatial foresight
Original title: DexForesight: Grounding Dexterous Policies with Implicit Spatial Foresight
On 11 DexJoCo tasks success rises from 49.4% to 60.5%; three real-robot tasks reach 58.3% versus 50.0%.
DexForesight trains an action-conditioned spatial world model on offline scene depth, kinematic hand geometry, and demonstrations, then freezes it and aligns policy queries with its predictive world queries. At deployment only the policy remains, using RGB, language, and robot state without depth or explicit future prediction. The authors report gains over a base policy in simulation and over the strongest baseline on three real-robot dexterous tasks.
- The world model predicts 3D hand-point motion and visual feature changes, including occluded fingers.
- Simulation covers 11 DexJoCo tasks; real evaluation uses three manipulation tasks.
- Deployment drops depth input and explicit future prediction.
From the original article
1 min read at huggingface.co
Dexterous manipulation requires rich spatial foresight over both the surrounding scene and the robot hand. Existing world-action models often acquire such predictive knowledge through explicit future prediction. We introduce DexForesight, a framework that learns spatial foresight implicitly by injecting dynamics-aware world representations into the latent space of a dexterous policy.
By Zhongxi Chen, Yi Yang, Xinyi Chen · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.