D-JEPA: A Decision-Aligned Latent World Model
D-JEPA aligns latent world-model distances with executed outcomes, improving robot and driving action selection.
D-JEPA is a decision-aligned latent world model that learns relations among competing futures from executed outcomes, addressing cases where a closer predicted goal produces a worse action. A bounded, permutation-equivariant operator combines goal-relative features with ordinal evidence and keeps the result compatible with JEPA-style latent-distance planning. Tests span latent control, manipulation, pretrained action models, physical robots, and autonomous driving, with 87.89% success on PushT, a 15.04-point average gain on RoboTwin, and a 17-point gain on physical robot tasks.
- D-JEPA learns which candidate futures succeed from executed outcomes, not prediction error alone.
- A bounded permutation-equivariant operator refines pretrained geometry where choices matter.
- It reached 87.89% success on PushT and supports latent-distance planning.
- Reported gains are 15.04 points on RoboTwin and 17 points on physical robots.
From the original article
1 min read at huggingface.co
Latent world models predict the consequences of actions, but accurate prediction does not guarantee that latent distance reflects which candidate will execute successfully. We identify a decision-local prediction gap: among the few futures competing for execution, a candidate predicted closer to the goal can produce a worse realized outcome than an available alternative.
By Shuaijun Liu, Chengyu Wu, Qifu Wen · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.