Semigroup-JEPA: Latent Dynamics Consistency for Zero-Shot Physics Generalization
SG-JEPA world model conditions latent prediction on physical parameters, halving open-loop prediction error versus DINO-WM and boosting robotic control success.
SG-JEPA extends the LeWorldModel JEPA framework by supplying the governing physics parameter to the temporal model via action-conditioning and jointly training an encoder and predictor through autoregressive latent rollout. On out-of-distribution gravitational-field tasks it reduces open-loop prediction error by up to 2x versus DINO-WM on 2D datasets and increases 3D robotic control success rate up to 2.5x using independently trained diffusion policies. A linear feature analysis attributes most of the gain to the encoder learning features that the predictor can carry forward through rollout.
- Extends LeWorldModel with physics-parameter action-conditioning and latent rollout
- Up to 2x lower open-loop prediction error than DINO-WM on 2D tasks
- Up to 2.5x higher 3D robotic control success with diffusion policies
- Attributes gains to encoder feature quality rather than predictor dynamics
- AI models
- SG-JEPAJEPADINO-WMLeWorldModel
From the original article
1 min read at arxiv.orgJoint-Embedding Predictive Architecture (JEPA) world models learn a compact latent representation of the world that supports prediction and planning, but their capability to learn physics and generate physically realistic dynamics remains hitherto untested.
By Andy Zeyi Liu · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.