Slot-JEPA adds object slots and masks to latent world models
Original title: Slot-JEPA: Object-Structured Prediction Interfaces for Latent World Models
Frozen slot features stay transferable on seven benchmarks and improve planning on six planar and four robot tasks.
Slot-JEPA splits dense JEPA representations into stable semantic slots and time-varying spatial masks so downstream predictors can reuse object structure. A Transition Memory Predictor stores visual transitions as compact masks and separates slot-specific dynamics from shared cross-slot dynamics. Unbalanced optimal transport supervises mask prediction and scores plans by moving mass between predicted and goal masks. The paper reports transferable features and better goal-conditioned planning on the REALM-JEPA benchmark.
Unconfirmed: Benchmark gains are reported by the authors without independent confirmation.
- The slot interface is pretrained once and then frozen for downstream use.
- TMP retrieves context using statistics of corresponding slots.
- Planning covers six planar environments and four robotic arm tasks.
- Mask scoring uses unbalanced optimal transport rather than pointwise distances.
From the original article
1 min read at huggingface.co
As a prototypical latent world model, Joint-Embedding Predictive Architectures (JEPAs) provide strong semantic representations. However, dense patch representations expose a redundant interface to downstream predictors, which must implicitly infer spatial organization and temporal correspondence, presenting three challenges.
By Peisong Wen, Qianqian Xu, Minye Lei · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.