T2-z: Test-Time Optimization of Environment Latents for Robotic World Models
T2-Z adapts frozen robotic world models at test time by optimizing a compact environment latent.
T2-Z adds a learnable environment latent to a robotic world model so predictions account for unobserved factors such as friction, mass, and environmental rules. Training alternates between learning action-conditioned dynamics with fixed latents and optimizing those latents with the world model frozen. At test time the model stays frozen and a new latent is fit from interactions. Across 7 simulated settings and 4 real tasks, object prediction error fell from 15.20 to 7.00 and action success rose from 33.73% to 72.96%.
- Learnable latent captures unobserved physical and rule factors.
- Test-time fit can use a single interaction trajectory.
- Object prediction error fell from 15.20 to 7.00.
- Action success rose from 33.73 percent to 72.96 percent.
- Tested on seven simulations and four real robotic tasks.
From the original article
1 min read at arxiv.orgPredicting how the world responds to an action requires more than what can be seen: robots must account for environmental factors that are neither directly observable nor explicitly specified, from physical properties such as friction and mass to human-defined environmental rules.
By Chenyuan Zhou · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.