Imperceptible goal-image attacks spoof latent state in LeWorldModel
Original title: Spoofing the Latent State: Imperceptible Goal-Image Attacks Derail Planning in LeWorldModel
Epsilon 4/255 perturbations spoof block pose in 73-95% of scenes and drop LeWorldModel planning success from 87% to 32%.
The paper investigates test-time robustness of the latent planning interface in LeWorldModel, a joint-embedding predictive architecture that plans by minimizing squared Euclidean distance between predicted and goal embeddings. Task-relevant object state dominates latent distance, with shifting the block by a single pixel displacing the embedding 3 times further than an agent shift. A white-box l_infinity perturbation of epsilon=4/255 applied to a single input frame causes the encoder to register the block at an arbitrary attacker-chosen pose in 73-95% of scenes without disturbing the encoded agent state. Perturbing either the goal image or live camera observation drops planning success from 87% to 32% across all four standard LeWM benchmarks.
- 4/255 perturbation fools block pose in 73-95% of scenes
- Planning success drops to 32% from 87%
- Attack persists across Push-T, Two-Room, Reacher, and OGBench-Cube
- At 8/255 bridges 82-97% of squared latent distance
From the original article
1 min read at arxiv.orgJoint-embedding predictive architectures (JEPAs) have recently emerged as general-purpose visual world models for control. A prominent example is LeWorldModel (LeWM), which trains end-to-end from pixels using an isotropic Gaussian anti-collapse regularizer (SIGReg) and plans by minimizing the squared Euclidean distance between predicted and goal embeddings.
By Kunal Bhosikar · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.