MEND: Label-Free Detection, Localisation, and Correction of Latent Hallucination in World Models
MEND detects and localizes silent latent hallucinations in frozen world models using a single score field.
The paper identifies latent hallucination in world models: a predicted next latent can decode to a scene that never occurs, and the error compounds because it is fed back autoregressively. Masked Empirical-Bayes Neural Denoising (MEND) trains one conditional score network by denoising score matching on real transitions of a frozen self-supervised model, without ground-truth error labels. On two navigation environments, score magnitude detects hallucination with AUROC up to 0.80 without using actions, localizes it with per-token AUPRC up to 0.87, and defines a correction that reduces single-step latent error. Part of the error lies tangent to the data manifold, so the authors emphasize detection and localization.
- Latent world-model hallucinations decode to scenes that never occur and compound.
- One score network detects, localizes, and suggests an inference-time correction.
- Detection AUROC reaches 0.80 without using actions, above a Gaussian baseline.
- Per-token localization AUPRC reaches 0.87 on two navigation environments.
From the original article
1 min read at arxiv.orgWorld Models are appearing as the next major frontier in computer vision. However, their robustness is currently largely unexplored. We identify the phenomenon of hallucination in latent World Models: given a state and an action, the predicted next latent can decode to a scene that never occurs.
By Ali J Alrasheed · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.