OmniConfess: Eliciting Token Confessions to Mitigate Omni-Modal Hallucination
OmniConfess re-scores tokens under evidence interventions to reduce omni-modal hallucinations on a 3,540-example benchmark.
OmniConfess is a training-free method for reducing hallucinations in omni-modal models that combine text, images, audio, and video. It fixes a candidate response and re-scores it token by token under controlled channel-wise evidence interventions, producing a confession of evidential dependence, then preserves grounded content and corrects commitments driven by irrelevant or contradictory evidence. The authors evaluate it on OmniHalluBench, a 3,540-example benchmark built from six datasets covering judgment and free-form generation across all four modalities. Experiments report hallucination reductions across heterogeneous settings, and the code and benchmark are public.
- Training-free method re-scores tokens under channel-wise evidence interventions.
- Confessions show which modality evidence supports each generated commitment.
- OmniHalluBench has 3,540 examples from six datasets across four modalities.
- Code and the benchmark are publicly released on GitHub.
From the original article
1 min read at huggingface.co
Omni-modal large language models (OmniLLMs) unify text, images, audio, and video, yet hallucinate when generation relies on the wrong evidence. Existing inference-time methods can reduce hallucinations, but rarely reveal which evidence sustains a generated commitment. We introduce OmniConfess, a training-free method for mitigating omni-modal hallucinations.
By Huiqiang Rong, Haoran Luo, Hui Feng · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.