Omni-Woodpecker rebalances hidden states to cut Omni LLM hallucinations
Original title: Omni-Woodpecker: Hallucination Correction via Evidence-Prior Rebalancing for Omni LLMs
On Qwen2.5-Omni-7B the training-free method adds 12.9 and 8.6 points on CMM and AVHBench.
Omni-Woodpecker is a training-free method that mitigates cross-modal hallucinations in Omni LLMs by rebalancing fused representations rather than editing outputs. It detects conflicts from modality-specific views, separates grounding evidence from competing priors with orthogonal decomposition, then suppresses misleading priors before decoding. With Qwen2.5-Omni-7B it reports gains of 12.9 and 8.6 points on CMM and AVHBench, plus 0.67–2.25 points on four general reasoning benchmarks. It also improves VideoLLaMA2-AV by 4.6 and 3.9 points on the same two benchmarks. Code is public.
Unconfirmed: Benchmark gains are the authors' reported results and have not been independently confirmed here.
- The method does not require additional training.
- Gains of 0.67–2.25 points are reported on four general OLLM benchmarks.
- VideoLLaMA2-AV improves by 4.6 and 3.9 points on CMM and AVHBench.
- Code is at github.com/ZiiJianLiu/Omni-Woodpecker.
From the original article
1 min read at huggingface.co
Recent Omni LLMs (OLLMs) have demonstrated remarkable capabilities in joint audio-visual reasoning, yet cross-modal hallucinations remain a major challenge. Existing mitigation methods mainly operate at the output level, overlooking the entanglement of grounding evidence and learned priors within fused representations.
By Zijian Liu, Chaoyou Fu, Yan Yang · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.