Cross-Modal Attention Acts as a Frequency Filter: Why Verbose Prompts Improve Robustness in Vision-Language Models
Study explains why verbose prompts make vision-language models more robust to image corruption, framing cross-modal attention as a frequency filter.
The paper shows question wording affects vision-language model robustness in two opposite ways: verbose prompts increase robustness, while semantically complex or fine-grained questions increase fragility under image corruption. Both effects are explained by question-conditioned cross-modal attention inducing a spectral filter over image patches, with answer drift largest when the filter and corruption share spatial frequencies. Experiments on Qwen3-VL and LLaVA-OneVision across GQA and CLEVR show verbose paraphrasing reduces drift variance by 70-81% on 8B models and improves accuracy.
- Verbose questions broaden the attention frequency filter, improving robustness
- Fine-grained questions concentrate the filter, increasing corruption sensitivity
- Verbose paraphrasing cut drift variance 70-81% on 8B models
- Tested on Qwen3-VL and LLaVA-OneVision over GQA and CLEVR
- Tags
- cross-modal-attentionimage-corruptionllava-onevisionprompt-engineeringqwen3-vlrobustnessvision-language-models
- AI models
- Qwen3-VLLLaVA-OneVision
From the original article
1 min read at arxiv.orgVision-language models (VLMs) are fragile under image corruption. We find that the wording of the question affects VLMs in two opposite ways. Verbose questions make VLMs substantially more robust---e.
By Farooq Ahmad Wani · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.