NeuronEye: Query-Guided Visual Concept Activation for Vision-Language Reasoning
NeuronEye activates query-relevant visual concepts in frozen VLMs, lifting Qwen2.5-VL-7B CV-Bench by 3.1 points.
NeuronEye is a plug-in that builds a sparse, concept-level neuron vocabulary from a frozen vision-language model's intermediate states and activates only query-relevant visual concepts during inference. A suppression step reduces dominant perceptual directions so weaker relevant cues remain. On Qwen2.5-VL-7B it raises CV-Bench overall accuracy by 3.1 points, Distance by 9.5, and BLINK Multi-view by 8.3, with similar trends on LLaVA-1.6-7B.
- NeuronEye builds a sparse concept vocabulary from intermediate VLM representations.
- The language query activates relevant clusters and injects evidence in one forward pass.
- On Qwen2.5-VL-7B, CV-Bench rises 3.1 points and Distance by 9.5.
- BLINK Multi-view improves by 8.3, with similar gains on LLaVA-1.6-7B.
- AI models
- Qwen2.5-VL-7BLLaVA-1.6-7B
From the original article
1 min read at arxiv.orgCurrent vision-language models (VLMs) encode visual information in dense hidden states where object identity, spatial layout, and local attributes are implicitly entangled rather than explicitly disentangled, limiting their ability to isolate and modulate the specific visual evidence required by a given language query.
By Ruiyu Yan · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.