C2-Routing cuts VLM visual tokens by routing crop or compress
Original title: C$^2$-Routing: Complementary Cropping and Compression for Efficient Vision-Language Inference
With a 1,024-token cap already applied, the method reports 36.53% fewer FLOPs and a 0.77% mean relative score drop.
Researchers propose C2-Routing, a model-agnostic way to reduce visual input for visual question answering by choosing cropping or whole-image compression from the image and question. A safety gate uses predicted joint correctness under original and reduced inputs to fall back when reduction would drop accuracy. Training and evaluation ran on RTX 3090 GPUs across three VLMs and 18 datasets covering local detail, global scenes, scene text, and documents. Against full-image baselines with the same 1,024 visual-token cap, they report 36.53% lower end-to-end FLOPs and 42.36% fewer visual tokens, with a 0.77% mean relative task-score decrease.
- Cropping is used for small targets; compression keeps context for large or distributed evidence.
- The safety gate can fall back to the original input to protect accuracy.
- Gains are measured beyond a shared cap of 1,024 visual tokens.
- Average visual-token reduction is 42.36% and FLOPs reduction is 36.53%.
From the original article
1 min read at arxiv.orgVisual Question Answering (VQA) imposes substantial computational overhead on vision-language models (VLMs), motivating model-agnostic input reduction methods that can be readily applied across diverse VLMs, including in black-box settings. Simply enforcing a visual-token cap reduces computation, but it can discard information needed to answer the question.
By Guorui Lu · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.