OpenVLHarness adds tools and memory for visual agent reasoning
Original title: OpenVLHarness: Real World Visual Agentic Tool-Use with Multimodal Memory
On 22 benchmarks, average scores rise 9.7–12.6 points over matched no-tool baselines across eight backbones.
OpenVLHarness is a multimodal reasoning harness with a capability layer and persistent multimodal memory for visual agent tool use. The layer exposes heterogeneous tools, renders their outputs as model-readable observations, stores images, structured data, and web references, and runs generated programs over stored data. A controlled study on 22 benchmarks found that raising the base model’s inference budget alone gave only modest gains. With eight open and proprietary backbones, the authors report average gains of 9.7–12.6 points, including +11.2 with Qwen3-VL-8B, +9.7 with Kimi K3, and +9.9 with GPT-6 Astra. With GPT-6 Luna at medium reasoning effort, they report 7.7–8.5 points over the base model, Codex, and Visual Sketchpad at maximum effort, at most 37% of those systems’ model-token cost.
Unconfirmed: Score and cost figures are reported by the paper’s authors.
- Evaluation used 22 benchmarks and eight open and proprietary backbones.
- Reported gains include +11.2 Qwen3-VL-8B, +9.7 Kimi K3, and +9.9 GPT-6 Astra.
- GPT-6 Luna at medium effort beat three maximum-effort systems by 7.7–8.5 points.
- That comparison used at most 37% of their model-token cost.
From the original article
1 min read at arxiv.orgMultimodal large language models (MLLMs) have made rapid progress on visual reasoning, yet many tasks require evidence that is difficult to obtain through internal reasoning alone: our controlled study on 22 benchmarks shows that simply increasing the inference budget of the base model yields only modest gains.
By Yu Zhou · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.