Agents keep finding exploits after they stop, SCoRe study finds
Original title: Stopping Is Not the Limit: Understanding and Eliciting Agent Capability in Long-Horizon Vulnerability Exploitation
With SCoRe, MiMo reached 67.0% capability coverage at 2,000 turns, above a 58.6% GPT-5.5 reference.
A paper on ExploitBench tests whether language-model agents can keep producing verified exploits after their first stop. DeepSeek-V4-Flash-0731 and MiMo-V2.6-Pro gained further verified successes, but returns fell as stop responses and impossibility judgments accumulated. SCoRe compacts context at stop bursts and treats those judgments as unproven, beating summary-based continuation and other context-management baselines.
Unconfirmed: Capability scores are benchmark results, not evidence of attacks outside ExploitBench.
- Additional verified gains appear after the first stop, then diminish.
- MiMo scored 32.4% under the original stopping policy.
- The GPT-5.5 comparison uses ExploitBench's 300-turn reference results.
- SCoRe marks retained impossibility judgments as unproven.
- Products
- ExploitBenchSCoRe
From the original article
1 min read at arxiv.orgVulnerability exploitation tests whether language model agents can develop working exploits from known software flaws. Sequential test-time scaling can improve exploitation performance, but larger budgets do not ensure sustained exploration: agents often stop with budget remaining. Yet what happens when execution continues beyond stopping remains underexplored.
By Deokhyung Kang · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.