Microsoft and UC Berkeley release TokenEconomizer for coding agents
On 277 SWE-Bench tasks it cut mean cost 18.4% to 30.9% across five agent-model settings without changing weights.
5
Stories per day · 30 days
Relatedcoding-agents7benchmarks3evaluation3agent-harnesses2ai-agents2terminal-bench2agent-evaluation1agent-harness1benchmark1claude-code1code-generation1code-graphs1context-compression1decision-theory1early-stopping1embeddings1file-selection1gemma1
38 stories, newest first · one card per story
Stories tagged #swe-bench
Framework argues AI agent restart monitors should be judged by decision value, not accuracy
A 90% accuracy monitor can swing from a 5-point loss to a 35-point gain in agent success depending on residual restart budget, an exact analytical result.
Small coding agents gain little from code graphs past one hop
On SWE-bench Verified, structured tools added 17–21 points at 1.7–4B but only 4–5 points at 32B.
Independent review lifts coding-agent accuracy at modest extra cost
A critic review gains 5.9–9.5 points on Terminal-Bench and 3.9–6.6 on SWE-bench Verified at about baseline cost.
Coding-agent harnesses cost more without better benchmark scores
Harness swaps changed outcomes about as often as reruns, while system prompts made some tasks cost up to three times more.
6
SWE-DIVE evaluates coding agents in multi-turn sessions
A user simulator and conversation validator run inside Harbor to score whether the agent solves the task.
HarnessAdapt-RL trains coding-agent policy and harness together
On a 9B model, pass@1 rises 9.8, 12.6, and 6.4 points over GRPO on three SWE-bench sets.
HarnessPro evolves agent harnesses for better performance at lower cost
With GPT-5.5 and Codex, heavy mode gained 2.0 pass-rate points and cut per-task cost 26.1% versus RHO.
Taste-Bench finds top models weak at long-term agent choices
GPT-5.6 Sol scored 59.7 percent on 502 binary forks; trained Qwen3.6-27B advice more than doubled SWE-bench Pro success.
12
StopLoss aborts likely-failing coding-agent runs early
On SWE-bench Verified it cut cost per retained solve by 4.5 percent while keeping 99 percent of successful runs.
HSPR recovers software-agent benchmark scores from small task subsets
At a 10% task budget on four SWE-bench family banks, it had the lowest MAE on three and the lowest MaxErr on all four.
Coding agents fix more issues when shown whole files
On 300 SWE-rebench issues, verbatim file selection beats compressors by 6.3 points with either model.
Show HN: Parsimony – an open-source benchmark for how much code AI models add
Parsimony ranks AI coding models by net syntax units added on SWE-bench Verified patches.
ANTE: Anticipate, Then Execute -- Learning Internal World Models for Coding Agents from Verified Beliefs
ANTE lifts Qwen3-32B to 57.2% pass@1 on SWE-bench Verified using verified action forecasts.
JetBrains Releases Mellum2.1: A 12B MoE Open Model for Coding Agents
JetBrains released Mellum2.1, an Apache 2.0 12B MoE coding model with 2.5B active parameters.
Opera: A Verbal Critic Framework for Long-horizon Coding Agents
Opera, a persistent verbal critic, lifts coding-agent resolve rates by up to 15 points on SWE-Bench-style benchmarks.
Before They Can Solve: Predicting Post-Training Coding-Agent Performance from Base Models
Three screens rank base models by likely coding-agent performance after expensive post-training.
[AINews] Reflection Beam - 501B-A23B American Open Model
Reflection launched Beam, a US-trained 501B-total, 23B-active MoE for coding and agents, with Apache 2.0 weights due this month.
6
FC-SWE trains coding agents on recovery after failed patches
With Qwen3.5-4B and SWE-agent it reports 41.7% Resolved@1 versus 38.9% for GRPO.
AutoCompact: Learning When to Compact Context in Long-Horizon Coding Agents
AutoCompact trains coding agents to compact context, raising SWE-bench Verified pass rates by 9.2 points.
PivotOPD: Learning to Recover from Pivotal Mistakes in Multi-Turn Agents
PivotOPD trains agents to avoid and recover from early pivotal mistakes, lifting ALFWorld and SWE-Bench scores.
PivotOPD: Learning to Recover from Pivotal Mistakes in Multi-Turn Agents
PivotOPD trains multi-turn agents to avoid pivotal mistakes and recover within a few turns.
Do LLM Agents Execute the Plans They Declare? From Planning-Mode Declaration to Pattern-Specific Execution
Pattern-specific executors close LLM agents' plan-execution gap, lifting ALFWorld success from 0.48 to 0.92.
Google Research Open-Sources RRSI: AI Agents That Improve Their Own Harness Without Overfitting
Google Research open-sourced RRSI, letting frozen-weight agents improve their harness without overfitting benchmarks.
5