TB-MathModel benchmark introduced for verifiable mathematical modeling
First agentic verifiable benchmark using containerized terminal tasks for contest-style math modeling problems.
Stories per day · 30 days
3 RSS
Relatedagent-harnesses2swe-bench2agent-cost1agent-harness1agentic-ai1benchmark1calibration1coding-agent1decision-models1droid1factory1language-model-agents1llm-agents1mathematical-modeling1miniwob1pareto-optimization1post-training1recursive-self-improvement1
23 stories, newest first · one card per story
Stories tagged #terminal-bench
Factory releases Droid 1.0 coding-agent harness with script mode
Programmatic tool calling is now default; Factory reports 65.2% on Terminal-Bench 4 with Claude Opus 5.5.
1
Trained agent policies carry harness preferences that break under interface changes
Reordering MiniWoB++ observation fields drops trained success from 85.0% to 19.2% without changing optimal decisions.
Independent review lifts coding-agent accuracy at modest extra cost
A critic review gains 5.9–9.5 points on Terminal-Bench and 3.9–6.6 on SWE-bench Verified at about baseline cost.
HarnessPro evolves agent harnesses for better performance at lower cost
With GPT-5.5 and Codex, heavy mode gained 2.0 pass-rate points and cut per-task cost 26.1% versus RHO.
Jev 1.13 raises agent cost 19% without improving pass rate
On 21 held-out Terminal-Bench 2.1 tasks, Jev matched a 73.0% pass rate but cost 22% more than a placebo.
EdgeWise: Bringing Long-Horizon Agents to the Edge via Selective Cloud Delegation
EdgeWise delegates only key agent steps to the cloud, cutting inference cost by up to 62%.
Scalable Weak-to-Strong Verifier Hardening for Agent Environments
A hacker-fixer loop automatically hardens agent verifiers on Terminal Bench, KernelBench, and SETA against reward hacking.
Scaling Trajectories for Complex Tasks through Recursive Self-Rewrite
Recursive Self-Rewrite converts harness-assisted trajectories into general training data, lifting Qwen-3.8-27B Terminal-Bench 2 pass@3 from 57.0% to 74.2%.
Self-Supervised Scaling of Terminal Environments for Scientific Domains
Self-supervised reconstruction of scientific software yields verified terminal-agent data and lifts Qwen on Terminal-Bench 2.
ActiveSaddler: Automated Curriculum Learning for Agent Harness Optimization
ActiveSaddler adapts agent-harness curricula, raising Pass@1 by 4.4 and 7.5 points on two benchmarks.
Google Research Open-Sources RRSI: AI Agents That Improve Their Own Harness Without Overfitting
Google Research open-sourced RRSI, letting frozen-weight agents improve their harness without overfitting benchmarks.
5
Claude Sonnet 5.5 gets faster without a price hike
Anthropic released Claude Sonnet 5.5, over 30% faster, with much stronger coding scores at the same API price.
Anthropic Releases Claude Sonnet 5.5: 70.6% on Terminal-Bench 4.0 at the Same $2/$10 Price
Anthropic launched Claude Sonnet 5.5, scoring 70.6% on Terminal-Bench 4.0 at the same $2/$10 price.
MILO: Automated Harness Discovery via Orchestrated Multi-Agent Evolution
MILO co-evolves agent harnesses and search strategy, beating prior harnesses on Terminal-Bench, PaperBench, and DeepSWE.
CliffCompaction: Cost-Efficient Compaction for Long-Horizon Coding Agents
CliffCompaction cuts long-horizon coding-agent context cost by up to 50% without rewriting retained text.
xAI launches Grok 4.7 at bargain prices, but benchmarks reveal a wide gap to Claude and GPT-6
xAI released Grok 4.7 at low API prices, but benchmarks trail Claude Fable 5.1 and GPT-6.
An Empirical Study of Harness Design for Coding Agents
Empirical study of coding agent harnesses across 176 settings shows context management prevents overflow failures and bash-only interfaces cut costs for capable models.
Don't Mask the Environment: Observation Supervision Changes How Agents Explore Under RL
ActObs adds observation-token supervision to SFT, yielding higher pass@k for Qwen3 agents after GRPO on Terminal-Bench 2.0 and code editing.
An Empirical Study of Harness Design for Coding Agents
Empirical study of 176 matched coding-agent harness configurations shows context management cuts overflow failures while planning and action-space choices trade accuracy against cost.
Can LLMs Engineer Their Own Agent Harness? ByteDance Seed’s HarnessDev Says Only 34 of 64 Changes Generalize
ByteDance Seed's HarnessDev benchmark finds LLM-built agent harnesses trail human engineering on code and search, with only 34 of 64 revisions generalizing.
9
T1: Terminal Agent Reinforcement Learning for Long-Horizon Tasks
T1, a 122B MoE terminal agent trained with reinforcement learning, reaches 64.0% on Terminal-Bench 2.1, surpassing GPT-5.4 and GLM-5.1 on long-horizon tasks.
Benchmarking Qwen3.8 27B quantizations: 4-bit holds up, 1-bit collapses
Quesma benchmarks Qwen3.8 27B quantizations: 4-bit Q4_K_M matches BF16 on key benchmarks while 1-bit collapses to random chance.
10