Taste-Bench finds top models weak at long-term agent choices
GPT-5.6 Sol scored 59.7 percent on 502 binary forks; trained Qwen3.6-27B advice more than doubled SWE-bench Pro success.
48
Product
SWE-Bench Pro
- Mentions · 7d0
- Mentions · 30d3
- All mentions5
- First seenSep 7, 2026
- Last mentioned12d ago
Timeline
5 articles · newest firstOpera: A Verbal Critic Framework for Long-horizon Coding Agents
Opera, a persistent verbal critic, lifts coding-agent resolve rates by up to 15 points on SWE-Bench-style benchmarks.
52
Coding-Agent Benchmarks Should Match Their Users' Task Flows
Coding-agent benchmarks should be calibrated to real IDE session task flows, not issue tasks alone.
52
Can LLMs Engineer Their Own Agent Harness? ByteDance Seed’s HarnessDev Says Only 34 of 64 Changes Generalize
ByteDance Seed's HarnessDev benchmark finds LLM-built agent harnesses trail human engineering on code and search, with only 34 of 64 revisions generalizing.
48
SWE-Bench Pro Verified: A Reliable Benchmark for Software Engineering Agents
SWE-Bench Pro Verified is a corrected benchmark showing prior coding-agent scores were inflated by reward hacking and flawed tasks.
35
Appears with
- GPT-5.5ai model · 2
- Terminal-Bench 2.1product · 2
- ByteDance Seedvendor · 1
- DeepSWEproduct · 1
- Gemini 3.1 Proai model · 1
- M-A-Porganization · 1
- Georgia Institute of Technologyorganization · 1
- SWE-Bench Pro Verifiedproduct · 1
- ByteDance Seedorganization · 1
- JetBrainsorganization · 1
- Taste-Benchproduct · 1
- HarnessDevproduct · 1
- Terminus-2product · 1
- Alibabavendor · 1
- Opus 4.8product · 1
- Seed 2.0 Proproduct · 1
Entities are extracted by the model from each article. First seen , last mentioned . Watching keeps it in this browser only (no account); the watchlist page and dashboard alerts use it.