Taste-Bench (Product): news timeline & CVEs

AI agents have bad taste in long-term decisions and a new benchmark proves it

Taste-Bench finds leading models choose the better long-term agent path only about 60 percent of the time.

Researchers introduced Taste-Bench, 502 questions that ask models to choose the better long-term path at decision forks in coding and AI-research agent trajectories. Of 14 models, GPT-5.6 Sol led at 59.7 percent and GPT-5.5 at 59.5 percent, versus 25 percent for chance under both-order scoring. Accuracy fell from 62.3 percent to 21 percent when the key evidence appeared only after later work, and extra reasoning did not help. Distilling a teacher into Qwen3.6-27B raised accuracy from 30 to 47.9 percent and lifted a separate agent from 14.6 to 33.7 percent on 41 held-out SWE-bench Pro tasks.

The DecoderOct 18, 08:49 UTC · 3d agoAI research9· 3 reads

The Tasteful Agent: Measuring and Improving Taste in Long-Horizon Tasks

Researchers introduce Taste-Bench, a benchmark measuring LLM agents' ability to make good long-horizon decisions at critical decision forks.

This paper introduces Taste-Bench, a benchmark designed to measure the 'taste' of LLM agents, defined as their ability to make good long-horizon decisions at critical forks. The benchmark is automatically constructed from agent trajectories in engineering and research tasks, presenting decision points where one direction leads to a better outcome. Evaluation shows that even the best frontier model answers only 59.7% of taste questions correctly. The researchers also demonstrate that taste can be improved by distilling judgment from a teacher model that has access to the full trajectory outcome.

Hugging Face daily papersSep 21, 20:00 UTC · 29d agoAI research1· 1 read

Taste-Bench

Timeline2 articles

Appears with

AI agents have bad taste in long-term decisions and a new benchmark proves it

The Tasteful Agent: Measuring and Improving Taste in Long-Horizon Tasks