AI agents have bad taste in long-term decisions and a new benchmark proves it
Taste-Bench finds leading models choose the better long-term agent path only about 60 percent of the time.
Researchers introduced Taste-Bench, 502 questions that ask models to choose the better long-term path at decision forks in coding and AI-research agent trajectories. Of 14 models, GPT-5.6 Sol led at 59.7 percent and GPT-5.5 at 59.5 percent, versus 25 percent for chance under both-order scoring. Accuracy fell from 62.3 percent to 21 percent when the key evidence appeared only after later work, and extra reasoning did not help. Distilling a teacher into Qwen3.6-27B raised accuracy from 30 to 47.9 percent and lifted a separate agent from 14.6 to 33.7 percent on 41 held-out SWE-bench Pro tasks.
56