MirrorCode (Product): news timeline & CVEs

AI agents have bad taste in long-term decisions and a new benchmark proves it

Taste-Bench finds leading models choose the better long-term agent path only about 60 percent of the time.

Researchers introduced Taste-Bench, 502 questions that ask models to choose the better long-term path at decision forks in coding and AI-research agent trajectories. Of 14 models, GPT-5.6 Sol led at 59.7 percent and GPT-5.5 at 59.5 percent, versus 25 percent for chance under both-order scoring. Accuracy fell from 62.3 percent to 21 percent when the key evidence appeared only after later work, and extra reasoning did not help. Distilling a teacher into Qwen3.6-27B raised accuracy from 30 to 47.9 percent and lifted a separate agent from 14.6 to 33.7 percent on 41 held-out SWE-bench Pro tasks.

The DecoderOct 18, 08:49 UTC · 3d agoAI research9· 3 reads

Trajectory-Level Security Debt in LLM Coding Agents

A new metric tracks static-analysis security debt across intermediate states of LLM coding agents, not just final code.

The authors define the Security Debt Line Integral (SDLI), which accumulates static-analysis risk as an LLM coding agent reaches a new best test-pass ratio. They apply four SAST tools to 830 passing SWE-bench runs, 712 ProgramBench workspaces, and 13 public MirrorCode trajectories. Two-tool CWE agreement is only 3.9% of SWE-bench runs and 26.2% of 80 ProgramBench runs that pass at least 90% of tests; excluding three advisory-heavy classes drops the latter to 6.2%. The authors stress these are scanner findings, not validated vulnerabilities, and that SDLI's use for steering agents remains unproven.

arXiv cs.CRSep 28, 13:57 UTC · 22d agoAI safety & security1· 1 read

MirrorCode

Timeline2 articles

Appears with

AI agents have bad taste in long-term decisions and a new benchmark proves it

Trajectory-Level Security Debt in LLM Coding Agents