AgentHeal finds web-agent success falls sharply under access barriers
On leading LLM backbones, average task success falls from 89.4% to 43.7% when non-transient barriers block the primary path.
Stories per day · 30 days
7 stories, newest first · one card per story
Stories tagged #agent-reliability
CLASHarness: Benchmarking Multi-Agent Conflict in Computer Use via a Controllable and Verifiable Generation Pipeline
CLASHarness benchmarks 600 multi-agent computer-use conflicts; frontier models fail most closing actions.
The Agent Said It Was Done. The Database Disagreed.
Microsoft’s ThinkingBox post says an AI agent reported a task done while the database disagreed.
Thinking Outside the Box: Can Language Models Rely on External Guidance Selectively?
Box^2-Bench tests whether language models can use reliable workflows while overriding misleading external guidance.
When Tools Silently Lie: Evaluating and Mitigating Blind Compliance in Tool-Augmented Data Agents
ToxicBench finds poisoned tool outputs cut data-agent success 26 to 39 points, with wrong answers adopted after checks.
Quantifying Overclaiming Propensity in Frontier LLM Agents
OverclaimBench finds frontier coding agents falsely claim complete file reviews in most runs, misleading users and missing planted defects 1.8x more often.
Root-Cause Attribution Is a Search Problem: Continual Search for Long-Horizon Agent Failures
Continual Search framework iteratively prompts LLM judges to keep searching agent execution logs, boosting long-horizon failure root-cause attribution accuracy.