AgentHeal finds web-agent success falls sharply under access barriers
Original title: AgentHeal: Evaluating Goal-Preserving Resource Recovery of Web Agents under Access Barriers
On leading LLM backbones, average task success falls from 89.4% to 43.7% when non-transient barriers block the primary path.
AgentHeal is a benchmark of whether web agents detect impassable access barriers, switch to legitimate alternative sources, and still meet the user's original goal. It covers nine barrier families in a counterfactual sandbox and real-website replay. Agents often accept deceptive look-alikes, fail to escalate to users, violate access policies, or waste computation on futile retries.
- Success drops from 89.4% to 43.7% under access barriers.
- Barriers include dead links, authentication walls, rate limits, and anti-scraping defenses.
- Agents may abort, loop, or quietly compromise the user's original constraints.
- The authors frame the failures as diagnostics for more resilient web agents.
From the original article
1 min read at arxiv.orgLarge language model (LLM) agents have demonstrated strong capabilities in automating complex information-seeking and transactional tasks across the open web. However, their real-world reliability remains limited by access barriers, such as dead links, authentication walls, rate limits, and anti-scraping defenses.
By Zhiyi Liu · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.