AgentProofBench tests reusable web-agent verifiers across 97 tasks
Original title: AgentProofBench: Benchmarking Reusable Verifiers for Web Agents
DeepSeek Flash and Pro keep most source programs under replay tests, while panel errors track failed deliveries.
AgentProofBench links a frozen acceptance rule to later execution from identical saved openings. It covers 97 tasks, 79 families, and four web services. Experiments with DeepSeek Flash and Pro report retained source programs of 82/96 and 79/96 across 48 families. On 22 execution families, error-free panels match correct delivery far more often than panels with errors.
- Reuse-rebuild intervals are [-5.47, 1.57] and [-10.16, 0.00] percentage points.
- 32 of 33 error-free panel assignments delivered both openings correctly, versus 1 of 11 with errors.
- Neutral continuation had the highest correct delivery under the tested generic feedback.
- Instructions, states, programs, reference rules, and trajectories are released.
From the original article
1 min read at huggingface.co
Reusable Web-agent verification requires a rule to follow changing requirements and support subsequent execution. We introduce AgentProofBench, which connects paired tests of a frozen acceptance rule to execution from identical saved openings. The resource spans 97 tasks, 79 families, and four Web services.
By Jingjie Ning, Xueqi Li, Yi Lu · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.