WebArena-Pro benchmarks web agents on 300 tasks across 20 apps
Original title: WebArena-Pro: A Heterogeneous, Multimodal, Reproducible Benchmark for Web Agents
Codex with GPT 6.1 Sol and Claude Code with Claude Opus 5.5 both score 62.3%, the paper's best result.
WebArena-Pro is a reproducible benchmark of 300 tasks on 20 self-hosted web applications in six domain categories, treating audio and video as observations alongside text and vision. Tasks run in isolation for parallel evaluation and are human-validated after LLM-assisted triage. The paper reports 62.3% for its top two agent setups and 30.3% for the strongest GenericAgent configurations.
Unconfirmed: Scores are reported by the paper's authors; independent replication is not stated.
- OpenCode with Kimi 3.1 Pro scores 60.7%.
- Gemini 3.1 Pro omni and Gemma4 31B both score 30.3%.
- Dedicated actions extract information from embedded audio and video.
- The authors call it the broadest human-curated reproducible web-agent set to date.
From the original article
1 min read at huggingface.coWeb agents powered by large language and vision-language models are increasingly applied to realistic browser work that spans heterogeneous applications, multimodal content, and stateful workflows. However, existing reproducible web-agent benchmarks cover only a small number of web applications drawn from a few software categories, and restrict modality to text and vision.
By Imene Kerboua, Fatemeh Pesaran Zadeh, Xing Han Lù · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.