HyperBrowseComp: A Multilingual and Multimodal Stress Test for Web-Browsing Agents
HyperBrowseComp stress-tests web agents with 423 hard multilingual, multimodal questions needing obscure public evidence.
HyperBrowseComp is a multilingual, multimodal benchmark of 423 manually authored, human-validated questions across 13 languages for web-browsing agents. Each item has a concise, publicly verifiable answer that requires obscure evidence, multi-step clue chains, or sources such as videos, scans, images, or maps. Easier questions answerable from parametric knowledge were filtered using models without internet access. Several models were evaluated with provider-native search and a shared retrieval harness, with a human sample used to contextualize performance.
- 423 human-validated questions span 13 languages and multiple evidence types.
- Questions require obscure web evidence, multi-step clues, or non-text sources.
- Easier items were filtered using models without internet access.
- Models were tested with native search and a shared retrieval harness.
- A human sample contextualizes model performance and effort.
From the original article
1 min read at huggingface.co
We introduce HyperBrowseComp, a multilingual and multimodal browsing benchmark comprising 423 manually authored and human-validated questions across 13 languages, written by native or highly proficient speakers. Questions are designed to be extremely challenging.
By Alham Fikri Aji, Faiz Rizki Ramadhan, Zayd M. K. Zuhri · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.