OmniDR-Bench scores agents on video-centered deep research reports
Original title: Beyond Watching and Searching: Benchmarking Agents for Video-centered Deep Research Reports
Fifty bilingual tasks and 6,414 criteria leave the best of 18 agent setups at 63.3 out of 100.
The paper defines video-centered deep research: given a video and request, an agent forms questions, retrieves external multimodal evidence, and writes an interactive report linked to video moments. OmniDR-Bench and Omni Atomic Rubric Trees evaluate audiovisual grounding, evidence use, and interactive presentation. Audited Rubric Construction combines independent evidence collection, automated checks, behavioral calibration, and human review. The authors say current agents still lack coverage, cross-modal reasoning, and accurate citations. Code and data are released.
- The benchmark has 50 Chinese and English tasks in 19 subcategories.
- Each rubric tree has three dimensions and 23 shared directions.
- Eighteen agent configurations were compared.
- The best reported score is 63.3 out of 100.
From the original article
1 min read at huggingface.co
Deep research agents synthesize open-web evidence into long reports, but current benchmarks do not evaluate research centered on a source video. We introduce Video-Centered Deep Research Report Generation. Given a video and user request, an agent identifies research questions, retrieves external multimodal evidence, and produces an interactive report linked to relevant video moments.
By Jiahao Meng, Qize Yang, Yuxuan Wang · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.