MMEnterprise-Bench tests agents on evolving multimodal workplace records
Original title: MMEnterprise-Bench: Evaluating Enterprise Agents over Evolving Multimodal Records
On 1,613 questions, extra retrieval raised evidence recall without consistently improving answer accuracy.
MMEnterprise-Bench evaluates agents on simulated collaboration records that change across documents, chats, and figures. Its 1,613 questions cover evidence reasoning, conflict resolution, and unanswerability. Ten proprietary and open-source multimodal LLMs struggled when sources conflicted over time or evidence spanned multiple images. Code and data are public.
- Agents must resolve cross-source and temporal conflicts or detect insufficient evidence.
- A larger retrieval budget improved recall but not consistently answer accuracy.
- A larger retriever likewise failed to raise accuracy consistently.
- The strongest tested models still struggled when evidence spanned multiple images.
From the original article
1 min read at huggingface.co
Enterprise agents answer questions over workplace records that evolve through team collaboration. As information is shared, corrected, and revised across documents, chats, and figures, multiple versions of the same information accumulate in the workspace.
By Seohyun Lee, Jongha Kim, Dohwan Ko · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.