Skip to content
NewA calmer, faster ZeroHour

AgentWorld: Benchmarking Long-Horizon Collaboration of Multi-agent LLMs

Hugging Face daily papersby Raphael Shu, Yusen Zhang, Young Min Cho ()1 min read2
AI summary · grok-4.7Summaries by Vani AI

AgentWorld finds even the best tested LLM succeeds on only 52 percent of long-horizon multi-agent tasks.

AgentWorld is an open benchmark of 100 human-annotated tasks and 100 augmented variants for long-horizon collaboration among 3–20 LLM agents. Tasks span more than 50 interaction rounds in an MMORPG sandbox and require communication, joint planning, and resource sharing while each agent acts without access to others' internal states. Causal Collaboration Effectiveness traces which actions actually contributed to the outcome. Tests of Gemini 3 Flash, Claude Haiku 4.5, GPT-5 Mini, and DeepSeek R1-70B found a best task-success rate of only 52.0%, with failures from communication breakdowns, role confusion, and lost shared plans.

Key facts
  • 100 annotated tasks plus 100 variants in an MMORPG sandbox
  • Tasks need 3–20 agents across more than 50 rounds
  • CCE measures how much team effort causally contributed
  • Best tested model reached only 52.0% task success
Explain this AI news with Vani AIWhat was announced, what is new and why it matters, in a new chat

From the original article

1 min read at huggingface.co

Existing multi-agent benchmarks primarily test in competitive settings, short-horizon interactions under 20 steps, or simply aggregate individual performance, failing to isolate and highlight genuine collaboration capabilities of LLM-based agents. We introduce AgentWorld, a benchmark of 100 human-annotated tasks (with 100 augmented variants) for evaluating long-horizon, multi-agent collaboration.

Hugging Face daily papers

By Raphael Shu, Yusen Zhang, Young Min Cho · huggingface.co

Read the full article at Hugging Face daily papers

A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.