InteractiveWorld benchmarks egocentric MLLM assistants on 120 tasks
Original title: InteractiveWorld: A Verifiable Human-Centric Environment for Building Better Interactive MLLM Assistants
Across 12 frontier models, Claude Fable 5.1 reaches at most 20% and 25% on long-horizon tasks.
InteractiveWorld is a closed-loop simulator in which an MLLM assistant instructs a human agent and the environment updates what is seen next. InteractiveBench has 120 multi-turn tasks in six categories, scored with state-based rewards. Tests of 12 frontier MLLMs find them unreliable as physical-world assistants. The strongest reported model, Claude Fable 5.1, reaches at most 20% on long-horizon Guidance and 25% on Planning.
- The setup is human-centric rather than a robot-centric simulator.
- Each task scores the assistant with verifiable, state-based rewards.
- Twelve frontier MLLMs were evaluated.
- Claude Fable 5.1 is the strongest model reported on those long-horizon tasks.
From the original article
1 min read at huggingface.co
People have long dreamed of an assistant that is always there to guide them and remind what they forget, from new dishes to lost keys. Powered by multimodal large language models (MLLMs), egocentric AI assistants on wearable devices are bringing this dream within reach: they see what the user sees and give instructions for user's physical work when necessary.
By Qinsi Wang, Wei Wen, Chong Zhou · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.