AgentRoboCritic analyzes frontier agents on direct robot control
Original title: AgentRoboCritic: Understanding the Capability of Frontier Agents on Direct Robot Control
Across 12 manipulation benchmarks, fine-tuned Muse-Glimmer-30B raises failure-mode accuracy from 32.58% to 74.24%.
Researchers introduce AgentRoboCritic, an intent-grounded framework that critiques each low-level robot action against a stated local intent. Applied to frontier agents such as GPT-6 Astra across 12 manipulation benchmarks, it finds closed-loop recovery, semantic grounding, and zero-shot manipulation, but grasping, alignment, and uneven recovery remain limits. They validate the critic on 400 human-annotated actions and release 12,844 annotated actions plus Muse-Glimmer-30B.
- Analysis covers 12 manipulation benchmarks at the action level.
- The critic is checked against 400 human-annotated actions.
- The released dataset contains 12,844 annotated actions.
- Muse-Glimmer-30B failure-mode accuracy rises from 32.58% to 74.24%.
From the original article
1 min read at arxiv.orgFrontier multimodal language agents such as GPT-6 Astra have demonstrated strong capabilities in controlling robots directly via low-level numerical actions without high-level primitives. However, existing benchmarks report only episode-level success rates, which show neither how these agents succeed nor where they fail.
By Yiyang Du · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.