Frontier models scored on LIBERO-10 robot tasks without fine-tuning
Original title: From Reasoning to Action: Benchmarking Frontier AI Models on LIBERO-10 Robot Manipulation Tasks
Across 1,800 episodes, GPT-6 Astra led at 44% success and Gemini-4 Argon followed at 43%.
Researchers evaluated six frontier models on the LIBERO-10 robot benchmark using a closed-loop setup that sends end-effector poses and gripper commands from vision, proprioception, and tool calls. No task-specific fine-tuning or pretrained manipulation skills were used. GPT-6 Astra reached 44% success and Gemini-4 Argon 43% across 18 configurations and 1,800 episodes. More reasoning effort generally helped, at extra inference cost, while precise spatial and multi-step tasks remained weak. The framework and results were released.
- Models include GPT-6 Astra, GPT-6.1 Sol, GPT-6 Luna, Claude Opus 5, Claude Opus 5.5, and Gemini-4 Argon.
- Each model was tested at three reasoning-effort levels.
- Placement tasks went better than precise arrangements and complex multi-step interactions.
- The authors released the evaluation framework and experimental results.
- Products
- LIBERO-10
From the original article
1 min read at arxiv.orgRecent advances in frontier AI models have demonstrated remarkable capabilities in visual understanding, reasoning, planning, and tool use, raising the possibility of using general-purpose AI models for robot manipulation without task-specific training. However, their manipulation capabilities have not been systematically evaluated on standardized robotics benchmarks.
By May Espinola · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.