GPT-6-Astra evaluated on EBench for direct and policy-harness robot control
Original title: Profiling a Frontier Agent with EBench: From Direct Control to Harness with Policy
Direct control leads on short tasks but lags on long-horizon work; calling a VLA or WAM matches top policies at lower token cost.
A report evaluates GPT-6-Astra as a robot-control agent under the same visual and proprioceptive protocol as post-trained vision-language-action and world action model policies. Direct waypoint control is competitive overall, strongest on short-horizon tasks, and weaker on long-horizon, tabletop, and high-precision contact-rich tasks. Letting the harness invoke a post-trained policy reaches the strongest compared policies' overall success while cutting token cost, but stays below the per-task upper bound.
- Direct control shows the least variation across generalization dimensions.
- Gaps come from suboptimal policy calls, fixed instructions, and grasp disruption.
- The agent had no robot-specific training for the EBench comparison.
- Products
- EBench
- Organizations
- InternRobotics
- AI models
- GPT-6-Astra
From the original article
1 min read at arxiv.orgFrontier multimodal large language models (MLLMs) can control robots without robot-specific training, but how their capabilities compare and interact with post-trained vision-language-action (VLA) and world action model (WAM) policies across manipulation demands remains to be thoroughly evaluated.
By Yifei Yao · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.