Show-Harness: Just a VLM Agent Can Play Robots
Show-Harness lets VLM agents control robots via discrete semantic action units, outperforming VLA baselines zero-shot and after light fine-tuning.
Show-Harness is an embodied agent harness that exposes discrete semantic action units a VLM reasons over, with embodiment-specific interpreters grounding them into local robot actions. It enables zero-shot robot control with closed-source frontier VLMs and low-cost adaptation of small open-source VLMs using only a few GPU-hours of fine-tuning. The companion GUMI (GUI Manipulation Interface) extends the same semantic action space to GUI-based demonstration collection without specialized teleoperation hardware. Experiments show robust generalization across tasks, embodiments, and environments, beating representative agentic and VLA paradigms.
- Discrete semantic action units keep the VLM responsible for decisions while interpreters ground actions deterministically.
- GUMI enables human and agent demonstration collection across embodiments without teleoperation hardware.
- Reports gains over agentic and VLA paradigms without added model capacity or embodiment-specific pretraining.
From the original article
1 min read at arxiv.orgFoundation vision-language models (VLMs) exhibit broad intelligence about the world, yet translating this intelligence into robot control remains challenging. We present Show-Harness, an Embodied Harness that enables VLMs to "play" robots through a compact semantic interface linking intent to action.
By Yanzhe Chen · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.