Show-Harness: Just a VLM Agent Can Play Robots
Show-Harness enables VLM agents to control robots via a semantic action interface, achieving zero-shot frontier control and few-GPU-hour adaptation of small VLMs.
Show-Harness exposes discrete semantic action units that VLMs reason over, with embodiment-specific interpreters deterministically grounding them into local robot actions. It enables zero-shot closed-source frontier VLM control and adapts small open-source VLMs for low-cost deployment with a few GPU-hours of fine-tuning. The companion GUMI interface extends the same semantic action space to GUI-based demonstration collection without teleoperation hardware, and Show-Harness-equipped agents outperform representative agentic and VLA paradigms.
- Semantic action interface keeps VLMs responsible for fine-grained physical decisions.
- Zero-shot closed-source frontier VLM control; small open VLMs adapted in few GPU-hours.
- GUMI enables GUI-based demonstration collection without teleoperation hardware.
- Outperforms representative agentic and VLA paradigms across embodiments and environments.
- AI models
- Show-HarnessGUMI
From the original article
1 min read at huggingface.co
Foundation vision-language models (VLMs) exhibit broad intelligence about the world, yet translating this intelligence into robot control remains challenging. We present Show-Harness, an Embodied Harness that enables VLMs to "play" robots through a compact semantic interface linking intent to action.
By Yanzhe Chen, Zechen Bai, Zhijun Cao · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.