Show-Harness: Just a VLM Agent Can Play Robots

Show-Harness lets VLM agents control robots via discrete semantic action units that embodiment-specific interpreters deterministically ground into local robot actions, achieving zero-shot closed-source frontier VLM control and adaptation of small open-source…

Show-Harness is an embodied agent harness that exposes discrete semantic action units for a VLM to reason over, while embodiment-specific interpreters deterministically ground those units into local robot actions, keeping the VLM responsible for decisions (Report 1 describes this as responsibility for fine-grained physical decisions). It enables zero-shot robot control with closed-source frontier VLMs and low-cost deployment of small open-source VLMs after only a few GPU-hours of fine-tuning. A companion interface, GUMI (GUI Manipulation Interface), extends the same semantic action space to GUI-based demonstration collection without teleoperation hardware (Report 2 specifies 'specialized' teleoperation hardware and adds that it supports both human and agent demonstration collection across embodiments). Both reports claim robust generalization across tasks, embodiments, and environments, with Show-Harness-equipped agents outperforming representative agentic and VLA paradigms; Report 2 specifies these gains come without added model capacity or embodiment-specific pretraining. Neither report provides quantitative results (no benchmark scores, success rates, or task counts).

• Paper: 'Show-Harness: Just a VLM Agent Can Play Robots' - surfaced via Hugging Face daily papers on 2026-09-08 and arXiv cs.AI/cs.LG/cs.CL on 2026-09-09.
• Mechanism: the VLM reasons over discrete semantic action units; embodiment-specific interpreters deterministically ground them into local robot actions.
• Report 1: the semantic interface keeps VLMs responsible for fine-grained physical decisions.
• Enables zero-shot closed-source frontier VLM robot control (both reports).
• Small open-source VLMs can be adapted for low-cost deployment with only a few GPU-hours of fine-tuning (both reports).
• GUMI (GUI Manipulation Interface) extends the same semantic action space to GUI-based demonstration collection: Report 1 says without teleoperation hardware; Report 2 says without specialized teleoperation hardware and adds human and agent…
• Claimed result: Show-Harness-equipped agents outperform representative agentic and VLA paradigms across tasks, embodiments, and environments; Report 2 adds this is achieved without added model capacity or embodiment-specific pretraining.
• Neither source gives quantitative results (e.g., benchmark scores, success rates, or evaluation counts).