OS-Helix trains computer-use agents to mix GUI and code
Original title: OS-Helix: Unifying GUI Interaction and Code Execution in Native Computer-Using Agents
OS-Helix-27B-RL reaches 66.44% on OSWorld-Verified, a further 42.9% relative gain after RL.
OS-Helix is an end-to-end multimodal computer-using agent that puts GUI actions and code execution in one action space. Prompt-only addition of a code tool degraded a GUI agent, so the authors train on hybrid Ubuntu tasks built from application-artifact composition and verifiable evaluators. Supervised fine-tuning is followed by online RL with Step-balanced GSPO. OS-Helix-27B and OS-Helix-35B-A3B improve 35.0% and 33.7% over matched Qwen3.5 hybrid baselines, with reported transfer on WindowsAgentArena.
- OS-Helix-27B and OS-Helix-35B-A3B gain 35.0% and 33.7% relatively.
- OS-Helix-27B-RL reaches 66.44% success after a 42.9% relative RL gain.
- Step-balanced GSPO aggregates importance by action step rather than tokens.
- Ablations report cross-platform transfer on WindowsAgentArena.
From the original article
1 min read at huggingface.co
Real desktop automation is neither purely visual nor purely programmatic. Agents must understand user-facing interfaces while efficiently manipulating files, scripts, and system state. However, native GUI/code coordination cannot be obtained by prompt-level tool composition.
By Bowen Yang, Kaiming Jin, Zichen Ding · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.