HybridCUA: Learning to Orchestrate GUI and CLI for Computer-Use Agents
HybridCUA-9B combines GUI and CLI actions, reaching 53.6% on OSWorld, up 14.8 points.
HybridCUA trains computer-use agents to combine graphical interface actions with shell commands rather than relying on the GUI alone or application-specific APIs. Its data pipeline produces HybridCUA-8K, with 5,000 hybrid trajectories and 3,000 verified RLVR tasks spanning GUI-only, CLI-only, and interleaved behavior. Training uses supervised fine-tuning followed by reinforcement learning with CLI-aware rewards. HybridCUA-9B reaches 53.6% on OSWorld, 14.8 points above the base model, and gains 4.0 points on WindowsAgentArena.
- Teaches agents when to use the CLI instead of only the GUI
- HybridCUA-8K holds 5,000 hybrid trajectories and 3,000 RLVR tasks
- Training is supervised fine-tuning then CLI-aware reinforcement learning
- HybridCUA-9B gains 14.8 points on OSWorld and 4.0 on WindowsAgentArena
From the original article
1 min read at huggingface.co
Computer use agents (CUAs) have demonstrated strong capabilities in completing digital tasks. However, existing CUAs either rely solely on graphical user interface (GUI) interactions, which are often inefficient and error prone, or augment GUI interactions with application specific APIs or tools, which require substantial engineering effort and are difficult to scale across applications.
By Tongbo Chen, Junbo Niu, Zhengxi Lu · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.