SoftwareGym: Scaling Computer-Use Agent Training with Diverse Synthesized Software Environments
SoftwareGym's 182 synthetic apps lift a computer-use agent to 56.97% OSWorld Pass@3.
SoftwareGym automatically generates structurally diverse GUI applications, implements them as runnable software, and uses GUI evaluation with iterative repair to verify behavior. The authors synthesized 182 applications across twelve categories, achieving a 91.5% mean GUI task pass rate, and built the 120-task SoftwareGym-Bench. Training a computer-use agent on trajectories from these environments raised strictly out-of-domain OSWorld Pass@3 from 41.39% to 56.97% and produced 40.0% on SoftwareGym-Bench.
- SoftwareGym synthesizes runnable, runtime-verified GUI applications.
- Authors built 182 apps across 12 categories with 91.5% mean pass rate.
- SoftwareGym-Bench contains 120 tasks in those categories.
- Trained agent reached 56.97% Pass@3 on out-of-domain OSWorld.
- The same model scored 40.0% on SoftwareGym-Bench.
From the original article
1 min read at huggingface.co
Computer-use agents (CUAs) are rapidly improving, yet synthesizing diverse and reliable software environments at scale remains challenging. Native application environments require substantial construction and integration effort, which limits application coverage and increases the risk of application-specific overfitting. Browser-hosted environments provide limited interaction depth and workflow complexity.
By Tian Jin, Wenhao Wang, Jing Kang · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.