MAI-Gym: An Open All-in-One Platform for GUI Agent Evaluation and RL Training across Mobile and Computer Environments
MAI-Gym unifies GUI-agent evaluation and online RL; its 9B agent scores 78.4% on MobileWorld.
MAI-Gym is an open platform that unifies GUI-agent evaluation, data collection, and online reinforcement learning across mobile, computer, and web environments. A shared rollout pipeline covers ten benchmarks and scales to 20,000 parallel instances, with synthesized apps, cross-app worlds, and execution-validated verifiers. The authors train MAI-Gym-9B, reporting 78.4% on MobileWorld and 76.2% on OSWorld-Verified, the best open GUI-agent results they cite under 10B parameters. RL in a mocked ticketing app transfers to a real phone, raising task success from 28.8% to 48.1%. They release the framework, 2,000 verified tasks, and the checkpoint.
- MAI-Gym unifies GUI-agent evaluation, data collection, and online RL across mobile, computer, and web.
- The platform supports ten benchmarks and scales to 20,000 parallel environment instances.
- MAI-Gym-9B scores 78.4% on MobileWorld and 76.2% on OSWorld-Verified.
- Mocked train-ticketing RL transfers to a physical phone, lifting success from 28.8% to 48.1%.
- AI models
- MAI-Gym-9B
From the original article
1 min read at huggingface.co
Online reinforcement learning (RL) with verifiable rewards has driven rapid progress in mathematics, coding, and tool use, but scaling this success to GUI agents remains challenging due to the high cost of interactive environments and the scarcity of realistic training tasks.
By Panrong Tong, Quyu Kong, Pengxiang Li · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.