Online RL framework trains UI agents on live iOS apps
Original title: Online Dense Reinforcement Learning with Planning-Grounding Agent for UI Navigation on Live IOS Apps
An 8B planning-plus-grounding agent nearly septuples the SFT baseline on Android and ties Gemini-3.1-Pro on real iOS apps.
Researchers describe an online reinforcement-learning setup that trains UI navigation agents on live iOS applications instead of static trajectories. A planning agent and a UI grounding model are optimized with PPO using dense step rewards and a terminal reward from an LLM-as-a-Judge, plus GiGPPO neighbor-conditioned value estimates. Training runs on hundreds of concurrent virtual iOS devices. The authors report that 8B RL agents beat SFT and terminal-only RL baselines on UINavBench, iOSWorld, and AndroidWorld, including a nearly sevenfold gain over SFT in Android out-of-distribution tests, and outperform Gemini-3.5-Flash while tying Gemini-3.1-Pro on real iOS apps.
- GiGPPO uses reward signals from visually similar states across concurrent trajectories.
- The paper reports nearly sevenfold Android generalization gain over the SFT baseline.
- Agents outperform Gemini-3.5-Flash and tie Gemini-3.1-Pro on real iOS apps.
- Infrastructure orchestrates hundreds of concurrent virtual iOS devices.
- AI models
- Gemini-3.5-FlashGemini-3.1-Pro
From the original article
1 min read at huggingface.co
While frontier vision-language models demonstrate strong capabilities in user interface (UI) navigation, their high inference costs prohibit everyday use. Conversely, smaller models trained via supervised fine-tuning (SFT) or offline reinforcement learning often memorize static trajectories, leading to brittle execution and an inability to recover from errors.
By Peitong Duan, Khashayar Gatmiry, Jeffrey Nichols · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.