Adaptive Training Framework Cuts GUI Agent Rollouts by 56-61%
Original title: Where to Learn and How Far to Roll Out: Efficient Adaptive Training for GUI Agents
The two-stage method uses reusable checkpoints and action disagreement to improve RL efficiency on MobileGym and CUA-Gym-Hub.
Trajectory-level GRPO is inefficient for multi-step GUI agents as task-level rewards provide coarse supervision. The authors introduce a two-stage adaptive training framework that constructs reusable intermediate checkpoints from expert trajectories and selects informative pivots using action disagreement. Stage I performs one-step RL at these pivots to improve uncertain local decisions. Stage II progressively expands policy responsibility through adaptive Local, Suffix, and Root training. This reduces environment interactions by 56-61% while improving task performance.
- Reusable intermediate checkpoints enable training from intermediate states without replaying preceding trajectories
- Action disagreement measures uncertainty directly from sampled GUI actions
- Method tested on MobileGym and CUA-Gym-Hub
- Jointly allocates training across states and rollout ranges
- 56-61% reduction in environment interactions
From the original article
1 min read at huggingface.co
Trajectory-level GRPO is a common approach for training multi-step GUI agents, but repeatedly rolling out complete trajectories from the task root can be inefficient: task-level rewards provide coarse supervision over which intermediate decisions require improvement, while already-mastered prefixes are repeatedly re-executed before reaching difficult states.
By Yuxiang Chai, Quyu Kong, Panrong Tong · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.