MintAct: A Unified Visual Agent for Digital Environments
MintAct vision-language models at 2B, 4B, and 8B unify UI grounding, multi-step navigation, and visual tool use, matching per-domain specialists.
MintAct is a family of vision-language models trained at 2B, 4B, and 8B scales that unifies UI grounding, multi-step navigation across mobile, desktop, and web, and visual tool use. The work builds scalable environment and RL infrastructure hosting hundreds of concurrent instances across heterogeneous per-domain backends for trajectory collection and online RL. An asynchronous RL framework controls cross-domain training distribution under noisy feedback and off-policy drift. MintAct achieves state-of-the-art 48.9 on OSWorld-Verified across a wide range of benchmarks at comparable model sizes.
- Single model family covers grounding, navigation, and tool use at 2B/4B/8B
- Matches per-domain specialist models across capabilities
- State-of-the-art 48.9 on OSWorld-Verified
- Asynchronous RL framework handles heterogeneous domains and off-policy drift
Full article137 words · extracted from huggingface.co · click to collapse
We present MintAct, a family of vision-language models that unifies UI grounding, multi-step navigation across mobile, desktop, and web, and visual tool use, trained at 2B, 4B, and 8B scales. Through careful design of our environments, data, and training recipes, MintAct models match the performance of per-domain specialists across all of these capabilities. To enable this, we develop a scalable environment and reinforcement learning (RL) infrastructure. On the environment side, we host hundreds of concurrent instances across heterogeneous per-domain backends, serving both trajectory data collection and online RL. To enable efficient and scalable RL training, an asynchronous framework keeps explicit control over the cross-domain training distribution and remains stable under noisy environment feedback and off-policy drift. Experimental results show that MintAct achieves state-of-the-art performance (48.9 on OSWorld-Verified) across a wide range of benchmarks at comparable model sizes.
Text extracted automatically; images, tables and formatting may be missing. Original: https://huggingface.co/papers/2609.22083