MintAct: A Unified Visual Agent for Digital Environments
MintAct vision-language models at 2B, 4B, and 8B unify UI grounding, multi-step navigation, and visual tool use, matching per-domain specialists.
MintAct is a family of vision-language models trained at 2B, 4B, and 8B scales that unifies UI grounding, multi-step navigation across mobile, desktop, and web, and visual tool use. The work builds scalable environment and RL infrastructure hosting hundreds of concurrent instances across heterogeneous per-domain backends for trajectory collection and online RL. An asynchronous RL framework controls cross-domain training distribution under noisy feedback and off-policy drift. MintAct achieves state-of-the-art 48.9 on OSWorld-Verified across a wide range of benchmarks at comparable model sizes.
- Single model family covers grounding, navigation, and tool use at 2B/4B/8B
- Matches per-domain specialist models across capabilities
- State-of-the-art 48.9 on OSWorld-Verified
- Asynchronous RL framework handles heterogeneous domains and off-policy drift
From the original article
1 min read at huggingface.co
We present MintAct, a family of vision-language models that unifies UI grounding, multi-step navigation across mobile, desktop, and web, and visual tool use, trained at 2B, 4B, and 8B scales. Through careful design of our environments, data, and training recipes, MintAct models match the performance of per-domain specialists across all of these capabilities.
By Mingfei Gao, Rui Tian, Haiming Gang · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.