HiSTEVE 1.0 uses a hierarchical VLA agent for open-world Minecraft
Original title: HiSTEVE 1.0: Harnessing Human-Interface VLA Agents for Open-World Games
It scores 85.0% on MineStudio Bench and 37.4% average success on OpenHA, and can craft an iron pickaxe from scratch.
HiSTEVE is a hierarchical vision-language-action agent with a capability-aware harness for Minecraft. A planner and structured memory select subtasks, while one VLA controls all of them through vision-only perception and native keyboard-and-mouse actions. A policy-coupled detector uses VLA context to mark candidate subtask completion. Training combines world-knowledge injection, imitation learning, and reinforcement learning with environment feedback.
- MineStudio Bench score is 85.0%, described as state-of-the-art atomic-task performance.
- Average success rate on OpenHA is 37.4%.
- Regime-specific RMSNorm branches separate GUI and non-GUI interactions.
- Separate mouse-movement token embeddings are added at the control level.
- Products
- MinecraftMineStudio BenchOpenHA
- AI models
- HiSTEVE
From the original article
1 min read at huggingface.co
Building virtual embodied agents that operate in open-ended games like human players is a long-standing goal requiring both long-horizon autonomy and reliable native-interface control. As a procedurally generated open world, Minecraft provides a natural testbed for studying the integration of these capabilities.
By Zecong Tang, Zian Guan, Yuncheng Guo · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.