HarnessVLN: Unifying Training-Free Embodied Navigation through an Agent Harness
Researchers introduce HarnessVLN, a zero-shot training-free agent harness that sets new training-free SOTA on vision-language navigation benchmarks including R2R and HM3D.
HarnessVLN is a zero-shot, training-free framework for embodied vision-language navigation that coordinates perception, retrieval, grounding, navigation, recovery, and termination through a unified tool interface. It validates planner proposals against spatial evidence, geometric feasibility, and subgoal consistency, using hierarchical event memory and a persistent Spatiotemporal Graph that stores reusable spatial evidence and failure annotations. It reports success rates of 60.8% on R2R, 53.9% on RxR, 76.0% on HM3D-v2, and 59.3% on HM3D-OVON, surpassing prior training-free state of the art, with real-world humanoid deployment demonstrated.
- Zero-shot, training-free harness validates planner actions against spatial evidence and geometric feasibility
- Spatiotemporal Graph stores reusable spatial evidence and failure annotations for recovery
- Success rates: R2R 60.8%, RxR 53.9%, HM3D-v2 76.0%, HM3D-OVON 59.3%
- Deployed on real humanoid robots for instruction-following and object-goal navigation
- AI models
- HarnessVLNMLLMs
From the original article
1 min read at huggingface.co
Embodied navigation requires agents to interpret visual observations, accumulate spatial knowledge, and execute actions to follow instructions or locate objects. Training-based methods face generalization challenges, while training-free methods exploit multimodal large language models (MLLMs) but often lack mechanisms to reconcile proposed actions with spatial evidence, task progress, and execution failures.
By Yang Chen, Lirong Che, Zhenyu Huang · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.