HarnessVLN: Unifying Training-Free Embodied Navigation through an Agent Harness
Researchers introduce HarnessVLN, a zero-shot training-free agent harness that sets new training-free SOTA on vision-language navigation benchmarks including R2R and HM3D.
HarnessVLN is a zero-shot, training-free framework for embodied vision-language navigation that coordinates perception, retrieval, grounding, navigation, recovery, and termination through a unified tool interface. It validates planner proposals against spatial evidence, geometric feasibility, and subgoal consistency, using hierarchical event memory and a persistent Spatiotemporal Graph that stores reusable spatial evidence and failure annotations. It reports success rates of 60.8% on R2R, 53.9% on RxR, 76.0% on HM3D-v2, and 59.3% on HM3D-OVON, surpassing prior training-free state of the art, with real-world humanoid deployment demonstrated.
- Zero-shot, training-free harness validates planner actions against spatial evidence and geometric feasibility
- Spatiotemporal Graph stores reusable spatial evidence and failure annotations for recovery
- Success rates: R2R 60.8%, RxR 53.9%, HM3D-v2 76.0%, HM3D-OVON 59.3%
- Deployed on real humanoid robots for instruction-following and object-goal navigation
Full article178 words · extracted from huggingface.co · click to collapse
Embodied navigation requires agents to interpret visual observations, accumulate spatial knowledge, and execute actions to follow instructions or locate objects. Training-based methods face generalization challenges, while training-free methods exploit multimodal large language models (MLLMs) but often lack mechanisms to reconcile proposed actions with spatial evidence, task progress, and execution failures. We present HarnessVLN, a zero-shot, training-free framework whose Agent Harness coordinates perception, retrieval, grounding, navigation, recovery, and termination through a unified tool interface. The Harness validates planner proposals against spatial evidence, geometric feasibility, and subgoal consistency, incorporating structured tool feedback into subsequent decisions. Hierarchical event memory tracks task progress and execution history, while a persistent Spatiotemporal Graph maintains reusable spatial evidence and failure annotations for verification and recovery. A replaceable Navigation Executor converts validated targets into executable motions, allowing the same Harness protocol to support instruction-following and object-goal navigation. HarnessVLN achieves success rates of 60.8%, 53.9%, 76.0%, and 59.3% on R2R, RxR, HM3D-v2, and HM3D-OVON, respectively, surpassing prior training-free SOTA results. Humanoid deployment further demonstrates its applicability to both tasks in real-world environments. The project page is: https://harnessvln.netlify.app/.
Text extracted automatically; images, tables and formatting may be missing. Original: https://huggingface.co/papers/2609.15195