NavGPT-3: Harnessing Context in a Hierarchical Navigation Runtime
NavGPT-3 pairs an 8B vision-language policy with a hierarchical runtime and matches human navigation on RxR-CE.
NavGPT-3 connects a language-model reasoner to a low-latency action policy through an OS-like runtime that schedules reasoning, acting, and monitoring as separate threads. The 8B NavGPT VLA, trained on 19.28 million examples, alone reaches 74.51 success rate on R2R-CE and leads RxR-CE at 78.19. With the full harness, the system sets 81.51 SR on R2R-CE and matches reported human followers on RxR-CE at 90.43 versus 90.4 SR and 78.47 versus 77.7 nDTW, in 1 minute 22 seconds per episode. Minimum reaction time drops from 3-19 seconds per language-model decision to 0.5-1 second per policy step, and the authors plan to release models, code, and evaluations.
- 8B NavGPT VLA trained on 19.28 million examples
- Policy alone scores 74.51 SR on R2R-CE and 78.19 on RxR-CE
- Full harness reaches 81.51 SR on R2R-CE
- RxR-CE result matches humans: 90.43 versus 90.4 SR
- Reaction time falls to 0.5-1 second per action-policy step
- AI models
- NavGPT-3NavGPT VLA
From the original article
1 min read at huggingface.co
Language models trained with long-horizon agentic reinforcement learning can generalize knowledge through reasoning, express precise actions, and pursue goals over many steps, raising the ceiling on what an embodied agent can understand and decide. Physical interaction, however, remains the domain of action policies, which provide dense, low-latency control.
By Gengze Zhou, Yicong Hong, Jiazhao Zhang · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.