PanoVLN: Towards Effective Panoramic Vision-and-Language Navigation
PanoVLN uses panoramic views and a 4B backbone to raise VLN success rates.
PanoVLN studies vision-and-language navigation using panoramic observations rather than narrow perspective images. The authors add confidence-guided execution of longer action sequences, training routes with frequent branching, and combined semantic and geometric features from RGB panoramas without extra visual tokens. With a 4B backbone and RGB-only input, PanoVLN surpasses the previous state of the art by 11.9% and 8.7% success rate on R2R-CE and RxR-CE Val-Unseen. Real-world quadruped experiments show faster navigation with fewer pauses than prior VLN methods.
- Panoramas alone yield limited vision-and-language navigation gains.
- Confidence-guided execution chooses how many actions to run before replanning.
- A 4B RGB model beats prior SOTA by 11.9% and 8.7% success rate.
- Quadruped tests show faster navigation with fewer pauses.
From the original article
1 min read at huggingface.co
Recent vision-language models (VLMs) have advanced vision-and-language navigation (VLN), enabling models to predict navigation actions from visual observations and language instructions. In this work, we explore VLN with panoramic observations and introduce PanoVLN. The motivation is straightforward: more complete visual context should enable better-informed navigation decisions.
By Zhen Wang, Changpeng Wang, Zhe Liu · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.