ScaleNav pretrains language-conditioned robot navigation on internet video
Original title: ScaleNav: Scalable Internet Video Pretraining for Language-Conditioned Local Navigation
Authors report 86.5% success on adapted Short-Horizon OVON and 94.9% on HabitatGS-LN, then tested a real quadruped.
ScaleNav learns goal-directed visual dynamics from internet egocentric video and uses them to generate robot actions for language-conditioned local navigation. Training has three stages: instruction-conditioned video pretraining, action alignment on simulated action-labeled data, and joint fine-tuning. The authors say it beat evaluated baselines on a single-view Short-Horizon OVON adaptation and on HabitatGS-LN. They also deployed it on a real quadruped and present those results as support for real-world generalization.
Unconfirmed: Benchmark and real-robot results are author-reported and not independently verified.
- The robot must stop at a text target visible in its initial view.
- Action alignment trains the model to predict executable action chunks.
- Reported success was 86.5% on adapted Short-Horizon OVON.
- Reported success was 94.9% on the HabitatGS-LN benchmark.
- Code and videos are posted by the authors.
From the original article
1 min read at huggingface.co
We study language-conditioned local navigation, where a robot must navigate to a text-specified target visible in its initial observation and stop at the described location. End-to-end navigation policies require diverse training data to generalize across scenes, yet real-robot data is costly to collect and simulated data leaves a sim-to-real gap.
By Yihui Yang, Yuanhong Yu, Jiawei Wang · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.