OpenFlyV2 flies a quadrotor onboard from vision and language
Original title: OpenFlyV2: An onboard vision-language-action model for aerial perception and navigation
Authors report 48% zero-shot navigation success in four real environments with no real aerial training data.
OpenFlyV2 is an onboard vision-language-action model for aerial perception and navigation that shares one vision-language representation for semantic understanding, grounded perception, spatial reasoning, and continuous control. A compact action expert emits short-horizon waypoints, with temporal memory and action priors to cut closed-loop cost. The authors built a simulation dataset of about 70,000 navigation trajectories and 60,000 instruction–response pairs. On an NVIDIA Jetson Orin NX it produces 12-waypoint chunks at 10 Hz and, they report, transfers zero-shot to a physical quadrotor at a 48% success rate across four real environments. Code is planned at the AeroBrain GitHub repository.
Unconfirmed: The 48% real-world success rate is the authors' result; the simulation dataset name is missing from the text.
- About 70,000 simulated trajectories and 60,000 instruction pairs.
- Runs entirely onboard on an NVIDIA Jetson Orin NX at 10 Hz.
- Each action chunk contains 12 waypoints.
- 48% success across four real environments, authors say.
- No real-world aerial training data was used for the transfer.
- Vendors
- NVIDIA
- Products
- Jetson Orin NX
- AI models
- OpenFlyV2
From the original article
1 min read at huggingface.co
Human cognition flexibly draws on a shared perceptual foundation to understand environments, attend to referenced entities, reason about spatial relationships, and act.
By Junli Liu, Zhongrui You, Chenhui Li · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.