RynnVLA-Latent pretrains a 4B robot policy on 3D latent actions
Original title: RynnVLA-Latent: Scaling Robotic Manipulation Foundation Models with 3D-Grounded Latent Actions
A 4B policy pretrained on 745.52M descriptors from 30.1K hours of video posts strong LIBERO, VLABench, and RoboTwin scores.
RynnVLA-Latent learns robot policies from heterogeneous videos by predicting continuous, 3D-grounded latent actions instead of native robot controls. RynnLAM extracts those descriptors with residual future-feature prediction, camera-compensated 3D-flow supervision, and camera-motion decoupling. Applied to 30.1K hours of human, robot, and simulation video, it yields 745.52M latent-action descriptors. The 4B policy then predicts latent-action chunks via flow matching and adapts to embodiment-specific actions, reporting high success on LIBERO, LIBERO-Plus, VLABench, RoboTwin 2.0, and six real-world tasks.
Unconfirmed: Real-world success is reported only relative to the compared methods, without per-task rates.
- RynnLAM was applied to 30.1K hours of human, robot, and simulation video.
- The pipeline produced 745.52M latent-action descriptors.
- The 4B policy reached 98.8% on LIBERO and 84.7% on LIBERO-Plus without benchmark-specific adaptation.
- Average success was 51.9% on five VLABench tracks and 88.6% on RoboTwin 2.0.
- It recorded the highest observed success among compared methods on six real-world tasks.
From the original article
1 min read at huggingface.co
Scaling vision-language-action (VLA) models is constrained by scarce action-labeled demonstrations and incompatible control spaces across robot embodiments. We present RynnVLA-Latent, a framework that turns heterogeneous interaction videos into scalable supervision for robot policy pretraining.
By Haoyu Zhao, Xingyue Zhao, Zhian Su · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.