SimVTA: Simulation-Grounded Video-Trajectory Alignment for Driving World Action Models
SimVTA raises a driving world model's NAVSIM v1 PDMS from 86.2 to 91.3.
SimVTA post-trains driving world action models by pairing trajectory optimization with simulator-rendered video supervision for alternative futures. Candidate trajectories are sampled, high-reward ones are rendered in 3D Gaussian Splatting reconstructions, and higher-error futures are prioritized under a limited rendering budget. Group Relative Policy Optimization updates planning, while a flow-matching loss on matched videos updates shared representations. On NAVSIM v1, PDMS improves from 86.2 to 91.3; the method also reports 88.2 EPDMS on NAVSIM v2 and 38.8 average HD-Score on HUGSIM.
- SimVTA renders videos for high-reward candidate trajectories.
- Scenes are reconstructed with 3D Gaussian Splatting.
- GRPO optimizes planning while flow matching updates shared representations.
- Inference predicts trajectories without calling the simulator.
- NAVSIM v1 PDMS rises from 86.2 to 91.3.
- Products
- NAVSIMHUGSIMWorld Engine
- AI models
- SimVTA
From the original article
1 min read at huggingface.co
Driving world action models (WAMs) learn from paired trajectories and future observations. Post-training with reinforcement learning allows the policy to explore alternative trajectories and improve planning using driving rewards. However, driving logs contain future videos only for the recorded trajectories, leaving these alternatives without corresponding visual supervision.
By Changhui Jing, Haochen Tian, Hongchen Li · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.