VITAL: From Visual Motion to Precise Action
VITAL turns visual motion into driving actions, scoring 90.8 PDMS on NAVSIM with one denoising step.
VITAL is an action tokenizer for world-action driving models that couples a visually derived motion scaffold with a compact residual encoding centimeter-scale trajectory error. A flow-matching action expert predicts both from the current image, ego state, and navigation command, without generating future images at inference. With one denoising step it reaches 90.8 PDMS on NAVSIM v1 navtest, within 0.13 of its ten-step score at 3.29x speed, and scores 91.1 EPDMS on NAVSIM v2 and 38.6 on navhard without retraining.
- VITAL derives driving actions from visual motion plus a residual trajectory correction.
- One denoising step scores 90.8 PDMS on NAVSIM v1 navtest, 0.13 below ten steps.
- Inference is 3.29x faster end to end without generating future images.
- Without retraining it scores 91.1 EPDMS on NAVSIM v2 and 38.6 on navhard.
From the original article
1 min read at huggingface.co
World-action models aim to turn visual anticipation into driving decisions, yet typically train action experts on trajectory representations defined independently of visual dynamics. This motivates deriving action targets directly from visual motion. We first examine a simpler setting in which trajectories are reconstructed from recorded future images.
By Ao Liang, Youquan Liu, Mingxiao Li · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.