OPTED: On-Policy Fine-Tuning for End-to-End Driving using a Render-Free Teacher
OPTED decouples RL teacher training from closed-loop student post-training for end-to-end driving, raising TransFuser and VaVAM driving scores 1.6x and 9.5x.
OPTED trains a privileged teacher with reinforcement learning on vectorized HD-map and bounding-box inputs, then uses it to supervise pre-trained camera-based students during closed-loop post-training without costly sensor simulation. Applied to TransFuser and VaVAM in AlpaSim using 3DGS neural reconstructions of real driving logs, driving scores increased 1.6x and 9.5x respectively. In controlled experiments it matched direct RL post-training performance with roughly three orders of magnitude fewer simulator interactions.
- Privileged RL teacher on vectorized inputs avoids sensor simulation
- Driving scores up 1.6x for TransFuser and 9.5x for VaVAM
- Uses 3DGS reconstructions of real driving logs in AlpaSim
- About 1000x fewer simulator interactions than direct RL post-training
- Products
- TransFuserVaVAM
- AI models
- OPTED
From the original article
1 min read at arxiv.orgAs scaling pre-training data alone yields diminishing returns, post-training is becoming increasingly important across physical AI domains such as autonomous driving. End-to-end driving policies are pre-trained in open loop with behavior cloning on human demonstrations.
By Damiano Da Col · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.