VLA-Tug probe improves failure detection in VLAs
Original title: Failure Prediction and Classification in Vision-Language-Action Models Via Calibrated Latent Probing
VLA-Tug uses latent probing to detect and classify failures in VLAs, improving early detection by 30% and accuracy on unseen tasks by 44%.
The paper presents VLA-Tug, a lightweight probe over VLA latent layers for early failure detection. It supervises on pre-failure segments to bound false alarms. Under distribution shift, it maintains valid bounds in 86% of runs. It predicts failure modes and enables corrective actions.
- Supervising on pre-failure trajectory boosts early detection by 30%
- Bounds on missed-detection and false-alarm rates hold in 86% of shift runs
- Predicts failure mode with up to 44% higher top-3 accuracy on unseen tasks
- Enables lightweight steering to improve task success
- Releases dataset of 3,180 annotated manipulation episodes
From the original article
1 min read at arxiv.orgVision-Language-Action models (VLAs) enable general-purpose embodied intelligence, but their performance is brittle under distribution shift, where unanticipated failures can damage equipment or endanger people. Deploying VLAs in real-world systems therefore requires a runtime monitor that is timely, calibrated, and informative while being efficient enough to run on the edge.
By Michal P. Podolinsky · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.