Visual perturbations decouple actions from identical rationales in VLAs
Original title: Say the Same, Act Differently: Diagnosing and Steering Text-Action Divergence in Reasoning VLAs
Alpamayo and InstructVLA show TNAS steering shifts actions 4.64x-47.8x more than text perturbations while preserving rationales.
Visual perturbations can alter physical actions in reasoning VLAs while keeping generated rationales word-for-word identical. The text-neutral action-subspace (TNAS) diagnostic reveals this decoupling in Alpamayo and InstructVLA. TNAS-guided attacks induce new collisions in 52.6% of scenes compared to baselines. An unchanged rationale therefore cannot certify a safe action.
- TNAS shows action-sensitive direction orthogonal to text gradients
- Interventions shift actions 4.64x to 47.8x more
- Rationale preservation increases to 89.5% with TNAS
- Cuts text drift by 68.4% at comparable collision rate
- AI models
- AlpamayoInstructVLA
From the original article
1 min read at huggingface.co
Reasoning vision-language-action (VLA) models pair physical control with natural-language rationales that supervisors can read to monitor embodied safety. Prior attacks on VLA reasoning corrupt the rationale itself and leave visible evidence in the text. We show that visual perturbations can instead alter the physical action while keeping the generated rationale word-for-word identical.
By Zihao Feng, Qingzhao Zhang, Chunyu Xia · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.