Black-Box Adversarial Patch Attacks on VLAs via Ancestor VLM Exploitation
Adversarial patches optimized only on ancestor VLMs substantially cut success rates of deployed vision-language-action models.
Vision-language-action models adapted from public pretrained VLMs inherit visual perception and instruction-conditioned grounding. The authors propose three black-box adversarial patch attacks—vision disruption, instruction-grounded semantic evidence suppression, and a joint two-phase curriculum—optimized only on the ancestor VLM. Tests across VLA families in simulation and on static real-world images show large task-success drops, strongest for precise localization and weak when adaptation rewrites shared representations or the action head smooths perturbations.
- Black-box patches are optimized only on the public ancestor VLM.
- Three attacks disrupt vision, grounded evidence, or both jointly.
- Task success drops most when precise localization is required.
- Effect nearly vanishes if adaptation rewrites visual-semantic features.
From the original article
1 min read at arxiv.orgVision-Language-Action models (VLAs) are increasingly deployed in safety-critical physical environments, yet their adversarial robustness remains poorly understood. Existing attacks typically assume white-box access or rely on surrogate VLAs, which rarely holds in real-world deployments.
By Xiaoyi Pang · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.