Edge vision-language models match detectors on physical patch evasion
Original title: A Unified Evaluation Framework of Physical Adversarial Robustness in the Edge VLM Era
A unified metric finds compact grounding VLMs can fail with confident wrong boxes, not only missed people.
A paper evaluates whether an adversarial patch worn on a person evades detection by compact grounding vision-language models and conventional object detectors. It introduces conditional attack success rate, counted only when the model still localizes the person under a matched gray occluder, and splits failures into omission, substitution, and displacement. The authors report VLMs can be as vulnerable as detectors, with failure mode tied more to output format than architecture family. Code and data are on GitHub.
- Only omission is visible downstream; substitution and displacement return confident boxes.
- Under distribution shift, omission-prone VLMs shifted toward displacement or substitution.
- The benchmark compares both model families on the same person-detection outcome.
- Code and data are at github.com/ericliang1/physical-adversarial-robustness.
From the original article
1 min read at arxiv.orgRecent advancements in Vision-Language Models (VLMs) enable end-to-end visual grounding via language prompts and generative decoding, demonstrating largely improved zero-shot and open-vocabulary detection capabilities. As a result, VLMs are rapidly being deployed at the edge in safety-critical real-world systems.
By Eric Liang · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.