Perturbation Probing: A New Diagnostic for the Fragility of LLM Safety
Unit 42 research shows LLM safety refusals concentrate in a thin neural layer, motivating external, multi-layered AI security controls.
Palo Alto Networks Unit 42 introduces Perturbation Probing, a diagnostic technique for measuring the fragility of LLM safety mechanisms. The research finds that safety refusal behavior is localized within a thin neural layer, implying small perturbations can undermine built-in refusals. The authors argue this motivates external, multi-layered security defenses on top of model-internal safety training.