CodeMimicry: Exploiting Safety Generalization Lag in Large Language Models via Structured Code Completion
CodeMimicry uses structured code-completion prompts to jailbreak eight commercial LLMs, reporting a 96.25% success rate.
The paper identifies safety generalization lag, in which alignment trained mostly on natural language does not transfer to code and leaves a code-completion blind spot. CodeMimicry is a fully automated black-box method that places harmful intent inside syntactically valid object-oriented code prompts and elicits harmful outputs through code completion. Experiments on eight commercial large language models report a 96.25 percent attack success rate with 1.51 queries on average, above template-based and optimization-based baselines. The authors analyze refusal-related directions and activation steering and argue that safety alignment should cover structured domains such as code.
- Alignment trained mainly on natural language fails to transfer to code, a safety generalization lag.
- CodeMimicry is an automated black-box method using structured code-completion prompts.
- Authors report 96.25 percent success on eight commercial LLMs, averaging 1.51 queries.
- Latent-space analysis covers refusal directions and activation steering in code domains.
From the original article
1 min read at arxiv.orgLarge language models have achieved remarkable capabilities across diverse domains, yet their safety alignment remains vulnerable to jailbreak attacks. In this work, we identify a previously underexplored failure mode - safety generalization lag - where alignment trained predominantly on natural language fails to transfer to the code domain.
By Zhen Liang · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.