Safety mid-training lowers LLM attack success before post-training
Original title: Safety Mid-Training: Internalizing LLM Safety as a Foundational Capability
Authors report a 72% to 81% relative attack-success drop versus Qwen3 1.7B, 4B, and 8B bases.
The paper adds a safety mid-training stage between pre-training and post-training so safety and general reasoning share one structure. A base model generates structured documents and reasoning traces from safety and utility prompts for that stage. Relative to Qwen3-1.7B, 4B, and 8B base models, the authors report a 72% to 81% lower average attack success rate and improved general capability. They say the safety gain largely remains after general-only supervised fine-tuning and GRPO without safety rewards.
Unconfirmed: Attack-success figures are the authors' evaluations, not an independent audit.
- Mid-training uses model-generated documents and reasoning traces.
- After general-only SFT, mid-trained models still show lower attack success.
- Safety is also retained under general-only GRPO without safety rewards.
- Later reinforcement learning from mid-trained models improves the safety-utility frontier.
- AI models
- Qwen3-1.7BQwen3-4BQwen3-8B
From the original article
1 min read at arxiv.orgSafety alignment for large language models (LLMs) is primarily performed during post-training, while recent work has also begun incorporating safety data into pre-training. Yet safety can degrade when later training no longer includes safety data, and gains in safety often come at the cost of general capability. This tension suggests that safety and general reasoning should not be treated as separate training goals.
By Zhengyue Zhao · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.