ASCENT: First-Order Optimal Fine-Tuning with Recalibration for Safety--Utility Co-Enhancement
ASCENT fine-tunes LLMs with periodic safety calibration, raising utility up to 20.3% and cutting attack success up to 35.5%.
ASCENT is a supervised fine-tuning method that jointly improves LLM safety and utility using first-order optimal safety calibration and task updates. Under a fixed rank and Frobenius-norm budget, the top-r singular components of the safety-function gradient are used for periodic recalibration, while task updates stay close to the original update but penalize estimated safety loss. Across multiple LLM families and downstream tasks, utility rose by up to 20.3% and attack success rate fell by up to 35.5%. Code is released in the ZJU-LLM-Safety/ASCENT repository.
- Models safety as a first-order function of LLM parameters.
- Top-r singular components of the safety gradient maximize estimated safety change.
- Alternates safety-preserving task updates with periodic calibration.
- Raises downstream utility by up to 20.3 percent.
- Cuts attack success rate by up to 35.5 percent.
From the original article
1 min read at arxiv.orgSupervised fine-tuning can substantially improve the downstream utility of large language models (LLMs) but may compromise their safety.
By Weiwei Qi · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.