SimpleLoRIF traces LLM safety degradation to fine-tuning data
Original title: Follow the Gradient: Tracing Safety Degradation Back to Training Data
SimpleLoRIF filters gradient-aligned unsafe examples from LLM fine-tuning datasets.
The paper develops SimpleLoRIF, a gradient embedding method that identifies safety-degrading fine-tuning examples by aligning with unsafe behavior. It validates the approach on emergent misalignment cases, achieving AUC up to 0.865. Removing the 20% most aligned examples produces SafeTulu, which lowers GCG attack success rates by 22% on tested models while preserving capability.
- SimpleLoRIF processes datasets of up to a million examples
- AUC reaches 0.865 on misaligned model gradients
- SafeTula reduces GCG success by up to 22% across Qwen, Llama, and Olmo
- Analysis identifies tool-execution failures and narrative arithmetic as safety sources
- Custom SafeTulu variants supported via Hugging Face Space
From the original article
1 min read at huggingface.co
Fine-tuning Large Language Models (LLMs) on instruction data can quietly erode their safety, even when no training example looks harmful. Because this safety-degrading data looks benign, standard content filters struggle to catch it. We tackle this problem by developing SimpleLoRIF, a scalable gradient embedding method that identifies fine-tuning examples whose gradients align with unsafe behavior.
By Saket Reddy, Amro Abbas, Anshuman Suri · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.