Pretraining seeds affect jailbreak success more than data recipes in language models
Original title: Identically Trained Language Models Break on Different Jailbreaks
Identical pretraining runs with different seeds produce models with varying susceptibility to 91 jailbreak templates.
The study trains 150 language models from 25 data recipes, 3 pretraining seeds, and 2 post-training seeds. It finds high correlation in non-refusal rates for same pretraining (0.984) but lower for different seeds (0.811) or recipes (0.779). Pretraining runs act as fingerprints for identification.
- Correlation drops to 0.811 with different pretraining seed
- Pretraining fingerprints identify models from 3,211 prompts with 98% accuracy
- Best attack template consistent across seeds in only 27% of pairs
From the original article
1 min read at huggingface.co
Jailbreak benchmarks test one checkpoint and treat which jailbreaks break it as a property of its pretraining data and safety training. We ask whether another pretraining run of the same data, given the same safety training, would break on the same jailbreaks.
By Chi-Wei Chang, Mu-Tien Kuo, Kuan-Wei Lee · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.