Paper shows utility-hijacking can poison instruction-data selectors
Original title: Characterizing and Defending Utility-Hijacking in Instruction-Tuning Data Selection
Generate-and-rerank keeps malicious records in top-k selection; CONCERT cuts that concentration without poison labels.
Researchers describe utility-hijacking, a supplier-side attack on utility-oriented instruction-data selectors used for fine-tuning. Harmful records are rewritten so they still score highly while keeping payloads for over-refusal, false facts, or insecure code. Across six selectors and Qwen models from Qwen2.5-7B to Qwen3-32B, they report refusal rising from 0% to 94% and insecure code up to 56 of 90. Their CONCERT defense flags suppliers chosen far more often than chance and keeps only records that two dissimilar selectors both rank highly.
Unconfirmed: Harm and concentration figures are the authors' experimental results, not independently confirmed incidents.
- False-fact enrichment rose with scoring-model scale from 2.84x to 7.04x.
- On Qwen3-32B, CONCERT cut code poison concentration from 6.67x to 0x.
- Fact concentration fell from 3.59x to 0.67x, with harm down 75%.
- CONCERT needs no content detector or poison labels and keeps the same budget.
Impact: Poisoned fine-tuning data can pass utility selectors
| Product | Affected |
|---|---|
| Alibaba Qwen2.5-7B | — |
| Alibaba Qwen3-32B | — |
- 1.Re-select training data with two dissimilar selectors under the same budget.
From the original article
1 min read at arxiv.orgInstruction-data selectors choose which instruction-following records are used for fine-tuning, improving alignment quality, raising training efficiency, and filtering out harmful records. A selector's choices thus directly affect the fine-tuned model's safety, yet despite their widespread use, the security of these selectors remains under-studied.
By Yahao Ren · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.