OLMo 2 membership inference study finds limited success
Original title: From Training Exposure to Membership Evidence: A Controlled Study of OLMo 2
Controlled OLMo 2 1B training with matched document pairs shows reference-ratio AUC of 0.767 but detects only 3.8% members at 1% FPR.
Researchers re-ran the Allen Institute for AI's OLMo 2 1B mid-training stage after inserting 7,814 domain-matched document pairs. A coin flip determined which document went into each pair, with the official same-seed run serving as counterfactual. Reference-ratio membership inference reaches 0.767 AUC but catches just 3.8% of members at 1% FPR; 13 pairs are needed for 80% power to detect a collection at 5% significance. Reference-free attacks stay near chance, the full post-training recipe retains 89% of the effect, and substantial document-level variability limits individual detection.
Unconfirmed: Exact rates may vary by domain and reference choice.
- One exposure raises likelihood but rarely identifies the document.
- Reference-ratio attacks achieve 0.767 AUC but detect 3.8% members at 1% FPR.
- 13 matched pairs give 80% power at 5% level for collection detection.
- Reference trained with different seed recovers 92% of AUC advantage.
- Substantial document-level variability limits individual membership inference.
Impact: Membership inference privacy on language models
| Product | Affected |
|---|---|
| Allen Institute for AI OLMo 2 | 1B |
- Organizations
- Allen Institute for AI
- AI models
- OLMo 2
From the original article
1 min read at arxiv.orgWhether a language model was trained on a document matters for copyright, privacy, and benchmark contamination. Yet most membership-inference (MI) evaluations on language models collect members and non-members after the fact, so success may reflect distribution shift and failure is inconclusive.
By Ali Banaei · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.