Deduplication-while-Training: A Resilient Paradigm for Privacy-Preserving Cross-Client Deduplication in Federated Learning
DwT-FL deduplicates federated LLM training data concurrently, cutting recovery and join overhead by over 93%.
The paper introduces Deduplication-while-Training so privacy-preserving removal of cross-client duplicate LLM data runs alongside federated training rather than as a blocking preprocess. DwT-FL uses concurrent state claims and hot-cold dual-queue scheduling to continue through client disconnects and dynamic joins. Compared with a state-of-the-art scheme, failure-recovery overhead drops by up to 93.04% and dynamic-joining overhead by up to 94.18%. The design aims to reduce memorization and privacy risk in unstable federated environments.
- Deduplication runs concurrently with federated LLM training instead of beforehand.
- State claims and hot-cold queues tolerate disconnects and dynamic joins.
- Failure-recovery time falls by up to 93.04 percent versus prior work.
- Dynamic-joining overhead falls by up to 94.18 percent.
From the original article
1 min read at arxiv.orgCross-client duplicate data in large language model training corpora degrades the efficiency of federated learning (FL) while exacerbating model memorization and privacy risks. Privacy-preserving cross-client deduplication effectively mitigates this issue by eliminating duplicate training data. However, existing schemes all follow a "Deduplication-before-Training" paradigm.
By Rongxi Wang · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.