VeilTune fine-tunes LoRA adapters entirely inside secure MPC
Original title: VeilTune: Secure Fine-Tuning of Private LoRA Adapters on Public Pretrained Language Models
From one 32-sample mini-batch of hidden states, authors copied a provider adapter on 99.2% of test inputs.
The paper says existing secure fine-tuning protects samples but reveals hidden states or logits from which a data owner can reconstruct the provider's adapter. From hidden representations of one 32-sample mini-batch, the authors fit an adapter agreeing on 99.2% of test inputs; logits alone yield about 94%. VeilTune runs backpropagation entirely in MPC, nearly matching plaintext training on four RoBERTa-large few-shot tasks and averaging 5.3 points above SecP-Tuning.
Unconfirmed: Extraction rates and speedups are author-reported and not independently verified.
- Hidden states from 32 samples copied the adapter on 99.2% of test inputs.
- Logits alone reproduced agreement on about 94% of test inputs.
- On SST-2, VeilTune is 3.5x faster and sends 3.6x less than SecP-Tuning.
- Versus full MPC fine-tuning, it is 26x faster and 17x lower in communication.
- Products
- LoRA
- AI models
- RoBERTa-large
From the original article
1 min read at arxiv.orgFine-tuning public pretrained language models with lightweight adapters such as LoRA has become the mainstream way to adapt them to downstream tasks. A service provider often trains its adapter on a data owner's samples, so two assets need protection: the data owner's samples and the provider's adapter.
By Huiyu Xie · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.