arXiv shows gradient inversion attacks on federated LoRA ViTs
Original title: Breaking the Privacy Promise: Gradient Inversion Attacks on Federated Low-Rank Adapted Vision Transformers
LoRA-adapted Vision Transformers weaken gradient signals allowing private data reconstruction.
Researchers investigate data leakage risks in federated learning with LoRA-adapted Vision Transformers. Low-rank updates reduce gradient signal, enabling reconstruction attacks with two coupled ambiguities. They propose a spatially anchored generative inversion framework incorporating positional embeddings to improve reconstruction quality.
- PSNR improves by 10.14 dB with proposed method on CIFAR-10
- SSIM improves by 0.45 and LPIPS by 0.098
- Gains maintained at higher batch sizes in FedSGD and FedAvg
- Freezes positional embeddings create missing spatial cues
- LoRA alone does not prevent data leakage in federated learning
From the original article
1 min read at arxiv.orgParameter-efficient fine-tuning (PEFT) has been increasingly adopted to fine-tune pretrained models with low computational cost and is therefore also preferred in the federated learning paradigm to reduce computational and communication overhead.
By Ritik K. Badiya · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.