Paper recovers neural-network parameters by extraction and distillation
Original title: Extraction meets Distillation - cryptanalytic techniques to recover parameters from a Deep Neural Network
On the largest synthetic net, Leaky ReLU recovered 51-57 deepest-layer neurons while ReLU recovered none.
The paper combines cryptanalytic weight-direction extraction with learning from predicted labels to reconstruct fully connected networks. It adapts ReLU extraction to Leaky ReLU, compares sign search, and estimates biases and output layers. Assisted extraction uses privileged gradients, hidden activations, and victim parameters, then label-based reconstruction. On two CIFAR-10 victims, reconstructed models reach 56.40% and 58.29% agreement and beat a distillation baseline on three metrics.
Unconfirmed: Extraction experiments use privileged access; black-box recovery without that access is not established.
- Leaky ReLU recovered 51-57 deepest neurons; ReLU recovered none on that architecture.
- That advantage does not hold uniformly across architectures and layers.
- On 40 runs, held-out agreement ranged from 0.9890 to 0.9992.
- Removing victim-weight access produced less accurate signs despite high prediction agreement.
- Authors say prediction agreement does not identify network parameters.
From the original article
1 min read at arxiv.orgWe study fully connected neural-network reconstruction by combining cryptanalytic weight-direction extraction with learning from predicted labels. We adapt a ReLU extraction implementation to Leaky ReLU, compare sign-search methods, and estimate biases and output layers.
By Biprarshi Biswas · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.