SparseBleed infers attributes from accelerator sparsity timing
One query’s timing recovers Sex at 66.39% and Surgical Procedure at 99.8% on Texas-100X, without gradients or weights.
Stories per day · 30 days
1 RSS
Relatedtransformers3dinov32elastic-inference2pruning2attribute-inference1continual-learning1efficiency1hardware1mixture-of-experts1pretraining1side-channel1timing-attack1vision-transformer1vision-transformers1
11 stories, newest first · one card per story
Stories tagged #sparsity
Nestor nested sparsity beats MoE transformers at equal compute
Attention-side sparsity nests with mixture-of-experts sparsity and adapts to new domains with less forgetting.
Your Transformer Is Secretly Elastic
A retraining-free method turns transformers into elastic models that prune smoothly up to 60% sparsity.
Your Transformer Is Secretly Elastic
A retraining-free method makes pretrained transformers elastic, pruning smoothly to 60% sparsity with small accuracy loss.
SparseDecoding: Decoding-Aware Pruning for Accurate and Efficient LLM Inference
SparseDecoding prunes LLMs using decoding activations, speeding generation up to 1.48x.
Sparsifying Stochasticity, Not Capacity: Partial Stochasticity via Deep Weight Factorization of Prior Scales
A Bayesian network method sparsifies stochastic parameters via prior-scale factorization while matching fully stochastic accuracy.
Scaling Laws for Looped Mixture of Experts
New scaling laws jointly model looped recurrence and MoE sparsity, matching larger models at equal compute.
ID Balancing: Stable Training of Extremely Sparse MoE via PID-Based Load Control
ID Balancing uses an integral-derivative controller to stabilize extremely sparse MoE expert load during training.
Scaling Laws for Looped Mixture of Experts
New scaling laws jointly model recurrence and sparsity, predicting looped MoE loss and guiding compute-efficient design.
GeoPair: Geometry-Preserving Cross-Layer Factorization for Training-Free Transformer Compression
GeoPair compresses transformers training-free by pairing compatible layers and sharing factorized dictionaries.
Not All Objectives Are Born Equal: Priority-Constrained Descent for Hierarchical Multi-Objective Optimization
Priority-Constrained Descent optimizes hierarchical objectives while limiting distortion of the primary gradient.