Paper estimates ReLU-net mutual information from routing over linear regions
The routing term uses a plug-in estimator and, when well sampled, approximates a lower bound on joint information.
Stories per day · 30 days
1 RSS
Relatedcryptography1federated-learning1mutual-information1piecewise-linear1relu1representation-learning1secure-aggregation1
8 stories, newest first · one card per story
Stories tagged #information-theory
Paper bounds secure aggregation cost when federated-learning users drop out
One adaptive scheme meets the optimal masked-gradient communication rate for every feasible dropout set under T-colluding security.
Verification with Transfer: Exact Information Frontiers and Their Price in Calls
Theory paper prices source-task calls that cut verification cost, with Lean-checked bounds and a transformer test.
The Price of Privacy: Randomness Complexity of Graph-Based Multi-Secret Sharing
The paper bounds randomness needed to share correlated secrets on graphs, including all four-party cases.
Conformal Prediction Sets Quantify Information Gain: A Theoretical Perspective
Paper shows conformal prediction set-size reduction tracks information gain and relates to Shannon mutual information.
Task-Sufficient Contraction: Source Selection for Machine Information Interfaces
A theory paper shows when a task-fixed source reduction preserves the full downstream rate-regret problem.
Private Information Retrieval With Arbitrary Privacy Requirements: Introduction and Capacity Results
Researchers formulate private information retrieval under arbitrary graph-based privacy requirements, deriving capacity bounds and introducing pyramid storage graphs.
The Price of Sparsity: Sufficient Conditions for Sparse Recovery using Sparse and Sparsified Measurements
Researchers derive sufficient sample-size conditions for recovering sparse binary signals from sparse Gaussian measurements, quantifying an information-theoretic threshold of order slog(p/s)/log(ds/p).