ZeroHour

Search: “statistics”

5 stories in the last 24h

Double descent is the principle of least action

A statistical mechanics analysis explains double descent: finite-time diffusion induces effective weight decay that regularizes models as parameters grow.

The paper models stochastic gradient-based training as a particle diffusing over the training-loss energy landscape at an induced temperature, sampling parameters via a Boltzmann distribution. Finite training time carries an effective weight decay, making every parameter a quadratic degree of freedom governed by the equipartition theorem. Adding parameters at fixed training loss lowers the temperature and the L2 norm of the stationary path, increasing effective regularization and explaining the double descent phenomenon.

arXiv cs.AI / cs.LG / cs.CL · 15h agoAI research

Exponential Hardness of Off-Policy Evaluation under History-Dependent Logging

Researchers prove off-policy evaluation under history-dependent logging requires exponentially many episodes, resolving a hardness question for model-based POMDP evaluation.

The paper constructs POMDPs with at most two latent states per stage, three actions, and a three-memory-state logger where evaluating a known deterministic target policy to accuracy 1/8 requires Θ((3/2)^H log(1/δ)) episodes for any horizon H≥3. Coverage and outcome-revealing conditions hold with constants independent of H, yet a reset erases the unknown transition that determines the target value. The authors characterize the resulting statistical experiment exactly, derive a matching optimal estimator, and validate predictions on a two-lane gridworld. This settles the history-dependent-logging, model-based case posed by Zhang and Jiang (arXiv:2503.01134).

arXiv cs.AI / cs.LG / cs.CL · 14h agoAI research

Flag Game: A Toy Model for Mechanistic Swarm Interpretability

Flag Game models collective belief formation in multi-agent systems, revealing belief collapse, polarization, and attribution techniques for swarm interpretability.

The paper introduces the Flag Game, a toy model where bounded agents observe only private crops of a hidden country flag and exchange beliefs while weighing social evidence. It reproduces non-monotonic performance scaling with population size, collective belief collapse at small populations, and polarization at large ones that drives performance decline. The authors propose social circuit attribution with causal agent-patching interventions, and a statistical-mechanical theory that matches the empirical phase diagram, as first steps toward mechanistic swarm interpretability for collective alignment.

Low-Rank Masking for Single-Server Matrix Multiplication

Researchers prove rank-r additive masks for outsourced matrix multiplication achieve maximal-correlation secrecy of at most q^-r, with a matching lower bound.

An arXiv paper analyzes statistical privacy for outsourcing matrix multiplication over a finite field to a single server using additive masks of rank at most r. Uniform rank-ball masks and products of independent uniform factors yield maximal-correlation secrecy bounded by q^{-r}, with encoding and decoding costing O(n^2 r) field operations. The authors prove an asymptotically matching lower bound for r=o(n), showing these samplers are optimal among input-independent additive masks even with secret invertible transformations. They also show every such mask requires delta approaching 1 in entry-level (epsilon, delta)-differential privacy for fixed field size.

arXiv cs.CR · 16h agoResearch

Normal Alignment: Improved Cryptanalytic Sign Recovery on Hard-Label Networks

Researchers propose Normal Alignment, improving cryptanalytic sign recovery for hard-label neural networks and enabling polynomial-time full model extraction.

The paper improves on Carlini et al.'s EUROCRYPT 2025 cryptanalytic extraction of hard-label (S1) DNNs, whose Future Toggle sign-recovery method offered only marginal advantage over random guessing and triggered exponential-time enumeration on errors. Normal Alignment infers neuron signs via expected length differences between projected normals of adjacent decision facets at dual points, delivering higher voting accuracy and low-confidence errors. Combined with the SOE extension, it achieves exact polynomial-time full sign recovery: CIFAR-10 (192-64x8-10) and MNIST (64-96x3-32-10) models are fully recovered where the prior method required 2^52 or 2^82 sign guesses.

arXiv cs.CR · 18h agoResearch