Search: “gradient-leakage”

Smart search ranks by meaning as well as keywords (one row per story, last 45 days).

Privacy Failure in Split-LLM Training, The Returned Gradient Nullifies the Decoys

Researchers show split-LLM training leaks privacy via zero-valued gradients on decoy rows, exposing which activations are real despite passing forward-channel checks.

A systems-security case study of a two-node split-LLM training setup found that the returned output gradient from an Untrusted Cloud Node is exactly zero for decoy rows, revealing which rows are real. Across nine seeds, zero patterns identified real rows in 4,096 of 4,096 frames per run, and an attack on frame contents recovered 0.65 to 1.50 percentage points of extra tokens over a baseline. Both datasets passed forward-channel privacy and quality checks but failed once the returned gradient was included. Row-wise gradient clipping and noise closed the leak for roughly 0.01 nats of held-out cross-entropy, though five unmeasured attack classes remain.

Hugging Face daily papers·Sep 2, 20:00 UTC · 14d agoAI safety & security

58

Cascading Gradient Inversion via LT-Code Inspired Peeling in Federated Learning

New gradient inversion attacks tied to erasure-coding theory recover 94–100% of ImageNet batches, showing federated learning privacy leakage is underestimated.

The paper connects gradient inversion in federated learning to erasure-correcting code theory, constructing analytic attacks that exceed previously known recovery bounds. The attacks recover batches exactly, with every sample's label, from a single FedSGD round, and certify each recovery without ground-truth data. On eight image and tabular benchmarks, even a passive attacker observing an honestly trained network recovers 94–100% of ImageNet batches up to size 128, and more than 90% actively at batch sizes of several hundred. The authors conclude that federated learning's privacy leakage has been underestimated.

arXiv cs.CR·Sep 9, 03:17 UTC · 7d agoResearch

28

Anatomical Grounding and Leakage-Aware Multimodal Contrastive Learning for Alzheimer's Disease Classification from Structural MRI

Study of label leakage and anatomical grounding in multimodal MRI models for Alzheimer's staging shows cognitive-score fusion accuracy of 87.3% is leakage-driven.

The authors train a ResNet18 slice-based encoder with a one-layer Transformer on 1,075 ADNI-1 T1 MRI scans, using FastSurfer segmentations and YOLOv8 localization (mAP_50 above 0.96) as anatomical reference. Grad-CAM shows the image-only classifier often attends to skull and background rather than disease-relevant structures. A CLIP-style image-tabular contrastive framework organized along a label-leakage spectrum yields 87.3% three-way accuracy with cognitive scores versus 73.0% with regional volumes, and cropping to the medial temporal lobe raises image-only accuracy from 58.7% to 65.1%. Results come from single runs on a small balanced test set with reported confidence intervals.

arXiv cs.AI / cs.LG / cs.CL·Sep 14, 17:13 UTC · 2d agoAI research1

16

DRIFT: Removing Diffusion Watermarks by Deflecting the Generative Trajectory

Introduces DRIFT, a black-box attack removing diffusion watermarks by deflecting generative trajectories, achieving 98-100% success across nine watermarking schemes.

Researchers propose DRIFT, a black-box watermark removal attack combining partial forward diffusion with stochastic reverse resampling to break trajectory-dependent verification. The paper derives information-theoretic and Wasserstein source-dependence bounds and shows the first verifier-rejected rung is least distorted among rejected rungs. Across nine watermarks spanning three paradigms, DRIFT achieves 98-100% attack success with the best image quality among compared attacks, without secret keys, verifier internals, or per-image gradient optimization.

arXiv cs.CR·Sep 8, 03:54 UTC · 8d agoResearch

38

Target leakage, not model class, explains reported accuracy in survey-based cardiovascular screening: a leakage-tiered audit of glass-box and tabular foundation models

Audits of 10 classifiers on BRFSS show target leakage, not model class, drives the reported 0.89 AUROC in survey-based cardiovascular screening.

The study benchmarks ten model classes, including glass-box and tabular foundation models, for prevalent myocardial infarction on 442,067 respondents of the 2022 BRFSS across five feature tiers of decreasing leakage risk. Removing two post-diagnostic features costs every model 0.049-0.051 AUROC and collapses performance into a 0.0045-wide band, and the explainable boosting machine matches all alternatives within 0.005 while scoring roughly 104x faster than the strongest foundation model. Frozen models transport within 0.002 AUROC to 2023 data; the authors conclude evaluation practice and feature sets, not model capacity, are the binding constraint.

arXiv cs.AI / cs.LG / cs.CL·Sep 10, 17:29 UTC · 6d agoAI research1

28

General Quantification of Covariate and Concept Shifts

Paper proposes γ*-concept shifts via entropic optimal transport, deriving estimable generalization bounds unifying covariate and concept shift under distribution shift.

The authors show existing definitions of concept shift break when source and target supports mismatch and propose γ*-concept shifts grounded in entropic optimal transport. They derive a general error bound covering broad loss functions, label spaces and stochastic labeling, plus estimators with concentration guarantees. The resulting DataShifts algorithm quantifies distribution shifts and estimates the error bound in most applications, addressing learning bounds that were previously non-estimable from samples.

arXiv cs.AI / cs.LG / cs.CL·Sep 10, 17:57 UTC · 6d agoAI research1

28

Understanding the Impact of Model Pruning on Long-Tail Forgetting and Explanation Reliability in Medical Imaging

Systematic study finds model pruning causes frequency-dependent long-tail forgetting in medical imaging and that gradient-informed methods best preserve explanations.

Across two long-tailed medical imaging datasets, two CNN architectures, four pruning methods, and sparsity up to 95%, the study measures predictive performance, explanation stability, and faithfulness. Rare classes degrade earlier and more severely than frequent ones, while explanation reliability depends mainly on the pruning strategy, with gradient-informed methods degrading least. Mechanistic analysis ties explanation collapse to loss of class-discriminative gradients rather than vanishing feature activations, recommending class- and explanation-aware evaluation of compression.

arXiv cs.AI / cs.LG / cs.CL·Sep 7, 17:44 UTC · 9d agoAI research

27

Quantile-based Loss Filtering for Outlier-Robust Stochastic Gradient Descent

Quantile-k-Loss SGD filters corrupted component losses by quantile sampling, proving linear convergence while outperforming standard and min-k-loss SGD.

The paper proposes Quantile-k-Loss SGD (Q(k)L-SGD), a loss-filtering framework for finite-sum optimization with corrupted components that samples k losses per iteration and updates using an index from the lower empirical q-quantile. The authors prove linear convergence under standard convexity, requiring sample size to scale with the number of corruptions, plus a complementary small-sample probabilistic analysis. Experiments on polynomial regression, regularized logistic regression, and hinge loss show intermediate quantiles often outperform both standard SGD and min-k-loss SGD.

arXiv cs.AI / cs.LG / cs.CL·Sep 11, 16:38 UTC · 5d agoAI research

18

Local gradient neural operator

Researchers propose LGNO, a lightweight interpretable neural operator using learnable local stencils, matching global-operator accuracy on PDE benchmarks with fewer parameters.

LGNO builds on nonlinear gradient discretization priors and uses multilayer perceptron convolutional layers to learn translation-invariant local kernels resembling discrete stencils. A zero consistent stencil factorization separates coefficient learning from field reconstruction, and network folding shares equivalent components to cut parameter counts for symmetric problems. Evaluations on linear and nonlinear, static and dynamic, and low- and high-dimensional PDE benchmarks show maintained accuracy, parameter efficiency, and rollout stability, with applicability to diffusion, flow, and quantum problems.

arXiv cs.AI / cs.LG / cs.CL·Sep 7, 16:53 UTC · 9d agoAI research1

28

Reference-Based Bias Detection in LLMs via Relative Representations of Hidden States

Researchers propose auditing LLM bias via relative hidden-state representations, detecting bias increases with 3-50x less compute than output-level benchmarks.

The paper introduces a reference-based bias auditing method that compares hidden-state representations across model variants, such as before and after fine-tuning, by encoding sentences relative to a fixed anchor set. The resulting Representational Bias Shift (Delta-B) correlates with output-level bias change in 15 of 18 tested settings, reaching |r| = 0.84 under full fine-tuning across WildGuardMix, DecodingTrust, and ToxiGen benchmarks. Thresholding Delta-B detects checkpoints whose bias increased with ROC AUC between 0.65 and 0.99 and beats a SEAT-based baseline, while auditing a model in about three minutes with 3-50x less compute.

Hugging Face daily papers·Sep 8, 20:00 UTC · 8d agoAI research1

42

A Theoretical Analysis of Generalization Dynamics in Neural Networks under Gradient Descent with Weight Decay

Theoretical framework bounds generalization for gradient descent with weight decay, deriving conditions that explain delayed generalization and grokking.

The paper proves convergence of gradient descent with weight decay to a neighborhood of global minimizers of the empirical l2 loss for a broad class of neural networks. It decomposes population error into data, optimization, and prediction variation errors, deriving cellwise and layerwise approximate-homogeneity bounds on prediction variation along the training trajectory. The resulting necessary and sufficient conditions explain layerwise generalization differences and provide a theoretical characterization of grokking.

arXiv cs.AI / cs.LG / cs.CL·Sep 7, 16:59 UTC · 9d agoAI research

12

FreeFlow: A Bias-free Hierarchical Transformer for Optical Flow Estimation

FreeFlow is a bias-free hierarchical transformer achieving state-of-the-art optical flow results on Sintel, KITTI-2015, and Spring benchmarks.

FreeFlow replaces task-specific inductive biases like correlation volumes and iterative warping with a single feed-forward encoder-decoder combining window, shifted-window, and reduced-resolution global attention. It reaches 0.68/1.48 EPE on Sintel Clean/Final, 3.23 Fl-all on KITTI-2015, and 3.192 1px on Spring. The architecture scales consistently from small to large variants and remains memory efficient at 1080p inference.

Hugging Face daily papers·Sep 9, 20:00 UTC · 7d agoAI research

30

When Does Scale-Invariant Optimization Become Unstable? An Exact Schedule Law with Weight Decay

Researchers derive an exact law linking learning-rate schedules and weight decay in normalized networks, pinpointing when scale-invariant optimization destabilizes.

The paper shows that normalization makes large parts of neural networks scale-invariant, creating a hidden feedback loop where learning-rate schedules and weight decay interact through the parameter norm to control the effective optimizer step. An exact discrete-time law with a single scalar quantity separates contraction- and expansion-dominated effective learning-rate regimes, and the balance point is intrinsically unstable, so constant learning rate with weight decay produces recurrent behavior instead of a stable equilibrium. A unified homogeneous-optimizer framework explains why adaptive methods stabilize more weakly under normalization. The law is validated with high precision on MLPs, CNNs, and GPT-2 across MNIST, CIFAR, WikiText, and OpenWebText, with code released on GitHub.

arXiv cs.AI / cs.LG / cs.CL·Sep 8, 17:46 UTC · 8d agoAI research

30

Predicting Privacy Leakage from Weight Spectral Density

Study shows WeightWatcher spectral metrics like stable rank correlate with membership inference vulnerability, enabling cheaper ML privacy auditing.

The paper tests whether spectral metrics from the heavy-tailed self-regularisation framework can proxy membership inference attack (MIA) vulnerability without training expensive shadow models. On image and tabular classification tasks, stable rank correlates positively with overall MIA success, while Log alpha-Norm correlates negatively at the low false-positive regime. These correlations are stronger than those obtained from the generalisation gap, suggesting weight spectra capture leakage information overfitting measures miss. The authors propose spectral analysis as a scalable direction for privacy auditing.

arXiv cs.CR·Sep 10, 16:34 UTC · 6d agoResearch

32

Subgroup Membership Inference Audits of Differentially Private Synthetic Text

Audits of 32 differentially private synthetic-text releases show subgroup membership leakage is concentrated in few records and systematically underestimated by average-case attacks.

The paper defines a subgroup-targeted membership inference game in which the target pool is an explicit parameter, to audit residual leakage in differentially private synthetic text releases. The audit instantiates 32 proxies across four datasets, three generators (DP-SGD fine-tuning, API-based prompting, and activation steering), and five privacy budgets. DP substantially reduces average leakage at every budget, but remaining leakage is concentrated: roughly a tenth of records carries about 40% of it, and the noise removes more measured leakage from random records than from high-risk ones. Which records leak depends on the release mechanism, so record-level risk cannot be assessed independently of the release.

arXiv cs.CR·Sep 9, 07:56 UTC · 7d agoResearch

45

Implementing a White-Box Undetectable Backdoor for Random Fourier Features

Researchers implement Goldwasser's CLWE-based undetectable backdoor for Random Fourier Features models in numpy/scipy, confirming practical realizability with no detectable differences from clean models.

The paper provides an end-to-end implementation of the Goldwasser et al. white-box undetectable backdoor for models trained with the Random Fourier Features algorithm, using only numpy and scipy. It derives two samplers for the core GP_d(b_k) distribution: a rejection-sampling proxy and an exact closed-form sampler verified against its analytic form. Statistical indistinguishability tests covering weight-space and functional black-box comparisons found no detectable difference between backdoored and clean models across sparsity ratios. The underlying lattice hardness reduction was not reproduced, and the work demonstrates the threat is realizable with commodity scientific-computing tools rather than specialized cryptographic infrastructure.

arXiv cs.CR·Sep 14, 22:18 UTC · 1d agoResearch

50

Unlocking Lossless Speedups in LLMs via Discrete Diffusion

Uno pairs autoregressive LLMs with lightweight diffusion weights to draw multiple tokens in parallel, delivering up to 3x lossless speedup without a draft model.

The paper introduces diffusion-augmented LLMs: autoregressive weights trained with the standard next-token objective plus lightweight diffusion weights trained via a Diffusion Distillation phase to emit multiple tokens in parallel. Psi-Spec samplers enable lossless acceleration without the separate draft model required by speculative decoding. The 8B Uno model outperforms the 26B open DiffusionGemma and proprietary Mercury 2 on agentic tool use, coding, and long-context reasoning benchmarks, with up to 3x throughput gains over the base model at all evaluated batch sizes. Code and checkpoints are released publicly.

Hugging Face daily papers·Sep 2, 20:00 UTC · 14d agoAI research

55

Model-Aware Schedules Improve Generation via Fiberwise Optimal Transport

Researchers introduce model-aware diffusion schedules via fiberwise optimal transport, cutting flow-matching FID on CIFAR-10 by 38.6% at 16 function evaluations.

The paper proposes constructing diffusion and flow-matching sampling schedules from a fiberwise prediction risk defined via optimal transport, combined with coefficient-path kinetic action, yielding a closed-form time allocation. Across DDPM and flow-matching experiments spanning targets, datasets, and architectures, the schedules beat model-agnostic baselines, including a 38.6% relative FID reduction for flow matching on CIFAR-10 at 16 function evaluations. Normalized fiberwise-risk profiles from independently trained models align closely, suggesting empirical universality, and a frozen analytic allocation template retains most of the gains.

arXiv cs.AI / cs.LG / cs.CL·Sep 10, 17:30 UTC · 6d agoAI research

32

Rare Not Random Using Token Efficiency for Secrets Scanning

Researcher proposes token efficiency (string length divided by BPE token count) as a better post-regex filter than entropy for secrets scanning, validated on CredData.

The post explores whether Byte-Pair Encoding tokenization can replace Shannon entropy as the primary filter for candidate secrets captured by regex in tools like Gitleaks. It defines 'token efficiency' as string length divided by token count under the cl100k_base tokenizer; secret-like strings such as GitHub tokens tokenize into many small tokens and score low, while natural text scores high. Evaluating labeled secrets from the CredData dataset shows a usable separation, with roughly 2.5 suggested as a minimum cutoff versus Gitleaks' 3.5 entropy threshold. The technique is positioned as a post-regex filtering step rather than a standalone detector.

Lobsters · security·Sep 12, 06:42 UTC · 4d agoResearch

45

Mask Forcing: Improving Autoregressive Video Diffusion Distillation via Dual-Noise Masking Rollout

Researchers propose Mask Forcing, a dual-noise masking rollout that mitigates mode collapse in autoregressive video diffusion distillation.

The paper targets over-saturation and over-smoothing in autoregressive video diffusion models distilled via Distribution Matching Distillation, attributed to reverse-KL mode-seeking behavior. Mask Forcing perturbs the student self-rollout with random masks along spatial and temporal axes, injecting cleaner tokens that act as denoising guidance for noisier tokens. Experiments show improved visual quality across multiple distillation methods without using real video data or extra post-training stages.

Hugging Face daily papers·Sep 7, 20:00 UTC · 9d agoAI research

22

Robust Coverless Linguistic Steganography via Sentence Embedding Space with Global Resynchronization

Researchers propose a coverless steganographic framework encoding messages as hierarchical clustering paths in sentence embedding space with a Global Resynchronization Mechanism for robustness.

An arXiv paper proposes encoding secret messages as hierarchical clustering paths in the sentence embedding space rather than token space, improving decoding stability against word- and sentence-level textual perturbations. A Global Resynchronization Mechanism (GRM) reframes variable-length bitstreams as discrete symbols anchored to semantic subspaces to prevent bit-slippage. Experiments show substantial robustness improvements while maintaining embedding capacity and resistance to statistical analysis.

arXiv cs.CR·Sep 4, 10:22 UTC · 12d agoResearch

18

Embedded Graph Flows for Categorical Graph Generation

Researchers propose Embedded Graph Flows, a generative model with learned categorical embeddings that beats DiGress and GruM on molecular graph benchmarks.

Embedded Graph Flows (EGF) learns continuous embeddings for node and unordered-edge categories and transports Gaussian noise toward these endpoints using a permutation-equivariant graph transformer. On QM9 it achieves the best result on all four reported metrics, with a Fréchet ChemNet Distance of 0.150 versus 0.717 for DiGress and 0.812 for GruM. On ZINC250k it retains the lowest NSPDK MMD, indicating close agreement with local substructures of reference molecules. Code is released on GitHub.

arXiv cs.AI / cs.LG / cs.CL·Sep 4, 16:26 UTC · 12d agoAI research1

22

Data Scarcity and Model Sparsity: Mixtures-of-Experts Overfit More to Repeated Data

Study finds Mixture-of-Experts models overfit faster than dense Transformers under repeated training data, with degradation tied to total parameter sparsity.

Across models from 80M to 1B active parameters (8.5B total), MoE architectures degrade more rapidly than dense models when training data is repeated, with the effect increasing with sparsity as dictated by total parameters. Dense 80M models tolerate 8x repetition with minimal loss while MoEs suffer at 4x and underperform dense models beyond 32x. Masking-based regularization such as dropout mitigates overfitting, letting MoEs beat dense models even at over 64x repetition, though no method matches all-unique training data. Routing stabilizes early and expert specialization correlates with overfitting to repeated data.

arXiv cs.AI / cs.LG / cs.CL·Sep 10, 17:57 UTC · 6d agoAI research

36

Online Draft Co-Training for Speculative Decoding in Large-Scale, Long-Context RL Post-Training

NVIDIA researchers detail an end-to-end system for online draft co-training that speeds speculative decoding in large-scale long-context RL post-training.

The paper tackles scaling online draft co-training for speculative decoding in RL post-training, where rollout generation dominates cost. It extends packed, load-balanced zigzag ring attention to merge rank-local branch attention with causal main-sequence attention for context parallelism, and introduces TapChannel to transport target features across pipeline-parallel stages without changing the schedule. Experiments show co-trained drafts tracking the policy baseline with substantial rollout and end-to-end speedups up to 122B parameters and strong scaling at 256K tokens.

Hugging Face daily papers·Sep 6, 20:00 UTC · 10d agoAI research

26

RAPID: A Real-Time Defense Against Unauthorized Model Distillation for Text-to-Image Services

RAPID embeds defensive perturbations in a T2I model's shared VAE decoder to block unauthorized black-box distillation in real time.

The paper defends text-to-image services against model theft via black-box output-based distillation, where adversaries collect prompt-image pairs to train substitute models. RAPID integrates defensive perturbations into the shared VAE decoder using self-referenced latent maximization plus reconstruction-guided color regularization, avoiding costly sample-wise online optimization. Across four T2I models and four datasets versus five baselines, it consistently degrades substitute-model generation quality while preserving visual fidelity.

arXiv cs.CR·Sep 14, 16:11 UTC · 2d agoAI safety & security

35

What Breaks Local Watermarks? A Robustness Benchmark for Local Invisible Image Watermarking

First systematic robustness benchmark of five local invisible image watermarking methods across 55 transformations finds all are vulnerable, with inpainting and geometric misalignment completely breaking payload…

The paper presents the first systematic robustness benchmark for local invisible image watermarks, covering 55 image transformations across signal distortions, coordinate alignment changes, indirect local edits, and direct watermark edits. It evaluates five methods: MaskWM, WAM, OmniGuard, TrustMark, and PixelSeal, all supporting localization natively or with minimal adaptation. Results show every method is vulnerable to some transformation; MaskWM offers the strongest payload recovery and localization but the lowest clean-image quality, and synchronization further improves its recovery under geometric transformations. Geometric misalignment and generative local edits such as inpainting and outpainting can completely impair payload recovery, while signal distortions are often tolerated.

arXiv cs.CR·Sep 15, 08:30 UTC · 1d agoResearch

42

Eliciting Weak-to-Strong Generalization with On-Policy Reverse Distillation

OPRD distillation enables weak-to-strong generalization by amplifying verifier-supported policy updates, outperforming existing RL and distillation methods with fewer student updates.

On-Policy Reverse Distillation (OPRD) evaluates a weak teacher's policy shift relative to its reference policy on student rollouts and amplifies the verifier-supported component of the student's policy gradient. This rescaling preserves the stationary points of policy optimization while letting the student learn beyond the teacher's capacity ceiling. In successive model transfer and multi-teacher distillation, OPRD achieves higher performance with fewer student updates than existing RL and distillation approaches, and response-style analysis shows students remain closer to verifier-RL-trained models than to their weak teachers.

Hugging Face daily papers·Sep 7, 20:00 UTC · 9d agoAI research

33

Likelihood-free inference with nuisance parameters through normalizing flows

Researchers decompose normalizing flows to derive near-pivotal statistics for likelihood-free inference with nuisance parameters, recovering the t-test and beating Welch limits.

A new paper decomposes neural-network normalizing flows to uncover pivotal statistics in the presence of nuisance parameters using only a sample generator from the distribution of interest. The statistic is near-pivotal in the sense of minimum average KL-divergence of its p-values and can incorporate prior knowledge of group invariances such as translation and scale. Experiments show it recovers the one-sample t-test almost exactly, outperforms the Welch test on worst-case size over a constrained variance-ratio range, and delivers higher power and much faster runtime than profile likelihood-ratio techniques on small-to-moderate samples.

arXiv cs.AI / cs.LG / cs.CL·Sep 9, 17:58 UTC · 7d agoAI research

16

AdamX: Cosine similarity meets gradient descent

Researchers propose AdamX, a cosine-similarity-based first-order optimizer with variance rectification that matches Adam-class convergence across benchmark datasets and architectures.

The paper introduces AdamX, a first-order optimizer that uses cosine similarity as an adaptive mechanism for controlling update magnitudes, plus a variance rectification scheme for smoother optimization early in training. The method is described as scalable, model-agnostic, and straightforward to integrate into existing pipelines. Empirically, AdamX shows competitive convergence rates measured by epochs to reach performance thresholds under a fixed hyperparameter budget, with code and experiments released on GitHub.

arXiv cs.AI / cs.LG / cs.CL·Sep 10, 17:42 UTC · 6d agoAI research

22

Adversarial Testing of Automated Program Repair Agents for Security Vulnerabilities

SWEADV benchmark shows adversarial issue descriptions make LLM program-repair agents write insecure fixes in 51.7% of cases, evading most detection tools.

Researchers built SWEADV, a benchmark of 750 adversarial issue descriptions derived from 150 SWE-bench Verified repair tasks, covering command execution, deserialization, path traversal, denial of service, and weak hashing attack types. Tested on mini_swe agents backed by GPT-5-Mini, MiniMax-M2.5, and DeepSeek-R, adversarial descriptions induced malicious behavior with successful repair in 51.7% of cases. Detection was weak: LLM-as-judge pre-repair screening reached only 62.3% accuracy, and post-repair detection via static analysis and LLM-as-judge achieved just 39.4% and 55.4%.

arXiv cs.CR·Sep 14, 17:53 UTC · 2d agoAI safety & security2

52