Search: “data-compression”

Smart search ranks by meaning as well as keywords (one row per story, last 45 days).

The Price of Sparsity: Sufficient Conditions for Sparse Recovery using Sparse and Sparsified Measurements

Researchers derive sufficient sample-size conditions for recovering sparse binary signals from sparse Gaussian measurements, quantifying an information-theoretic threshold of order slog(p/s)/log(ds/p).

The paper studies support recovery of sparse binary signals from noisy linear measurements. For sparse Gaussian designs, the authors identify sufficient minimal sample sizes for maximum-likelihood recovery in the high-SNR regime d*s/p -> infinity, yielding an information-theoretic threshold of order slog(p/s)/log(ds/p) that makes the price of measurement sparsity explicit. They also show a regime where the sample-complexity loss from sparsity is only logarithmic while computational gains are nearly linear, and prove that for independently sparsified dense Gaussian designs a sample size of order p/ψ² suffices for support recovery at any fixed error level.

Hugging Face daily papers·Sep 7, 20:00 UTC · 9d agoAI research

12

Rare Not Random Using Token Efficiency for Secrets Scanning

Researcher proposes token efficiency (string length divided by BPE token count) as a better post-regex filter than entropy for secrets scanning, validated on CredData.

The post explores whether Byte-Pair Encoding tokenization can replace Shannon entropy as the primary filter for candidate secrets captured by regex in tools like Gitleaks. It defines 'token efficiency' as string length divided by token count under the cl100k_base tokenizer; secret-like strings such as GitHub tokens tokenize into many small tokens and score low, while natural text scores high. Evaluating labeled secrets from the CredData dataset shows a usable separation, with roughly 2.5 suggested as a minimum cutoff versus Gitleaks' 3.5 entropy threshold. The technique is positioned as a post-regex filtering step rather than a standalone detector.

Lobsters · security·Sep 12, 06:42 UTC · 4d agoResearch

45

Quantization-Aware Healing: a compressed, 4-bit model that outperforms its full-precision original

Multiverse Computing details quantization-aware healing, producing a 4-bit compressed model that reportedly outperforms its full-precision original.

A Hugging Face blog post by Multiverse Computing's CAI team introduces quantization-aware healing for compressed models. The post claims the resulting 4-bit model outperforms the original full-precision model. No additional details or benchmarks were available in the provided text.

Hugging Face Blog·Aug 25, 11:39 UTC · 22d agoAI research

40

LACE: Layer-Wise Compression for Dynamic Frame Rate Codecs

LACE introduces layer-wise compression for dynamic frame rate audio codecs, cutting sequence lengths and speeding TTS inference while preserving quality.

LACE (Layer-Adaptive Codec Encoding) applies an independent compression step at each quantization layer of a neural audio codec, enabling layer-specific segmentation boundaries instead of shared ones. Union alignment and boundary anchor mechanisms keep durations consistent for downstream text-to-speech. On LibriTTS, LACE achieves a better rate-quality tradeoff than prior dynamic frame rate codecs and improves TTS inference efficiency at competitive synthesis quality. Code is released in the ESPnet3 codec recipe.

arXiv cs.AI / cs.LG / cs.CL·Sep 15, 17:46 UTC · 1d agoAI research1

15

X-AuT: Progressive Audio-Encoder Compression for Speech LLMs with Cross-Scale Distillation

XPeng AI's X-AuT prunes speech LLM audio encoders, cutting Qwen3-ASR-0.6B error from 5.61% to 5.27% with fewer parameters.

X-AuT is a progressive compression framework for speech LLM audio encoders that selects layer combinations via short behavioral probes and restores pruned models using cross-scale distillation and LoRA finetuning while keeping the language-model backbone frozen. Compressing Qwen3-ASR-0.6B from 18 to 16 audio-encoder layers lowered macro-average error from 5.61% to 5.27% on ten Chinese-English benchmarks. A 14-layer model reached 5.75% error with 20.7% fewer audio-tower parameters, and progressive pruning outperformed direct pruning (5.75% vs 6.73%).

Hugging Face daily papers·Sep 9, 20:00 UTC · 7d agoAI research

26

Lightweight Vision Transformer Compression for On-Device Plant Disease Detection in Resource-Constrained Agricultural Field Conditions

A unified ViT compression pipeline (H-BAC pruning, quantization, distillation) cuts plant-disease models 54.5x to 6.01 MB while keeping 95.13% accuracy.

Researchers combined Hessian-Balanced Adaptive Block Pruning (H-BAC), guided by second-order sensitivity estimation, with quantization and attention-based knowledge distillation to compress Vision Transformers for on-device chilli plant disease detection in India. On a 3-class cross-village, cross-device out-of-distribution dataset, the integrated pipeline reduced model size from 327.42 MB to 6.01 MB (54.5x) at 95.13 +/- 2.32% accuracy, matching the 95.13% FP32 baseline. Ablations also show a directly trained 6.01 MB INT8 student reaches 94.87% accuracy, indicating where pruning and distillation add limited value.

arXiv cs.AI / cs.LG / cs.CL·Sep 4, 16:40 UTC · 12d agoAI research

22

Grouped Value Attention: Efficient KV Caching via On-Demand Key Reconstruction

Grouped Value Attention stores grouped values and reconstructs content keys via a learned linear map, cutting KV-cache size about 45-47% versus GQA.

GVA stores only grouped values and reconstructs content keys with a learned linear map absorbed into the query at decode time, while a small shared decoupled RoPE channel preserves positional information via a separately cached positional key. At 350M parameters trained on 30B FineWeb-Edu tokens, the 16-dimensional positional variant scores 44.18 average accuracy across five tasks versus 44.36 for GQA and 43.88 for MLA. Custom decoding kernels are in development with an open-source release planned.

Hugging Face daily papers·Sep 7, 20:00 UTC · 9d agoAI research

42

Cadence: Error-Bounded Lossy Compression of Demand Time Series with a Time-Series Foundation Model

Cadence pairs Google's 330M-parameter TimesFM-3 foundation model with adaptive arithmetic coding, gaining 13-28% on 2026 demand series over classical predictors.

Cadence is an error-bounded lossy compressor for numeric time series combining the 330M-parameter Google TimesFM-3 foundation model with an adaptive arithmetic coder, guaranteeing a per-sample error bound. On 49 EIA-930 balancing-authority demand series from 2026 it gains 13.3% over the best of six classical predictors and 28.3% on 50 MTA ridership series, winning all 297 series-tolerance pairs with a 21.4% median gain. The paper also reports negative results, including that foundation models add negligible value for lossless coding and that PyTorch predictions are not bit-identical across batch sizes.

Hugging Face daily papers·Sep 4, 20:00 UTC · 12d agoAI research1

25

SQS: Bayesian DNN Compression through Sparse Quantized Sub-distributions

SQS unifies weight pruning and low-bit quantization via Bayesian variational learning, compressing Llama3.2 and Qwen2.5 at higher rates with comparable accuracy.

SQS introduces a unified Bayesian variational framework performing simultaneous pruning and low-bit quantization, using a spike-and-slab prior for sparsity and Gaussian Mixture Models to model quantized weights. The authors derive an efficient approximation for the intractable objective and provide a consistency result for the variational approach. Experiments on ResNet, BERT-base, Llama3.2, and Qwen2.5 show higher compression rates than prior baselines with comparable performance drops.

Hugging Face daily papers·Sep 6, 20:00 UTC · 10d agoAI research

28

A Princeton Researcher Proposes Recurrent Looped Transformer (RLT) that Carries Decoder State across Every Token, Fixing 96 Blocks per Token with Unbounded Temporal Depth

Princeton researcher Yifan Zhang proposes Recurrent Looped Transformer, carrying full decoder state across every token for unbounded temporal depth.

Yifan Zhang's technical report defines the Recurrent Looped Transformer (RLT), pairing a causal encoder with a recurrent decoder whose final output and layerwise sliding-window attention cache carry into every subsequent token with no prompt-response boundary reset. The reference configuration ties 48 encoder and 48 decoder layers, executing 96 logical blocks per token while the state path grows to 48t blocks after t tokens at fixed per-token compute. The report details RL replay contracts that rebuild all states under current parameters and exact prefix snapshots for multi-turn serving, but explicitly reports no measured efficiency, reasoning quality, or scaling results.

MarkTechPost·Sep 13, 17:02 UTC · 3d agoAI research1

40

Training and Finetuning Multi-Vector Embedding Models with Sentence Transformers

Hugging Face published a tutorial on training and finetuning multi-vector embedding models using the Sentence Transformers library.

Hugging Face's blog walks through training and finetuning multi-vector embedding models with Sentence Transformers. Multi-vector approaches store multiple vectors per document to support late-interaction retrieval. The post is a practical guide for developers building retrieval pipelines with the library.

Hugging Face Blog·Aug 26, 00:00 UTC · 21d agoAI tools & infra1

20

DataFlex-RL: An Evaluation Platform for RLVR Data Policies

DataFlex-RL benchmark of 13 RLVR data policies on Qwen2.5-7B finds none reproducibly beats uniform sampling under matched GRPO training.

DataFlex-RL is an evaluation platform comparing rollout-selection, reweighting, and mixture data policies for RLVR under a common GRPO recipe. Across 13 configurations and 12 matched seeds with Qwen2.5-7B-Base on 12 math, logic, and science benchmarks, uniform GRPO improved domain-balanced accuracy by 7.76 points, but no alternative policy achieved a statistically significant improvement. A corrected 12-seed Llama-3.1-8B-Base extension found no consistent winner, and math-heavy evaluation summaries were negatively correlated (-0.33) with domain-balanced summaries.

Hugging Face daily papers·Sep 4, 20:00 UTC · 12d agoAI research

35

Memory-Efficient Designs for Word-Wise Universal Fully Homomorphic Encryption

BXT framework mitigates FHE memory bottlenecks via ciphertext compression, serialization, delayed seeding, and digit pruning, achieving up to 3.8x CNN inference speedup.

A new paper proposes BXT, an optimization framework for word-wise Universal Fully Homomorphic Encryption that targets the memory bottleneck rather than compute. It combines four techniques: ciphertext compression via seed regeneration, bit-packed ciphertext serialization for L2-to-L1 transfers, delayed PRNG-heavy offline seed generation across aggregated operations, and fault-aware ciphertext digit pruning. On CNN inference, the BXT-CSO50 configuration achieves up to 3.8x speedup over a 100x GPU baseline with under 1% accuracy loss at 50% comparison precision.

arXiv cs.CR·Sep 4, 06:03 UTC · 12d agoResearch

24

Datamimic – don't let your coding agent invent its own test world

Datamimic is an open-source test data generation tool aimed at keeping coding agents from inventing their own test fixtures.

A Hacker News discussion (40 points) highlights Datamimic, an open-source rapiddweller GitHub project for generating realistic synthetic test data. The tool targets AI coding agents, aiming to prevent them from fabricating their own inconsistent test worlds. Only the repository link was shared, so details are limited.

Hacker News · AI·04:58 UTC · 16h agoAI tools & infraHN 40↑ · 6 comments

15

Data Scarcity and Model Sparsity: Mixtures-of-Experts Overfit More to Repeated Data

Study finds Mixture-of-Experts models overfit faster than dense Transformers under repeated training data, with degradation tied to total parameter sparsity.

Across models from 80M to 1B active parameters (8.5B total), MoE architectures degrade more rapidly than dense models when training data is repeated, with the effect increasing with sparsity as dictated by total parameters. Dense 80M models tolerate 8x repetition with minimal loss while MoEs suffer at 4x and underperform dense models beyond 32x. Masking-based regularization such as dropout mitigates overfitting, letting MoEs beat dense models even at over 64x repetition, though no method matches all-unique training data. Routing stabilizes early and expert specialization correlates with overfitting to repeated data.

arXiv cs.AI / cs.LG / cs.CL·Sep 10, 17:57 UTC · 6d agoAI research

36

Differential Privacy Meets Fixed Parameter Tractability: Algorithms and Lower Bounds

Theory paper combines differential privacy with fixed-parameter tractable encoders, improving approximation guarantees for combinatorial optimization and proving new lower bounds.

The paper studies combinatorial optimization under epsilon-differential privacy within the implicit encoder-decoder framework of Gupta et al. (SODA 2010), generalizing it to allow fixed-parameter tractable encoders. This circumvents approximation barriers inherent to polynomial-time algorithms and yields improved guarantees for fundamental combinatorial optimization problems. The authors establish the first representation-independent lower bounds: assuming a non-uniform variant of the Gap Exponential Time Hypothesis, no epsilon-DP encoder-decoder pair can achieve certain approximation guarantees with a subexponential-time decoder for sufficiently small epsilon. Representation-dependent lower bounds are also provided for larger epsilon.

arXiv cs.CR·Sep 11, 07:13 UTC · 5d agoResearch

18

General Quantification of Covariate and Concept Shifts

Paper proposes γ*-concept shifts via entropic optimal transport, deriving estimable generalization bounds unifying covariate and concept shift under distribution shift.

The authors show existing definitions of concept shift break when source and target supports mismatch and propose γ*-concept shifts grounded in entropic optimal transport. They derive a general error bound covering broad loss functions, label spaces and stochastic labeling, plus estimators with concentration guarantees. The resulting DataShifts algorithm quantifies distribution shifts and estimates the error bound in most applications, addressing learning bounds that were previously non-estimable from samples.

arXiv cs.AI / cs.LG / cs.CL·Sep 10, 17:57 UTC · 6d agoAI research1

28

Generative Late-Interaction Embeddings For Visual Document Retrieval

GLIE compresses visual document retrieval embeddings to four vectors per page while retaining nearly 80% of uncompressed nDCG@5 accuracy.

Researchers analyzing late-interaction retrieval embeddings found they lie exactly on the unit sphere and concentrate near a manifold of intrinsic dimension five to six. GLIE learns a few k vectors per page that serve as a lightweight index and a basis to regenerate the full embedding set for exact rescoring of top candidates at query time. On ViDoRe v1 with four vectors per page, GLIE retains nearly 80% of uncompressed nDCG@5 versus 70% for the best prior post-hoc method, using a 415K-parameter network trained in under three GPU-minutes on 1,000 pages.

Hugging Face daily papers·Sep 9, 20:00 UTC · 7d agoAI research

28

Robust Coverless Linguistic Steganography via Sentence Embedding Space with Global Resynchronization

Researchers propose a coverless steganographic framework encoding messages as hierarchical clustering paths in sentence embedding space with a Global Resynchronization Mechanism for robustness.

An arXiv paper proposes encoding secret messages as hierarchical clustering paths in the sentence embedding space rather than token space, improving decoding stability against word- and sentence-level textual perturbations. A Global Resynchronization Mechanism (GRM) reframes variable-length bitstreams as discrete symbols anchored to semantic subspaces to prevent bit-slippage. Experiments show substantial robustness improvements while maintaining embedding capacity and resistance to statistical analysis.

arXiv cs.CR·Sep 4, 10:22 UTC · 12d agoResearch

18

Privacy Failure in Split-LLM Training, The Returned Gradient Nullifies the Decoys

Researchers show split-LLM training leaks privacy via zero-valued gradients on decoy rows, exposing which activations are real despite passing forward-channel checks.

A systems-security case study of a two-node split-LLM training setup found that the returned output gradient from an Untrusted Cloud Node is exactly zero for decoy rows, revealing which rows are real. Across nine seeds, zero patterns identified real rows in 4,096 of 4,096 frames per run, and an attack on frame contents recovered 0.65 to 1.50 percentage points of extra tokens over a baseline. Both datasets passed forward-channel privacy and quality checks but failed once the returned gradient was included. Row-wise gradient clipping and noise closed the leak for roughly 0.01 nats of held-out cross-entropy, though five unmeasured attack classes remain.

Hugging Face daily papers·Sep 2, 20:00 UTC · 14d agoAI safety & security

58

Rethinking Heterogeneous System Disaggregation for Subquadratic Attention

SQD disaggregates LLM inference by quadratic versus subquadratic attention layers, improving energy efficiency up to 56% on heterogeneous systems versus GPU-only baselines.

SQD (SubQuadratic Disaggregation) splits decode not by operator type but by quadratic versus subquadratic attention, matching their distinct arithmetic intensity and memory footprints. For sparse attention LLMs it separates top-k selection (requiring full KV indexing) from top-k attention plus FFN; for linear and sliding-window models it separates dense attention layers from subquadratic layers plus FFN. On an adjusted 8xB200 heterogeneous proxy, tokens-per-joule improves 53% on GLM 5.2, 31% on Nemotron 3 Ultra, and 56% on Gemma 4 31B. A Rubin plus LPX analytical model shows 1.2x-1.5x tighter achievable latencies and up to 3.6x higher throughput versus attention-FFN disaggregation.

arXiv cs.AI / cs.LG / cs.CL·Sep 11, 17:55 UTC · 5d agoAI research

28

HyQuant: Hybrid-Precision Quantization for LLM Attention

HyQuant keeps most LLM attention states low-bit while preserving vertical-line tokens and local windows in high precision, maintaining near-lossless accuracy.

HyQuant is a hybrid-precision quantization framework for LLM attention that quantizes most attention states to low bits while keeping accuracy-critical vertical-line tokens and local-window states in full precision, selected via lightweight attention-pattern signals. In the prefill stage it uses a hybrid-precision attention operator, and in the decode stage it applies the same principle to KV-cache compression with fused dequantization and attention computation. Across diverse tasks, models, and datasets it maintains nearly lossless accuracy; code is available on GitHub.

Hugging Face daily papers·Aug 27, 20:00 UTC · 20d agoAI tools & infra1

33

Mask Forcing: Improving Autoregressive Video Diffusion Distillation via Dual-Noise Masking Rollout

Researchers propose Mask Forcing, a dual-noise masking rollout that mitigates mode collapse in autoregressive video diffusion distillation.

The paper targets over-saturation and over-smoothing in autoregressive video diffusion models distilled via Distribution Matching Distillation, attributed to reverse-KL mode-seeking behavior. Mask Forcing perturbs the student self-rollout with random masks along spatial and temporal axes, injecting cleaner tokens that act as denoising guidance for noisier tokens. Experiments show improved visual quality across multiple distillation methods without using real video data or extra post-training stages.

Hugging Face daily papers·Sep 7, 20:00 UTC · 9d agoAI research

22

Why Is Video Still So Expensive? A Survey of Inference-Efficiency Mechanisms in Video and Audiovisual LLMs

A survey catalogs inference-efficiency techniques for video and audiovisual LLMs, mapping bottlenecks in sampling, encoding, token reduction, and LLM decoding.

This survey covers inference-efficiency mechanisms for visual and audiovisual video LLMs, reporting reductions in parameters, FLOPs, latency, memory, and token counts. It organizes methods by pipeline stage, covering frame sampling, modality encoding, connector-level token reduction, and LLM prefilling and decoding for systems built since late 2022. The authors compile accuracy-cost comparisons under shared host models and input protocols, identify gaps in audiovisual efficiency and standardized evaluation, and maintain a public repository.

Hugging Face daily papers·Sep 8, 20:00 UTC · 8d agoAI research

40

Mi-Ripple: Restoring Images Degraded by Iterative AI Editing

Mi-Ripple is a diagnosis-guided restoration workflow that removes digital ripple artifacts introduced by iterative AI image editing while preserving structure.

Iterative reference-conditioned image editing can introduce grid-like and granular textures known as digital ripple. Mi-Ripple separates periodic lattice artifacts from content-entangled granular texture, then applies selective spectral notching, structure-aware smoothing, and cleaned-reference regeneration. In fourteen notch-only executions, whole-image residual standard deviation was 0.08-0.44 in CIELAB lightness units, and reference cleaning reduced output debris density by 45% in a paired example.

Hugging Face daily papers·Sep 9, 20:00 UTC · 7d agoAI research

25

When Models Edit Too Much: On the Fidelity of Minimal Code Edits

A 400-task BigCodeBench evaluation shows frontier LLMs widely over-edit code; a preservation instruction cuts excess edits and raises Pass@1 by 2.3 points.

Researchers built an evaluation framework from 400 BigCodeBench problems with injected AST-level corruptions, each with a known minimal patch, to measure over-editing in LLM code repair. Even strong models like GPT-5.5 produce unnecessarily large edits despite high Pass@1. Adding a preservation instruction reduced average excess Levenshtein distance from 0.195 to 0.131, cut added cognitive complexity by 26.6%, and raised Pass@1 by 2.3 points. Reinforcement learning post-training gave the best out-of-domain edit-fidelity trade-off, while supervised fine-tuning overfit to seen corruption patterns.

Hugging Face daily papers·Sep 2, 20:00 UTC · 14d agoAI research1

30

Measuring benchmark optimization in speech recognition

Hugging Face examines how much speech recognition systems overfit benchmarks and how to measure benchmark optimization in ASR.

A Hugging Face post on measuring benchmark optimization in automatic speech recognition, analyzing how model improvements on benchmarks reflect genuine capability gains versus overfitting. It is evaluation methodology research with no direct security impact.

Hugging Face Blog·Aug 21, 00:00 UTC · 26d agoAI research

22

Disentangling Representation Evolution in Transformers through Directional Decomposition

Decomposes transformer representation updates into parallel and perpendicular components, linking geometry to editing robustness and better pretraining.

The paper decomposes learned transformer updates into parallel and perpendicular components relative to the hidden state, finding substantial parallel components beyond the residual identity path across pretrained models. Targeted edits reveal exclude-self value-space parallel manipulation is more robust than residual-space or perpendicular alternatives, and perpendicular error separates compression methods more clearly. Applying full-aggregate parallel suppression during from-scratch pretraining lowers validation loss and improves downstream averages, with the value-space variant strongest. Code is released on GitHub.

arXiv cs.AI / cs.LG / cs.CL·Sep 14, 17:56 UTC · 2d agoAI research 2 sources1

35

Understanding the Impact of Model Pruning on Long-Tail Forgetting and Explanation Reliability in Medical Imaging

Systematic study finds model pruning causes frequency-dependent long-tail forgetting in medical imaging and that gradient-informed methods best preserve explanations.

Across two long-tailed medical imaging datasets, two CNN architectures, four pruning methods, and sparsity up to 95%, the study measures predictive performance, explanation stability, and faithfulness. Rare classes degrade earlier and more severely than frequent ones, while explanation reliability depends mainly on the pruning strategy, with gradient-informed methods degrading least. Mechanistic analysis ties explanation collapse to loss of class-discriminative gradients rather than vanishing feature activations, recommending class- and explanation-aware evaluation of compression.

arXiv cs.AI / cs.LG / cs.CL·Sep 7, 17:44 UTC · 9d agoAI research

27

BeaconKV: Key-Value Cache Compression Guided by Beacon Queries for Efficient Large Reasoning Model Inference

BeaconKV introduces training-free KV cache compression using beacon queries, cutting long-reasoning inference memory up to 5.8x while preserving accuracy.

The paper shows recency-based KV cache compression assumptions fail in long-horizon reasoning because Thought Revisiting Tokens (TRT) re-attend to distant context such as early task-solving plans. TRT queries cluster into a small number of similarity groups, which BeaconKV exploits by maintaining compact beacon query representatives to anticipate revisited KV pairs without storing full query history. The training-free method achieves up to 5.8x memory reduction and over 4.3x throughput improvement across four open-source large reasoning models while nearly preserving full cache accuracy.

Hugging Face daily papers·Sep 3, 20:00 UTC · 13d agoAI research1

33