Compressed 31B video diffusion model to 14B parameters
Hyperflow-H3-Flash reduces a 31B parameter video+audio diffusion model to 14B using 6 steps and INT8 quantization.
Stories per day · 30 days
2 RSS
Relatedadversarial-robustness1bci1coding-agents1diffusion1eeg1eegnet1embeddings1expert-pruning1gemma1llm-compression1mixture-of-experts1mllm1moe-expert-pruning1multimodal1optic1quantization1replacement-operators1self-improvement1
21 stories, newest first · one card per story
Stories tagged #model-compression
arXiv study shows coding agents can compress LLMs
29 of 36 agent submissions pass verification, achieving 77.33% downstream accuracy on compressed models.
MAEP prunes MoE multimodal experts by modality in one pass
The training-free method cuts about 27 GB of BF16 expert weights so an about 58 GB model fits a 40 GB GPU.
Replacement operators affect training-free MoE expert pruning
Survivors gate scale choice dominates effects in pruning OLMoE, DeepSeek-V2-Lite and Moonlight models; tilted novice operator adds 1.2-1.8% parameters and improves accuracy.
ORBITAL compresses EEG decoders under a robustness constraint
S25-TRADES reached 23.06% mean true-union robustness versus 21.27% for uncompressed Full EEGNet.
OPTIC compresses Gemma 4 E4B per-layer embeddings up to 1,002×
Authors report 97.8% of uncompressed performance at about 100× PLE compression, versus 66.3% for product quantization.
Layerwise Error Attribution for Fast and Robust Mixed-Precision Post-Training Quantization
Layerwise error scores speed mixed-precision quantization up to 2,570x and resist corrupted calibration data.
Paradee: Distilling Kokoro-82M into an 8M-Parameter Single-Voice Text-to-Speech Model
Paradee distills Kokoro-82M into an 8M single-voice TTS model scoring 4.41 UTMOS.
Depth as Time in One-Step Generative Models
Depth-as-time analysis shows one-step diffusion models reorganize temporal denoising across network depth, enabling 16.6x compression of MeanFlow SiT-L/2.
Six Layers Less: Encoder Pruning for Whisper with Label-Free Recovery
Six Whisper encoder layers are pruned and partly recovered with label-free distillation.
GeoPair: Geometry-Preserving Cross-Layer Factorization for Training-Free Transformer Compression
GeoPair compresses transformers training-free by pairing compatible layers and sharing factorized dictionaries.
Pruning LLMs Like a Physicist: Block Removal as an Ising Optimization Problem
Multiverse Computing frames LLM block pruning as an Ising optimization problem.
Not All Objectives Are Born Equal: Priority-Constrained Descent for Hierarchical Multi-Objective Optimization
Priority-Constrained Descent optimizes hierarchical objectives while limiting distortion of the primary gradient.
Neural Spectral Capacity: Measuring and Designing Architectures from Network Specification Alone
Neural Spectral Capacity scores architectures from their specs alone and prunes LLaMA-7B to 5.7B without calibration data.
Higher-order pruning of experts in mixture-of-experts language models
New HOPE method uses second-order objectives to prune Mixture-of-Experts LLMs, outperforming REAP at 50% pruning on models up to 122B parameters.
Label-Guided Knowledge Distillation for 3D-CNNs in Action Recognition
LGKD uses ground-truth labels to guide feature distillation for 3D-CNNs, combining sample-wise and class-wise distillation for action recognition.
X-AuT: Progressive Audio-Encoder Compression for Speech LLMs with Cross-Scale Distillation
XPeng AI's X-AuT prunes speech LLM audio encoders, cutting Qwen3-ASR-0.6B error from 5.61% to 5.27% with fewer parameters.
SQS: Bayesian DNN Compression through Sparse Quantized Sub-distributions
SQS unifies weight pruning and low-bit quantization via Bayesian variational learning, compressing Llama3.2 and Qwen2.5 at higher rates with comparable accuracy.
Lightweight Vision Transformer Compression for On-Device Plant Disease Detection in Resource-Constrained Agricultural Field Conditions
A unified ViT compression pipeline (H-BAC pruning, quantization, distillation) cuts plant-disease models 54.5x to 6.01 MB while keeping 95.13% accuracy.
Quantization-Aware Healing: a compressed, 4-bit model that outperforms its full-precision original
Multiverse Computing details quantization-aware healing, producing a 4-bit compressed model that reportedly outperforms its full-precision original.
Making Knowledge Distillation Cheap Enough to Run at Scale
Hugging Face blog by Multiverse Computing describes techniques making knowledge distillation cheap enough for large-scale training.