arXiv benchmark finds int8 quantization nearly matches float32 on text embeddings
int8 quantization retains 94.4% baseline effectiveness at 32x storage reduction across 38 MTEB tasks.
Stories per day · 30 days
4 RSS
Relatedagi1attention1chain-of-thought1classification1diffusion1dinov31elastic-inference1hierarchical-representations1mathematics1memory1mteb1omni-modal1open-weight-models1pruning1quantization1qwen3.51reasoning-distillation1rejection-sampling1
43 stories, newest first · one card per story
Stories tagged #efficiency
Route Anything framework improves AI system efficiency
Learned routers outperform fixed models by 0.3 points on 16 benchmarks while reducing policy time 24% and raising efficiency 8.82x on average.
Test-time memory evolution beats test-time training on math discovery
TTME updates external memory instead of weights and cuts FLOPs by 82.8–84.6% and runtime by 47.7–58.4%.
Learned Unmasking Policies Improve Diffusion Language Model Efficiency
A 856K-parameter policy trained on trajectory-contraction targets achieves best accuracy-NFE frontier on LLaDA-8B-Instruct.
MJev-Spatial: System One Spatial Intelligence via Native Attention Matching
MJev-Spatial classifies spatial answers on Qwen3.5-4B without decoding tokens, topping twelve benchmarks and running much faster.
Your Transformer Is Secretly Elastic
A retraining-free method makes pretrained transformers elastic, pruning smoothly to 60% sparsity with small accuracy loss.
CoTranslator: Reverse to Forward Chain-of-Thought Translator for Reasoning Distillation
CoTranslator turns reverse chain-of-thought into verified forward traces, cutting distillation compute while keeping most math accuracy.
I expect rapid progress but not towards general superintelligence
Essay expects rapid AI engineering and inference-efficiency gains, not general superintelligence.
2
When Do We Need On-Policy Distillation? Distilling on Offline Student Rollouts Is Often Better
Semi-OPD, distilling on offline student rollouts, beats on-policy distillation in 14 of 17 teacher-student pairs.
ForcingBooster: Adaptive Patch-wise Denoising for Fast Autoregressive Video Diffusion
ForcingBooster cuts autoregressive video diffusion steps about 50% by stopping easy patches early.
SanSi: A Looped Typed Decision Model for System 1.5 Thinking
SanSi adds looped System 1.5 thinking to typed decision models, reaching 72% accuracy.
Towards Looped Models Done Right, Part II: Rethinking at Fixed Points
Learned depth priors and orthogonal injection improve looped language models while enabling cheaper KV sharing.
Rethinking Long-Video Efficiency: A Joint Allocation Perspective on Frames, Pixels, and Front-End Latency
LoHi pairs dense low-resolution video with sparse high-resolution frames, raising long-video accuracy and cutting decode latency.
LESSER: Post-Training Data Selection with Output-Layer Gradients
LESSER cuts post-training data-selection cost by using output-layer gradients from forward passes, reducing FLOPs 9.7x for SFT while matching full-gradient performance.
Decoding Looped Transformers Better for (Almost) Free
Training-free LoopCD improves looped Transformer decoding and can cut recurrent compute by up to 48%.
cua-speedrun: Standardized Benchmarking of the Speed of Computer-Use Agents
cua-speedrun standardizes virtual-machine infrastructure to measure computer-use agent speed, cost, and efficiency.
LongHarness Bench: Stress-Testing Language Model Harnesses for Long-Context Reasoning
LongHarness Bench stress-tests long-context model harnesses; the best combination reaches 68% accuracy.
Video-RSI: Recursive Self-Improvement of Video Understanding Agents via Harness Evolution
Video-RSI lets video agents rewrite their own harnesses, improving accuracy while inspecting fewer frames.
Telescopic Language Models
Telescopic Language Models train one nested Transformer that is valid at every layer-prefix depth.
Beyond Selection: Token Parameterization for Extreme Visual Token Compression
Braco compresses visual tokens 23–144× while preserving VLM accuracy and cutting prefill FLOPs.
Just MLPs: Efficient Visual State Reconstruction for Multimodal Language Models
delta-Vision preserves every visual token and replaces repeated Transformer updates with lightweight MLP memory adapters.
SCOPD: Sparse-Context On-Policy Self-Distillation for Efficient Vision-Language Models
SCOPD self-distillation helps vision-language models retain accuracy when most visual tokens are pruned.
Change the Product, Keep the Parameters: Associative Algebra Layers for Transformers
Algebraic feed-forward layers lift small Transformer throughput 6–8% but cut downstream scores.
CoWindow Attention: Full Causal Coverage Is a Collective Property
CoWindow Attention covers causal history collectively, cutting 128K training latency up to 8.6x versus FullAttn.