Strong-to-weak OPD achieves 50x compute efficiency versus GRPO when using own checkpoint
On-policy distillation with the student's own GRPO checkpoint reaches target accuracy with 50 times fewer FLOPs than GRPO training.
Stories per day · 30 days
2 RSS
Relateddecision-models1jevbench1language-models1looped-inference1on-policy-distillation1reinforcement-learning1
5 stories, newest first · one card per story
Stories tagged #compute-efficiency
LEV deepens decision models by looping layers instead of tokens
A 2.6B LEV rose from 63.2% to 76.2% on JevBench between depth one and depth four.
BudgetPix: Compute-Adaptive Tokenization for Pixel-Space Image Diffusion
BudgetPix adapts image-diffusion token budgets to visual complexity, matching larger models at a fraction of compute.
TasteVal: Measuring the Experimental Research Taste of AI Systems Against Human Experts
TasteVal finds Opus 5.5 beats expert baselines on experimental research taste with a 2.3x compute multiplier.
How Model Growth, Recursion, and Boundary Operators Influence Scaling Exponents
Researchers show model growth via looped transformers improves scaling exponents; a 7.4B architecture matches GPT-3 13B with roughly 20x less compute.