ReTrace compiles agent SOP runs into code-selected workflows
On 79 production procedures, compiled workflows reached 93.8% completion versus 96.8% for the agent at 24% lower cost.
Stories per day · 30 days
15 RSS
Relatedself-distillation5image-generation4language-models4on-policy4reinforcement-learning4video-generation4autoregressive3robotics3world-models3anthropic2diffusion2flux2image-editing2ipo2llm-agents2lora2model-training2multimodal2
164 stories, newest first · one card per story
Stories tagged #distillation
APEX few-step streaming video drops the extra fake-score network
Condition shifting estimates the generated velocity inside the generator and cuts peak GPU memory 4.4% and iteration time 9.2% versus DMD.
PhysOPSD uses executable code as privilege for video generation
Authors say the method beats supervised fine-tuning and reinforcement learning with 3,000 samples while keeping general video quality.
TOPSD limits teacher supervision to verified prefixes in distillation
It beats conventional on-policy self-distillation on math reasoning in ten languages, especially lower-resource ones.
On-Policy Distillation Lacks Generalization Beyond Supervised Data
MOPD does not fully transfer teachers' out-of-distribution capabilities from multiple teachers.
P^2-MOPD accelerates multi-teacher distillation with parameter priors
P^2-MOPD jointly optimizes mixing coefficients and student weights to reach 94.7% recovery in half the steps of MOPD.
History Forcing improves long-horizon video world models
History Forcing is a distillation framework that strengthens historical supervision for few-step autoregressive video generators, achieving state-of-the-art results.
SpecOPD branch-parallel distillation accelerates speculative decoding
SpecOPD parallelizes on-policy distillation across branches for up to 41.45% higher mean acceptance length and 3.56times generation speedup.
Replacement-Aware Distillation for Video Dubbing
VidDub distills a teacher into a two-step student using replacement-aware cross-audio supervision.
Evidence bypass identified in on-policy distillation for scientific QA
On-policy distillation improves answer accuracy but weakens evidence responsiveness, with a new EP-OPD method recovering much of the lost evidence control.
OC-OPD improves on-policy distillation via observation-coordinated supervision
OC-OPD uses reverse and forward KL divergences with a four-observation matrix to coordinate gradients, achieving strong gains in mathematical reasoning and OOD transfer.
Order-Only On-Policy Distillation improves LLM students
O^{3}PD trains students to match teacher token ordering without probability values or temperature tuning.
TinyAstral distills harness improvements to 4B robot agents
TinyAstral transfers gains from large teacher harnesses to compact students reaching near teacher performance on manipulation tasks.
Muon-RT Reuses Teacher Trajectory for Momentum-Free Muon
Muon-RT eliminates momentum buffers by reusing the teacher-student displacement in self-distillation.
sDMD adds adaptive Sinkhorn correction to diffusion distillation
With 2–4 steps on FLUX.1-dev, Qwen-Image-20B, and Wan2.1-14B, authors report gains over DMD2 and other baselines.
Calibrated distillation consolidates specialists in dynamic MoE models
On Turing-20B-A2B the method scores 0.9988 across four domains and beats Qwen3-8B on math and code.
Intrinsic Distribution Matching improves few-step image distillation
At two network evaluations, IDM scored 94.25% on Chinese and 90.20% on English LongText-Bench.
Teacher-Evolution Self-Distillation improves on-policy model training
TESD evolves self-teachers with task rewards to provide reliable guidance and down-weights unreliable samples.
Few-step image models raise unsafe output; DICE cuts it
On FLUX Schnell, DICE lowers the unsafe-generation rate to 0.046, versus at least 0.161 for prior erasure methods tested.
NLD-Uni unifies visual understanding and generation in a diffusion LLM
Cross-paradigm distillation and progressive token corruption are used instead of conventional masked diffusion.
OpenAI patches compaction smuggling used to steal model reasoning
Xint received a $6,000 bounty on October 23 after OpenAI confirmed and fixed the bypass of its August mitigations.
3
Paper describes numerical fixes for softmax and cross-entropy
The authors say the fixes remove the Slingshot effect and stop student logits drifting from the teacher in distillation.
MMVistaReason open recipe trains multimodal reasoners with less data
MVR-4B averages 72.8 on 15 benchmarks, beating Qwen3.5-9B Instruct and MMFineReason-8B with about 70% fewer samples.
ASTEROID distillation teaches in-context exploration under partial observability
Across 11 environments it reports about 2x returns, about 100x fewer interactions than RL, and +40% real-robot success.