InfraRSI harness applies recursive self-improvement to LLM training
On four dense and MoE workloads, the authors report up to 2.75x speedup and mean MFU rising from 17.4% to 30.0%.
Stories per day · 30 days
2 RSS
Relatedreinforcement-learning4mathematical-reasoning2on-policy2qwen32self-distillation2benchmarks1bf161coding-agents1copyright1criticopd1distillation1dpo1fair-use1gpu-clusters1humaneval1mismatch-correction1mixture-of-experts1numerical-stability1
31 stories, newest first · one card per story
Stories tagged #llm-training
SWE-Trail uses history to guide coding agent trajectory collection
SWE-Trail improves SWE-bench resolution by 2.33 points over independent sampling for a 124B student model.
Wikimedia urges courts to split AI training from chatbot copying
In an amicus brief, Wikimedia says unlicensed LLM pre-training can be fair use, but query answers that replace the source need a license.
MTA-OPSD lets each token pick its own teacher in self-distillation
On Qwen3 math tasks, the method beats prior on-policy self-distillation, with gains of up to 5.9 over the base model.
DLC correction stabilizes LLM RL training-inference mismatch
Discrepancy-lag correction centers numerical errors and clips lag ratios to prevent feedback loops in on-policy LLM reinforcement learning.
Aligning BF16 rounding locations stabilizes LLM reinforcement learning
Keeping RoPE, SwiGLU, and residual RMSNorm intermediates in FP32 prevented collapse on four Qwen-family models.
AOPD weights teacher actions by student advantage in on-policy distillation
The estimator needs only student rollouts and terminal rewards, with no extra rollouts or reward evaluations.
Stochastic DPO graph pruning speeds training about 1.4x
Branch pruning cut training time about 30% versus stock DPO, with slightly lower mean accuracy.
TraceForge writes student-style reasoning traces for distillation
With the same 5,000 questions and teacher answers, HumanEval+ rises 3.7 points and MBPP+ 7.1.
RLAWM penalizes physics modeling drift during LLM training
Authors report gains of up to 12.2 points on PhysReason and 5.2 on PHYSICS, with no added inference cost.
Recovering Reliable Supervision via Critique-Guided On-Policy Distillation
CriticOPD uses teacher critiques to restore recoverability in on-policy distillation, lifting two Qwen3 students on math benchmarks.
Beyond Policy Rollouts: Embedding-Space Directional Exploration with Anchored Noise for LLM RLVR
EDEN adds anchored embedding noise to RLVR rollouts, improving exploration and solution coverage over GRPO.
Composing What Each Teacher Learned: Multi-Teacher On-Policy Distillation through Teacher-Relative Shifts
Δ-MOPD distills multiple teachers via teacher-relative logit shifts instead of endpoint policies, beating endpoint composition by 4.11 Math points.
RollVerify: Bridging Efficiency and Accuracy in Long-Tail Rollout Reinforcement Learning
RollVerify verifies and repairs partial RL rollouts so language models keep on-policy accuracy while cutting training cost.
Self-Retrospection Distillation: Turning Post-hoc Experiences into Prior Foresight
Self-Retrospection Distillation turns post-hoc agent trajectories into foresight, helping when rollout rewards lack contrast.
TRACE: Rollout-Guided Quantization-Aware Training for FP4 Reinforcement Learning of MoE Language Models
TRACE aligns FP4 train and rollout paths for MoE RL, matching BF16 quality with up to 5.4x rollout speedup.
FALCON: A Model and Dataset Agnostic Framework for Synthetic Data Generation for NL2SQL Pairs
FALCON generates realistic, ambiguity-aware NL2SQL training pairs with compact open models, matching or exceeding challenging benchmark complexity locally.
Fixing GRPO's credit assignment problem without evaluating every step
An arXiv paper claims to fix GRPO credit assignment without scoring every training step.
Looping Beyond Twice: A Scalable Recipe for Looped Mixture-of-Experts
LOOM scales looped Mixture-of-Experts Transformers to 9-12 loops, cutting 1.7B model perplexity from 9.62 to 7.77 with accuracy gains.
Make Sparse Rewards Count: Density-Aware Reward Aggregation for Multi-Reward RL
DARA reweights multi-reward RL by reward density, reaching compliance faster than GDPO on tools and math.
EasyPPO: Stabilizing the Critic Is Key
EasyPPO stabilizes the PPO critic for LLM training and outperforms vanilla PPO on coding, math, and search.
Rethinking Training-Inference Mismatch in LLM Reinforcement Learning: Where It Arises and How to Correct It
Calibrated importance sampling corrects token-probability mismatch between LLM training and inference engines.
1% of Tokens Can Be Enough: On Gradient Estimation in On-Policy Distillation
An information-efficiency ratio lets sparse on-policy distillation match full supervision at 0.1%–1% of tokens.
Score Centering Stabilizes Off-policy Reinforcement Learning
Paper attributes RL instability under training-inference mismatch to accumulating drift and introduces score centering, an additive correction that stabilizes training.