AutoScaffold builds training-time hints for language-model reinforcement learning
Scaffold-free students beat RL baselines on four tasks, including +7.2 ALFWorld success and +4.0 QA exact match.
Stories per day · 30 days
44 RSS
Relatedrobotics19qwen17multimodal16grpo15llm-agents14world-models11agents8benchmark8post-training8self-distillation8multi-agent7rlvr7coding-agents6credit-assignment6language-models6qwen36vision-language-action6alignment5
546 stories, newest first · one card per story
Stories tagged #reinforcement-learning
Theory says REINFORCE can collapse coverage in LM post-training
Resetting the KL reference in Approximate Policy Mirror Descent is shown to restore global convergence for stylized PPO and GRPO.
Off-policy RL entry scores 0.990 RDS on CoRL 2026 RoboEval
A π0.5 policy trained with AWR, RECAP, generated demos and failure recoveries also scores 0.987 on official seeds.
Audit says CAGE 4 defender scores measure less than assumed
Busy-agent handling, unseeded policies, and unpublished scoring choices make published CAGE Challenge 4 comparisons hard to reproduce.
ExRL adapts long-chunk robot policies by learning execution length
A lightweight off-policy RL adapter predicts how long to run each action chunk, cutting the adaptation space versus optimizing the full chunk.
EVAThink trains video models to reason when frame evidence is sparse
Using 6.7K curated samples, it improves the base model under both 32- and 64-frame budgets.
Matched compute reverses dense-reward gains over GRPO
Across 1,668 runs, a Monte-Carlo process reward trailed GRPO by 14.3 points at equal tokens on Qwen2.5-1.5B.
BAQ-TD keeps TD learning rates over rate-limited channels
Sending O(d log T) bits over horizon T is enough for the order-optimal 1/T rate, up to logarithmic factors.
Paper shows RL agents take profitable shortcuts despite feasible compliance
Under a 25% conflict mixture, an API task reached 97.9% certified profitable violations after 128 updates.
MaxIM frames incremental memory summarization as an RL reward-shaping problem
A variational mutual-information critic supplies dense rewards so bounded summaries stay predictive of future task targets.
Agentic-OPD recovers forgotten agent skills from discarded RL checkpoints
Authors report gains of up to 7.4 points across 24 benchmarks, with a 4B student beating GPT-5.6 on deep search.
LLM AutoML searcher jointly tunes architecture, hyperparameters, and data
Trained on one vision task, the searcher reports a 0.91 GenEval score after 896 GPU-days on a billion-parameter Diffusion Transformer.
Handshake AI podcast on what frontier labs buy for training data
The episode covers RL environments in biology, finance and alignment, plus Handshake’s claimed near-$1 billion annualized revenue.
1
WM-PPO values world-model predictions to improve sparse-reward agents
With a similar number of real steps, success rises from 63.5% to 87.5% on ALFWorld and from 39.3% to 52.9% on ScienceWorld.
Creative Priming separates the creative process from the response
A GRPO-trained priming model writes prompt-specific primers so a response model yields novel, high-quality, diverse outputs.
Paper bounds bandit regret when delayed rewards omit the round
Balanced Historical Optimism controls delay cost without optimal-arm reports; minimax regret is linear at c=1.
Paper measures mismatch on-policy distillation can tolerate
Model-specific FP8 recipes kept quality and fit at least 1.93x as many KV-cache tokens as BF16.
Offline policy optimization under all-policy linear realizability
The method uses one convex regression per layer and has sample dependence of order epsilon to the minus two.
Frontier coding LLMs reward-hack unseen repository tasks
With answer leaks blocked, models still mined git history, the object store, and the filesystem.
HQL balances one-step and multi-step returns for long horizons
A multi-step target is kept only when a dataset action beats best-of-N behavior-cloned actions.
Grasp-TAGS selects grasps via critic distillation for loco-manipulation
Grasp-TAGS ranks grasps by predicted downstream utility before contact in robot rearrangement tasks.
RL reward hacking mitigated via monitor-based interventions
Three environments show monitors suppress reward hacking but models evade them by exploiting probe flaws.
Quasipolynomial Approximate Planning for Decodable POMDPs
Introduces quasipolynomial-time approximate planning algorithms for ρ-decodable POMDPs, which are weaker than observability assumptions.
Video-Coder trains code-driven agents for video generation
Video-Coder achieves 7-point gains on PhyGenBench and 24-point gains on VBench-2.0 via agentic reinforcement learning