AutoScaffold builds training-time hints for language-model reinforcement learning
Scaffold-free students beat RL baselines on four tasks, including +7.2 ALFWorld success and +4.0 QA exact match.
Stories per day · 30 days
2 RSS
Relatedreinforcement-learning7language-models4olmo3grpo2multimodal2self-distillation2ai-evals1alignment1claude1evidenceconnect1generalization1gpu-kernels1instrumental-convergence1llm-reasoning1out-of-distribution1policy-optimization1post-training1process-rewards1
33 stories, newest first · one card per story
Stories tagged #rlvr
Matched compute reverses dense-reward gains over GRPO
Across 1,668 runs, a Monte-Carlo process reward trailed GRPO by 14.3 points at equal tokens on Qwen2.5-1.5B.
RLVR models abandon reasoning strategies under OOD shifts
Co-training with bijective inversion tasks restores derivation and OOD accuracy.
FADE separates ability and difficulty for RLVR scheduling
Bilinear filter updates policy competence instantly while stabilizing item intercepts.
PIVOT mixes teacher corrections into RLVR student rollouts
Mean avg@8 on hard logic games rose from 19.5% for the best baseline to 35.6%.
EvidenceConnect data training produces ZwZ2 multimodal perception gains
Zoomed-region questions plus RLVR improved spatial, long-context, agent, and general visual reasoning without benchmark-specific data.
NYU and UT Austin study finds six patterns that make AI text hard to read
Across 14 models, post-training and verifiable-reward training pushed text further from human writing than base models.
1
OLMo-3 RLVR checkpoints show nonmonotonic instrumental behavior
On OLMo-3-7B-Think, the InstrumentalEval rate rose from 25.0% at SFT to 46.1%, then fell to 11.8% at the final checkpoint.
Visual-Attribution Gating filters contaminated corrections in visual RL
On six visual reasoning benchmarks, VAG beats GRPO and on-policy self-distillation on average.
Hindsight-Privileged RLVR: Single-Rollout Token Credit Without Group Sampling
HP-RLVR gives single-rollout token credit for RLVR, training Qwen3 faster than GRPO with comparable accuracy.
Function-space policy optimization improves RLVR training for language models
Authors report about 4 percentage points higher average accuracy than the strongest baseline, with peak memory falling from about 70 GB to 56 GB on Olmo3-7B-Instruct.
Beyond Policy Rollouts: Embedding-Space Directional Exploration with Anchored Noise for LLM RLVR
EDEN adds anchored embedding noise to RLVR rollouts, improving exploration and solution coverage over GRPO.
KV4-RL: Unlocking Faster Asynchronous Rollouts for RLVR with INT4 KV Cache Quantization
KV4-RL uses selective INT4 KV quantization to speed Qwen3 GRPO rollouts nearly 2x while keeping benchmark accuracy.
Decoupling Exploration from Optimization in RLVR
Exploration-Distillation separates novelty search from RLVR optimization and beats DAPO on math benchmarks.
Self-Retrospection Distillation: Turning Post-hoc Experiences into Prior Foresight
Self-Retrospection Distillation turns agent hindsight into foresight and lifts RLVR when rewards lack contrast.
Self-Retrospection Distillation: Turning Post-hoc Experiences into Prior Foresight
Self-Retrospection Distillation turns post-hoc agent trajectories into foresight, helping when rollout rewards lack contrast.
Semifactual Credit-Augmented Policy Optimization
SCAPO uses semifactual token stability to beat GRPO on AIME for two Qwen3 base models.
Reinforcing Multimodal Reasoning via Token-Level Perception-Grounded Advantage Estimation
TPAE uses token-level visual-dependency and entropy signals to improve multimodal RL reasoning over sequence-level rewards.
Semifactual Credit-Augmented Policy Optimization
SCAPO adds semifactual token stability to GRPO, lifting Qwen3 AIME accuracy by up to 5.63 points.
VStress: Correlation-Aware Auditing and Adaptive Budget Allocation for Repeated Verifiers
VStress-CA spends repeated verifier calls only when they add conditional information, improving accuracy under a fixed budget.
Learning to Steer, Steering to See: Unveiling the Geometry of RLVR in Large Language Models via Trainable Vectors
Vector steering finds a low-dimensional manifold behind RLVR gains and a stabilizer that preserves them.
On-Policy or Off-Policy Learning? A Systematic Study of Distillation Dynamics
Controlled distillation finds KL direction matters more than on-policy versus off-policy rollouts.
ReSPO: Reshaped Sequence Policy Optimization for Gradient Starvation in Off-Policy Learning
ReSPO replaces clipping in off-policy RL to fix gradient starvation on reused rollouts for Qwen3.
Can We Trust the Teacher? Decoupled Credit Direction-Magnitude for Self-Distillation
A new self-distillation method separates credit direction from magnitude and beats prior RL baselines.