Search: “DAPO”

ThinkPrior: Zero-Rollout Difficulty Priors for Cold-Start Prompt Selection in RLVR

ThinkPrior builds zero-rollout difficulty priors via an offline verifier-anchored pass, halving silent groups in RLVR and cutting wasted rollouts on Qwen2.5-Math-7B.

In GRPO-based RLVR, groups where all rollouts are correct or all are wrong yield zero advantages and consume about 39% of a run's rollouts under uniform sampling. ThinkPrior initializes a Beta posterior from an external anchor pass's verifier-scored pass rate, selecting prompts by expected learnability before any target-policy rollout, without changing the loss or optimizer. On Qwen2.5-Math-7B across sixteen seeds it more than halves early silent groups and cuts wasted rollouts through step 30 by nearly a fifth, with no detected final-accuracy difference. The ThinkPrior+DAPO composition reduces generated rollouts by 10.6% at an equal 3,840-rollout update budget.

arXiv cs.AI / cs.LG / cs.CL·Sep 8, 17:30 UTC · 8d agoAI research1

Environments as Scaffold: Enriching Feedback to Bootstrap Self-Evolving Agents in Long-Horizon Tasks

Researchers propose Feedback-Enriched Environments (FEEs) that reduce reward sparsity and improve RL training of Qwen3-based agents on SciWorld and BFCL.

The paper proposes shifting from agent-side warmup (SFT) to environment-side adaptation via Feedback-Enriched Environments to address severe reward sparsity in RL training of long-horizon LLM agents. A pilot study defines a feedback strategy that transitions from action guidance to observation enrichment in later training stages. Large-scale experiments on SciWorld and BFCL across Qwen3 model scales and GRPO, GSPO, and DAPO show consistent gains, plus stabilized training dynamics and proactive exploration.

Hugging Face daily papers·Sep 7, 20:00 UTC · 9d agoAI research

[AINews] 10% worse, 100x cheaper, 10000x faster: Why Simulation is taking over

Latent Space argues AI training pipeline stages—rewards, data, teachers, curricula, environments—are flipping from human-made to model-made simulation.

Latent Space's AINews essay traces how each component of AI training has turned synthetic since 2022: reward models (InstructGPT, RLAIF), synthetic pretraining data (Microsoft Phi, NVIDIA Nemotron-4 340B), model teachers (Alpaca, DeepSeek-R1 distillation), and self-generated curricula (Self-Rewarding Language Models, SPIN). In 2026 it highlights Karpathy's autoresearch loop—700 experiments yielding 20 kept improvements, cutting GPT-2 training time from 2.02 to 1.80 hours—and Z.ai's GLM-5.3 fully synthetic RL environment, judging, and verification stack. It frames these shifts as 'simulation': 10% worse but 100x cheaper and 10,000x faster than human equivalents.

Latent Space·Aug 22, 07:36 UTC · 25d agoAI industry

5 useful things you'll learn in my new post-training textbook (shipping now!)

Nathan Lambert's new RLHF and post-training LLM textbook covers PPO, GRPO, GSPO, CISPO and related techniques, freely available online.

Nathan Lambert's book 'Reinforcement Learning from Human Feedback: Aligning and Post-training LLMs' is now shipping from Manning. It covers policy-gradient algorithms including PPO, GRPO, GSPO, CISPO, and RLOO, plus loss aggregation, truncated importance sampling, asynchronous RL systems, and post-training topics like rejection sampling, outcome reward models, and on-policy distillation. The book is freely available online with a 12-hour course, codebase, and exercises.

Interconnects·Aug 10, 13:02 UTC · Aug 10, 2026AI research

Nigeria Suspends Official Charged with Defrauding US

Infosecurity Magazine·May 20, 14:58 UTC · May 20, 2021Phishing & fraud