Beyond Policy Rollouts: Embedding-Space Directional Exploration with Anchored Noise for LLM RLVR
EDEN adds anchored embedding noise to RLVR rollouts, improving exploration and solution coverage over GRPO.
EDEN (Embedding-space Directional Exploration via anchored Noise) improves reinforcement learning with verifiable rewards by replacing a small fraction of standard rollouts with generations from the same prompt plus one perturbed embedding position. Gaussian noise creates variation and a shared anchor sets a common direction within each prompt group, while selected rollouts contribute to GRPO with downweighted advantages. Architecture, discrete decoding, and test-time inference are unchanged. Across model scales and mathematical reasoning benchmarks, EDEN beats GRPO and the tested exploration controls, with the largest gains in sampled solution coverage.
- EDEN perturbs a shared anchored embedding during a fraction of rollouts.
- Model architecture, decoding, and test-time inference stay unchanged.
- Perturbed rollouts enter GRPO with downweighted advantages.
- Largest reported gains are in sampled solution coverage.
- Noise and anchoring play complementary exploration roles.
From the original article
1 min read at huggingface.co
Reinforcement learning with verifiable rewards (RLVR) typically uses small rollout groups per prompt, where repeated sampling from the same policy can yield highly similar trajectories, reducing within-group reward variation and weakening group-relative advantage signals.
By Daijiao Liu, Huiyi Wang, Lina Yao · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.