Search: “test-time-reinforcement-learning”

Smart search ranks by meaning as well as keywords (one row per story, last 45 days).

ScienceBuddy: Recursive-in-Recursive Self-Improvement for Interactive Scientific Agents

ScienceBuddy couples harness evolution with model reinforcement learning so scientific agents continually self-improve from researcher feedback in an interactive workspace.

The authors release ScienceBuddy, an interactive scientific research workspace that transforms researcher requests, feedback, and execution evidence into tasks and evaluation rubrics for continual learning. Its recursive-in-recursive self-improvement paradigm couples harness evolution (inner recursion, model fixed) with model reinforcement learning under the improved harness (outer recursion). Case studies span four scientific task families, and the system is released to the scientific community as a research product.

arXiv cs.AI / cs.LG / cs.CL·Sep 15, 17:55 UTC · 20h agoAI research 2 sources

35

Entropy-Regularized Rank-Masked Policy Optimization for Test-Time Reinforcement Learning in Code Generation

Researchers propose ERPO, enabling test-time reinforcement learning for code generation via probe-executed consensus rewards, rank masking, and entropy regularization.

The paper introduces probe-driven test-time reinforcement learning (TTRL) for code generation, where output-free probe inputs are constructed from problem statements and candidate programs are executed on them to compute a Probe Consensus Reward (PCR). Because PCR can be gamed through spurious consensus, the authors propose Entropy-Regularized Rank-Masked Policy Optimization (ERPO), which turns low-PCR outcomes into conservative negative updates via rank masking and constrains policy drift with an entropy ceiling. On coding benchmarks, ERPO substantially improves pass@1 and pass@k in both in-domain adaptation and zero-shot transfer.

arXiv cs.AI / cs.LG / cs.CL·Sep 8, 17:54 UTC · 7d agoAI research1

38

ExecCritic: Learn to Test, Test to Improve for Coding Agents

ExecCritic separates test generation from patching for coding agents, lifting SWE-bench Verified resolution to 72.6%.

ExecCritic pairs a test-verify-revise scaffold with role-specific reinforcement learning: a Test agent writes repository-native tests and a Repair agent fixes code from execution feedback, both using Qwen-3.5-35B-A3B backbones. Post-trained Qwen agents compose to 72.6% on SWE-bench Verified, an 11.4-point gain over the 61.2% no-test baseline, without stronger-model or oracle feedback at evaluation time. The work shows test quality is the key variable: base-agent tests lowered resolution to 57.3% while GPT-5.6-sol tests raised it to 65.3%.

arXiv cs.AI / cs.LG / cs.CL·Sep 8, 17:53 UTC · 7d agoAI research1

36

Online Change-point Detection for Cooperative Multi-Agent Reinforcement Learning

Researchers propose Patterns of Past Rewards (PPR), a lightweight reward-based detector that flags environment shifts in cooperative multi-agent reinforcement learning training.

The paper introduces Patterns of Past Rewards (PPR), an algorithm-agnostic detector that smooths cooperative agents' return streams and applies statistical drift testing to flag environment or task changes. Evaluation in a custom Speaker-Listener environment built on the Multi-Agent Particle Environment under two non-stationarity scenarios shows PPR balances detection speed against alarm stability. It avoids the repeated alarms of a smoothed-return baseline and the missed shifts of raw-return detection, enabling MARL systems to reliably identify major changes during training.

arXiv cs.AI / cs.LG / cs.CL·Sep 4, 15:49 UTC · 11d agoAI research1

20

Environments as Scaffold: Enriching Feedback to Bootstrap Self-Evolving Agents in Long-Horizon Tasks

Researchers propose Feedback-Enriched Environments (FEEs) that reduce reward sparsity and improve RL training of Qwen3-based agents on SciWorld and BFCL.

The paper proposes shifting from agent-side warmup (SFT) to environment-side adaptation via Feedback-Enriched Environments to address severe reward sparsity in RL training of long-horizon LLM agents. A pilot study defines a feedback strategy that transitions from action guidance to observation enrichment in later training stages. Large-scale experiments on SciWorld and BFCL across Qwen3 model scales and GRPO, GSPO, and DAPO show consistent gains, plus stabilized training dynamics and proactive exploration.

Hugging Face daily papers·Sep 7, 20:00 UTC · 8d agoAI research

25

Bellman Policy Optimization

Bellman Policy Optimization, a critic-free RLVR method derived from Policy Mirror Descent, improves LLM mathematical reasoning without intermediate state-value estimation.

The paper introduces Bellman Policy Optimization (BPO), a critic-free reinforcement learning method for LLMs with verifiable rewards, derived from Policy Mirror Descent. BPO uses the Bellman equations to reformulate PMD as a trajectory-level objective for autoregressive generation with terminal rewards, avoiding state-value estimation at intermediate states. The authors prove BPO shares the same unique optimal solution as PMD and validate it on mathematical reasoning benchmarks.

arXiv cs.AI / cs.LG / cs.CL·Sep 14, 17:59 UTC · 1d agoAI research

38

When Agents Slow Down: Understanding LLM Agents' Test-Time Strategies via Elo-per-token Analysis

Elo-per-token analysis shows LLM agents' marginal gains drop below independent sampling at scale; parallel sessions beat one long session.

The paper proposes Elo-per-token analysis, using a Bradley-Terry model to measure how agent performance scales with token budget on open-ended tasks with continuous scoring. Across four agents and four benchmarks with sessions up to 100M tokens, agents initially convert tokens to Elo faster than independent sampling but eventually slow below the linear-in-log-compute reference. The authors define a scaling inflection point and show that splitting 100M tokens across parallel sessions on FrontierCS Polyomino Packing gains +264 Elo over one long session and +355 over ten short sessions. Human contestants on shared AtCoder Heuristic Contest tasks improve superlinearly, indicating headroom over current agents.

Hugging Face daily papers·Sep 13, 20:00 UTC · 2d agoAI research3· 2 reads

48

T1: Terminal Agent Reinforcement Learning for Long-Horizon Tasks

T1, a 122B MoE terminal agent trained with reinforcement learning, reaches 64.0% on Terminal-Bench 2.1, surpassing GPT-5.4 and GLM-5.1 on long-horizon tasks.

T1 is a 122B mixture-of-experts model trained with reinforcement learning to operate a real shell in a cloud sandbox for up to 300+ tool-call turns per task, rewarded by executing each task's own verifier. The recipe combines aggressive warm starts, dense process rewards, TITO construction, and rollout routing replay, cutting the training-to-inference log-probability difference from 0.021 to 0.013 with zero token drift. Training used an out-of-distribution corpus disjoint from Terminal-Bench 2.1. Post-training raised the base model from 43.8% to 64.0% resolved on Terminal-Bench 2.1 and 27.9% on Long-Horizon Terminal Bench.

Hugging Face daily papers·Sep 9, 20:00 UTC · 6d agoAI research1

55

Difficulty-Adaptive Tree-Structured Policy Optimization for Expanding Reasoning Coverage in RLVR

Researchers propose DATPO, a difficulty-adaptive tree-structured RLVR training method that expands reasoning coverage (pass@k) and improves test-time scaling on math benchmarks.

The paper identifies three rollout design principles for RLVR: difficulty-adaptive rollout expands pass@k, tree-based rollout beats parallel sampling, and sentence-entropy-guided forking overcomes token-level branching localization. DATPO combines difficulty-adaptive tree search with a sibling-diversity advantage term to promote semantic diversity during training. On mathematical reasoning benchmarks, DATPO outperforms baselines in pass@k, directly translating to superior test-time scaling performance.

Hugging Face daily papers·Sep 7, 20:00 UTC · 8d agoAI research

30

Not All Prompts Are Equal: Exploration-Guided Prompt Scaffolding for Multimodal Reinforcement Post-Training

Exploration-guided prompt scaffolding rewrites training prompts by Exploration Potential Score, boosting multimodal RL post-training accuracy up to 11.5%.

The paper proposes dynamically adapting the training prompt distribution during online RL post-training of multimodal LLMs using the Exploration Potential Score (EPS), a lightweight rollout-based proxy for prompt utility computed from on-policy statistics with no additional overhead. Rather than discarding low-utility prompts, a teacher model generates scaffolded rewrites that preserve task intent while making training more informative. Integrated with GRPO on Geo3K and MMK12, the method achieves up to 9.7% relative in-domain improvement plus 11.5% on MathVision and 11.1% on MMMU-Pro.

Hugging Face daily papers·Sep 13, 20:00 UTC · 2d agoAI research

35

Near-Optimal Reinforcement Learning with Multi-Step Transition Lookahead

Theorists prove multi-step lookahead RL planning is NP-hard for every fixed rational discount factor yet give a randomized polynomial-time approximation scheme.

The paper resolves open questions about reinforcement learning with multi-step transition lookahead. It shows exact planning remains NP-hard for every fixed rational discount factor in (0,1), not just discounts arbitrarily close to one, and introduces a randomized polynomial-time approximation scheme for every fixed lookahead depth. Extending to unknown transitions and stochastic rewards via optimism and variance-adaptive confidence bounds, the algorithm achieves cumulative regret matching classical tabular discounted RL up to logarithmic factors.

arXiv cs.AI / cs.LG / cs.CL·Sep 10, 16:58 UTC · 5d agoAI research1

22

Groupoid-Based Internal State Representations for Reinforcement Learning with Local Symmetries

Groupoid-based RL discovers local, state-dependent symmetries during interaction, learning in a symmetry-reduced space and beating standard Q-learning efficiency.

The paper proposes a reinforcement learning framework using groupoids to capture local, state-dependent symmetries and dynamically discover equivalence structures during interaction. The agent maintains orbit representatives together with transporters that map raw states to canonical forms, enabling learning and decision-making in a symmetry-reduced space while preserving local distinctions. Empirical results show improved sample efficiency and convergence over standard Q-learning in dense and large-scale environments with strong partial symmetries.

arXiv cs.AI / cs.LG / cs.CL·Sep 11, 16:32 UTC · 4d agoAI research1

20

Drift-Constrained Optimization: Only Direction Matters in Fine-Tuning Instruct Models

Drift-Constrained Optimization reformulates fine-tuning as update-direction selection, letting Qwen3 models improve target tasks within a behavioral drift budget.

The paper specifies a behavioral drift budget before optimization and shows that update direction is the remaining degree of freedom, reformulating fine-tuning as a direction-selection problem. In a stringent QA-only setting where instruct models must still generate multi-step reasoning at inference, a coarse layer-selective probe reverses the failure of QA-only fine-tuning. Across Qwen3-8B and Qwen3-14B, these directions substantially improve scientific reasoning and multilingual translation, matching or outperforming dedicated translation systems over 100+ languages and giving stronger initialization for reinforcement learning.

Hugging Face daily papers·Sep 11, 20:00 UTC · 4d agoAI research

48

Beyond Training: A Feasibility Taxonomy for Inference-Time AI Governance

A study maps twenty inference-time AI governance mechanisms, finding commercial readiness only against cooperative deployers and no adequate defense versus state-level adversaries.

The paper develops a feasibility taxonomy of twenty inference-time AI governance mechanisms across monitoring, verification, and enforcement, each rated on a four-point readiness scale against a four-vendor evidence base. Fifteen of the twenty mechanisms have commercial technical substrates in production today, though governance-grade assurance and adversarial robustness vary substantially. Stress testing shows readiness holds only against a cooperative deployer and low-to-medium-capability user: no mechanism rates adequate against a high-capability state-level deployer, and fine-tuning removes model-internal enforcement components. A second-rater reliability check on readiness ratings returned a quadratic-weighted Cohen's kappa of 0.74.

arXiv cs.CR·Sep 9, 12:37 UTC · 7d agoAI policy

28

Coupled Calibration and Learning: Mitigating Teacher Bias in LLM Distillation without Target-Domain Reward Feedback

CCL couples teacher calibration with student updates via token-level branching, provably removing teacher bias in LLM distillation.

The paper proposes Coupled Calibration and Learning (CCL), an LLM distillation algorithm that alternates teacher calibration using source-question reward feedback with student training on target questions under covariate shift. Each iteration calibrates the teacher on source feedback, trains the student on target questions, and lets the updated student inform subsequent calibration. The authors prove the student's expected KL divergence to the oracle student converges to zero at a polynomial rate, and show regularized direct matching error can remain bounded away from zero.

arXiv cs.AI / cs.LG / cs.CL·Sep 15, 17:15 UTC · 21h agoAI research

20

Learning Length-Extrapolatable Recurrent Models

Researchers propose Credit Stabilization through Time, a training method letting recurrent models extrapolate up to 128x their training length.

The paper argues that length extrapolation failure in BPTT-trained recurrent models is better explained through state credit, the signal through which future losses reach earlier recurrent states. It introduces Credit Stabilization through Time (CST), which locally rescales the state-credit signal during backpropagation without rotating the corrected component or changing forward computation. Controlled experiments show improved performance beyond the training horizon, with gains at up to 128x the training length.

arXiv cs.AI / cs.LG / cs.CL·Sep 8, 17:59 UTC · 7d agoAI research1

25

Ask Before You Optimize: Dynamic Pre-Formulation Clarification for Interactive Optimization

Researchers release OR-Clarify, a benchmark testing whether LLM agents ask clarifying questions before formulating optimization models from incomplete requests.

OR-Clarify evaluates pre-formulation clarification in operations research: each task gives a partial problem description, withholds structured hidden slots, and scores agents via bounded interaction with a simulated user, measuring slot recovery, stopping behavior, silent assumptions, and interaction cost. The authors also propose InterOPT, a two-stage framework that identifies formulation-critical gaps to decide when to ask or stop. In choice-based experiments InterOPT substantially outperforms all baselines in exact slot recovery and remains competitive in the open-ended setting.

Hugging Face daily papers·Sep 3, 20:00 UTC · 12d agoAI research1

35

Learning Multimodal One-step Flow Policy via Value-weighted Optimal Transport

OptiFlow learns one-step multimodal flow policies for offline RL via state-wise entropic optimal transport, avoiding critic overestimation and mode collapse.

The paper introduces OptiFlow, a framework that frames one-step flow policy learning as a structured sample-allocation problem in offline reinforcement learning. It jointly trains a value-aware reference flow policy and a one-step policy, coupling action samples through state-wise entropic optimal transport where critic values set distillation priority and action-distance cost ensures geometrically compatible pairings. By avoiding direct critic maximization, it anchors the policy to high-value dataset-supported modes without out-of-distribution divergence. Code is released on GitHub and the method performs strongly across diverse offline RL benchmarks.

arXiv cs.AI / cs.LG / cs.CL·Sep 14, 17:10 UTC · 1d agoAI research

14

COBRA-Skills: Contextual Bandit-Guided Evolution for Agent Skill Optimization

COBRA-Skills uses contextual bandits to guide LLM agent skill evolution, cutting optimization cost 55-58% versus SkillOpt while topping six agent benchmarks.

COBRA-Skills formulates LLM agent skill optimization as budgeted sequential optimization over a dynamically evolving candidate space. It couples contextual-bandit-guided prioritization with evidence-grounded skill evolution, selectively spending execution-based evaluations on promising candidates while refining skills from feedback. Across six heterogeneous agent benchmarks and three target models, it achieves the strongest average performance while reducing optimization cost by 55-58% relative to SkillOpt using only 50 unique optimization examples per benchmark. The method remains robust to agent harness changes and works when the target model generates its own skills.

Hugging Face daily papers·Sep 9, 20:00 UTC · 6d agoAI research

30

Multi-Agent Reinforcement Learning for Autonomous UAV Exploration in Wildfire Response

Researchers trained multi-agent deep reinforcement learning UAV agents for autonomous wildfire monitoring, with converging policies tracking fire boundaries in simulation.

The study develops a deep reinforcement learning framework for training UAV agents to navigate and monitor simulated wildfire environments. Agents showed increasingly stable and effective behavior over time, evidenced by converging loss trends, improved rewards, and consistent navigation patterns such as fire-boundary tracking. The findings highlight DRL-based UAV potential for autonomous wildfire monitoring and show that environmental structure and reward design influence policy effectiveness.

arXiv cs.AI / cs.LG / cs.CL·Sep 9, 16:45 UTC · 6d agoAI research1

18

MInTRL: Off-policy Intervention can boost On-policy RL

MInTRL injects sparse judge corrections into on-policy RL rollouts, expanding exploration beyond on-policy sampling while preserving learnability on math and code benchmarks.

Minimal Intervention Reinforcement Learning periodically has a judge-intervention policy replace erroneous suffixes of the current policy's output with short corrections, then returns control, keeping trajectories largely on-policy. Training uses a sequence-level advantage-regression objective that removes the need for importance sampling. Across math and code benchmarks it consistently beats standard on-policy and off-policy baselines, remains effective with self-intervention, and performs best at moderate intervention intensity.

Hugging Face daily papers·Sep 10, 20:00 UTC · 5d agoAI research

40

Online Learning with LLM Experts from Limited Feedback

Paper proposes bandit algorithms for adaptively routing prompts to LLM experts, minimizing regret under limited feedback budgets.

The paper formulates adaptive prompt routing to K LLM experts as a contextual bandit problem with d prompt features over T rounds. Proposed algorithms strategically select actions and observe rewards, achieving O(dT/m) regret in the full-information setting and O(dTK/m) in the bandit setting, where m is the feedback budget. Experiments demonstrate efficient learning of high-quality routing strategies across diverse LLMs from limited feedback.

Hugging Face daily papers·Sep 4, 20:00 UTC · 11d agoAI research

25

Revisiting Complete Reasoning Traces for Post-Training

Researchers show full reasoning traces provide limited benefit in LLM post-training, with heavily truncated or endpoint-only trajectories performing comparably.

A pilot study plus attention-based analyses and controlled token-removal studies show intermediate tokens in reasoning trajectories contribute minimally to final reasoning quality. Partial trajectories remain effective even under heavy truncation, and training on endpoints alone leads to consistent changes in reasoning behavior. The finding also benefits reinforcement-learning and on-policy distillation post-training; code is released at github.com/naver-ai/revisiting-trace.

Hugging Face daily papers·Sep 6, 20:00 UTC · 9d agoAI research

32

AgenticGen: Reward-Guided Agentic Video Generation for Advertising

AgenticGen applies DPO and GRPO reward-guided reasoning to ad video generation, improving TikTok CTR 2.72%, CVR 2.63%, and Advv 9.61%.

AgenticGen decomposes advertising video generation into two trainable reasoning stages, strategy selection and draft generation, supervised by online business feedback. It learns a performance-based reward from accumulated online feedback plus a rubric-based reward aligned with human quality standards, then optimizes policies with DPO followed by GRPO using process and outcome rewards. Online A/B experiments in the TikTok advertising system show CTR up 2.72%, CVR up 2.63%, and Advv up 9.61% over an SFT baseline.

Hugging Face daily papers·Aug 30, 20:00 UTC · 16d agoAI research

26

Expert-Space Exploration in MoE Reinforcement Learning

ESRL explores MoE expert-routing space during RL post-training, improving Qwen3-30B-A3B Pass@1 by 3.2 points over GRPO without extra compute.

The paper shows perturbing expert routing increases rollout diversity similarly to higher decoding temperature, but naive perturbation degrades quality. ESRL anchors high-confidence experts, restricts stochastic routing to a plausible candidate pool, adapts perturbation strength via router entropy, and replays recorded expert paths during policy optimization. It achieves the best results across top-K, top-1, and shared-expert MoE backbones on math, science, and code tasks; on Qwen3-30B-A3B it improves average Pass@1 and Pass@8 over GRPO by 3.2 and 4.5 percentage points.

Hugging Face daily papers·updated Sep 11, 16:58 UTC · 4d ago·first Sep 10, 20:00 UTC · 5d agoAI research 2 sources

45

Eliciting Weak-to-Strong Generalization with On-Policy Reverse Distillation

OPRD distillation enables weak-to-strong generalization by amplifying verifier-supported policy updates, outperforming existing RL and distillation methods with fewer student updates.

On-Policy Reverse Distillation (OPRD) evaluates a weak teacher's policy shift relative to its reference policy on student rollouts and amplifies the verifier-supported component of the student's policy gradient. This rescaling preserves the stationary points of policy optimization while letting the student learn beyond the teacher's capacity ceiling. In successive model transfer and multi-teacher distillation, OPRD achieves higher performance with fewer student updates than existing RL and distillation approaches, and response-style analysis shows students remain closer to verifier-RL-trained models than to their weak teachers.

Hugging Face daily papers·Sep 7, 20:00 UTC · 8d agoAI research

33

Searching for New Physics with Reinforcement Learning

Researchers apply reinforcement learning to identify SMEFT operators explaining particle physics anomalies, reproducing and improving known CDF W-mass results.

The paper introduces a reinforcement learning method to search the large Standard Model Effective Field Theory (SMEFT) operator space for explanations of measurement anomalies. It was validated on the CDF W-mass anomaly, reproducing and improving known results, then applied to a harder multi-anomaly scenario. RL efficiently navigates complex loop-level operator correlations that bias human-driven phenomenological analysis.

arXiv cs.AI / cs.LG / cs.CL·Sep 9, 16:07 UTC · 6d agoAI research

22

Learning to Solve Hard Problems in RL for LLMs by Never Giving Up

Paper introduces Never Give Up adaptive sampling, fixing RL's 'Matthew Effect' where compute is wasted on easy problems and hard problems see little improvement.

Researchers identify a 'Matthew Effect' in reinforcement learning for LLMs, where RL yields large gains on easy problems but minimal improvement on hard ones because compute is misallocated. They propose Never Give Up (NGU), an adaptive sampling method that keeps generating samples for a problem until one is correct, using asynchronous RL to filter easy problems cheaply and concentrate compute on hard ones. NGU improves performance per compute on the Deepscaler math benchmark and iteratively solves the Manufactoria coding task where standard GRPO with per-test reward fails.

Hugging Face daily papers·Sep 10, 20:00 UTC · 5d agoAI research

45

NeoHorse-1: Towards Recursive Self-Improvement via Agentic Post-Training with Routing Harness

NeoHorse-1 introduces agentic post-training with intelligent routing that lifts agent benchmark scores at 4B and 9B scales, prototyping recursive self-improvement.

NeoHorse-1 is a family of agent-native models trained through agentic post-training: routing-harness logs (predicted capability demand, service tier, interaction) become structurally validated training data organized into a three-stage SFT curriculum plus routing-guided on-policy distillation. Capability-guided allocation converts evaluation feedback into the next training mixture, closing an evaluation-selection-update loop. Post-training raises the macro-average from 58.94 to 64.87 at 4B and from 65.60 to 69.04 at 9B across eleven agent, tool-use, coding, and instruction-following benchmarks. The authors position it as a prototype of harness-mediated recursive self-improvement.

Hugging Face daily papers·Sep 7, 20:00 UTC · 8d agoAI research

42