One to More, More to One: Category-Aware Iterative Expert Training for Software Engineering Agents
Category-aware expert training and distillation lift a software-engineering agent to 58–59% resolution on SWE benchmarks.
The paper proposes category-aware expert training and policy integration for repository-level software engineering agents, addressing uneven gains across task categories under pooled reinforcement learning. SWE Labeler builds evidence-grounded training pools, and same-origin experts alternate Agentic-miniRL with Refresh-Repair-Expand, reusing verified successful trajectories for repair supervised fine-tuning. Label-routed multi-teacher on-policy distillation consolidates experts into one student without external solution trajectories. The final policy scores 58.04% mean resolution on Pro-618 and 59.00% on SWE-bench Multilingual, up 5.39 and 2.78 points over the base model.
- Category-specific RL plus Refresh-Repair-Expand addresses uneven progress across SWE task types.
- SWE Labeler organizes pools; experts need no external model for trajectories.
- Multi-teacher on-policy distillation merges experts into one student policy.
- MOPD reaches 58.04% on Pro-618 and 59.00% on SWE-bench Multilingual.
- Gains over the base model are 5.39 and 2.78 percentage points.
Full article210 words · extracted from huggingface.co · click to collapse
Repository-level software engineering (SWE) comprises heterogeneous task categories, whose progress under pooled agentic reinforcement learning can be uneven: gains in some categories coincide with regressions in others, while aggregate resolution obscures these changes. Motivated by this category see-saw, we develop a category-aware expert-training and policy-integration framework. Executable task construction and SWE Labeler, an evidence-grounded multi-axis labeling system, organize the training pools. Initial category-specific RL improves average training success while leaving uneven instance-level progress, motivating explicit consolidation of successful behavior and policy-adaptive task selection. Same-origin category experts alternate long-horizon Agentic-miniRL with Refresh-Repair-Expand (RRE): the updated policy refreshes instance mastery, reuses its own verified successful trajectories for Repair SFT, and reselects tasks for further RL. Label-routed multi-teacher on-policy distillation (MOPD) consolidates the experts into one deployable student, with ReLU-gated reward extrapolation keeping only each teacher's improving direction over the reference. Expert training and policy integration require no external model to provide solution trajectories or action targets. We evaluate Pooled RL and Balanced RL, expert development, and single-model integration through aggregate and per-category resolution, the minimum category lift over each joint-RL baseline, and expert-gain recovery. The final MOPD policy achieves mean resolution of 58.04% on Pro-618 and 59.00% on SWE-bench Multilingual, improving over the base model by 5.39 and 2.78 percentage points, respectively.
Text extracted automatically; images, tables and formatting may be missing. Original: https://huggingface.co/papers/2609.23377