ACLArena: Agent Continue Learning in Multi-stage Post-training
ACLArena compares agent continual-learning methods and proposes replay plus routed LoRA experts to limit forgetting.
ACLArena is a framework for studying agent continual learning when capabilities are acquired in separate post-training stages. The authors analyze forgetting and generalization at both model and token levels, then compare multi-teacher on-policy distillation, self-distilled fine-tuning, and model merging. Their recipe combines offline replay of high-quality trajectories with a routed network of LoRA experts specialized via reinforcement learning. Experiments on four reasoning and agentic tasks, in-domain and out-of-domain, show improved retention of earlier skills while learning new ones.
- ACLArena studies forgetting and generalization at model and token levels.
- Compared methods include multi-teacher on-policy distillation, self-distilled fine-tuning, and model merging.
- The proposed recipe pairs offline replay of high-quality trajectories with routed LoRA experts trained by RL.
- Evaluation covers four reasoning and agentic tasks, in-domain and out-of-domain.
Full article194 words · extracted from huggingface.co · click to collapse
Building general-purpose agents for industrial deployment requires integrating multiple capabilities, each typically acquired at a distinct stage of training. Yet there is currently no well-established recipe for Agent Continual Learning (ACL), with little understanding of the trade-offs among existing integration paradigms. To address this gap, we introduce ACLArena, a framework for comprehensively studying, analyzing, and evaluating ACL. We first build a sequential training pipeline and conduct an in-depth analysis that explains the mechanisms of forgetting and generalization from two complementary perspectives, the model level and the token level. Guided by these analyses, we systematically compare multi-teacher on-policy distillation, self-distilled fine-tuning, and model merging to assess their ability to recover previously learned capabilities while preserving newly acquired ones. Through extensive experiments, we develop a detailed understanding of how capabilities transfer across stages. Finally, we propose a new ACL recipe that combines offline replay over high-quality trajectories with a routed network of multiple LoRA experts each specialized via RL, substantially improving the agent's ability to learn across multiple domains. Comprehensive experiments on four reasoning and agentic tasks, evaluated under both in-domain and out-of-domain settings, demonstrate the value of our analysis and the effectiveness of our approach.
Text extracted automatically; images, tables and formatting may be missing. Original: https://huggingface.co/papers/2609.23989