RSI-Prime dataset trains LLMs to run training experiments
Original title: RSI-Prime: Training Models to Train Models
Training GLM-5.3 on 350 annotated trajectories raised PostTrainBench performance 31.95% relative to the base model.
The paper introduces RSI-Prime, a dataset of 350 complete experimental trajectories across seven task categories for teaching language models to conduct training. Fine-grained decision labels and decision-level loss masking withhold supervision on low-quality behavior. On PostTrainBench, training GLM-5.3 with RSI-Prime produced a 31.95% relative gain over the original model. The authors present this as a step toward recursive self-improvement.
Unconfirmed: The 31.95% gain and implications for recursive self-improvement are the authors' experimental claims.
- The dataset spans seven task categories of model-training practice.
- Loss masking excludes low-quality experimental decisions from supervision.
- The reported comparison uses the original GLM-5.3 as the baseline.
- PostTrainBench evaluates autonomous post-training.
- Products
- RSI-PrimePostTrainBench
- AI models
- GLM-5.3
From the original article
1 min read at arxiv.orgEnabling large language models (LLMs) to autonomously conduct training and improve target models is an important foundation for recursive self-improvement (RSI). As LLMs expand their role from being trained to conducting training, their model-training capability warrants dedicated study and systematic development.
By Dewu Zheng · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.