Language Models Learn to Improve Language Models
LM2LM uses a research world model as an RL reward so Qwen3.8-27B proposes stronger model improvements than GPT-6-sol.
The paper proposes LM2LM, which uses reinforcement learning to train an LLM to generate effective language-model improvement proposals. A frozen research world model predicts gains from experimental records and serves as a cheaper reward signal, while highly ranked proposals are periodically executed and their measured outcomes added to context. Across eight autoregressive and diffusion development environments, Qwen3.8-27B trained with LM2LM outperformed GPT-6-sol even when that model was given 300 real experimental records, with 33% lower data-generation cost, and generalized to an unseen model family.
- LM2LM trains an LLM with RL to propose language-model improvements.
- A frozen research world model predicts proposal gains as the reward.
- High-ranked proposals are periodically executed and added back to context.
- Qwen3.8-27B beat GPT-6-sol given 300 real records, at 33% lower data cost.
- The method generalized to a model family unseen during training.
- AI models
- Qwen3.8-27BGPT-6-sol
From the original article
1 min read at arxiv.orgLarge language models (LLMs) are increasingly used to automate AI research, including language-model (LM) development, offering a potential path toward recursive self-improvement. However, turning experimental feedback into a learned ability to propose effective improvements across models faces two challenges.
By Minhua Lin · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.