Agent Plasticity: Measuring Self-Improvement Through Experience
Researchers define agent plasticity to measure how efficiently AI agents improve from experience.
The paper introduces agent plasticity, the efficiency with which an agent turns experience into gains on future held-out tasks. Agents store past interactions as reusable artifacts inherited by later instances, and checkpoints measure training and out-of-distribution performance while accounting for learning cost. Frontier models follow sharply different trajectories despite similar chances to learn: some improve persistently, while others stay near or below their starting performance. Final capability and learning efficiency also diverge, and failures trace to unused artifacts or limits in artifact quality, generalization, or application.
- Agent plasticity measures how efficiently experience improves held-out performance.
- Frontier models show sharply different improvement trajectories.
- Gains inside the training regime transfer only partly out of distribution.
- The strongest final agent is not always the most efficient learner.
- Low-plasticity agents often fail to reuse relevant artifacts.
From the original article
1 min read at huggingface.co
AI agents increasingly operate in environments where they can diagnose failures and improve through experience, yet existing evaluations largely measure what an agent can do at a fixed point in time rather than how effectively it learns.
By Harman Singh, Anton Bakhtin, Rulin Shao · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.