Continual-ARC measures lifelong skill learning on repeated ARC-AGI tasks
Original title: Continual-ARC: Measuring Skill Acquisition over a Working Lifetime
Harness persistence rarely beat caching demonstrations; weight updates helped unseen tasks, harness methods did not.
Continual-ARC streams tasks from three ARC-AGI editions, returning tasks after dormant gaps with fresh instances while new tasks arrive. Learners may buy demonstrations, submit answers, or use a harness, and are scored by feedback budget used, then frozen and tested for memorization and generalization. On Qwen3.5-9B and GLM-5.3 Flash, persistence without expert guidance generally did not improve memorization, and no harness method gained on unseen tasks.
Unconfirmed: Findings are from the authors' runs; no external replication is described.
- Memory-based and weight-based systems share one scoring scale.
- Most harness methods learned less than a demonstration-cache floor.
- Qwen3.5-9B weights were also trained to imitate GLM-5.3 Flash.
- Weight-based learning showed substantial gains on unseen tasks.
- Products
- Continual-ARC
- AI models
- Qwen3.5-9BGLM-5.3 Flash
From the original article
1 min read at huggingface.coAgent harnesses have emerged as a promising mechanism for continual learning, alongside test-time parametric learning. However, the current benchmarks have several limitations: they generally focus on a single applied domain, serve each instance only once, and assume a single family of learner, so memory-based and weight-based systems are rarely compared on the same scale.
By Daniel Lenton, Yusha Arif, Haris Mahmood · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.