Skip to content
NewA calmer, faster ZeroHour
StoryAI researchinfo

Continual-ARC finds harness methods fail on unseen tasks

2 sources · 2 articlesFirst reported Updated ()1
What's new

First merged summary.

Merged summary · x-ai/grok-4.7 · rewritten as coverage arrivesSummaries by Vani AI

On Continual-ARC, harness methods do not gain on unseen tasks, while weight-based learning shows substantial gains.

Continual-ARC is a lifelong stream built from all three ARC-AGI editions. Tasks return after dormant gaps with fresh instances while new tasks arrive. Learners may buy demonstrations, submit answers, or act in a harness, and each instance is scored by the share of the feedback budget used. On Qwen3.5-9B and GLM-5.3 Flash, harness persistence generally does not improve memorization without explicit expert guidance, and most harness methods learn less than a cache of each task's demonstrations. No harness method gains on unseen tasks, while weight-based learning shows substantial gains.

Key facts
  • Tasks come from all three ARC-AGI editions and need no domain knowledge.
  • Models include Qwen3.5-9B, GLM-5.3 Flash, and a Qwen3.5-9B trained to imitate GLM-5.3 Flash.
  • Incorrect submission retries are allowed and charged against the feedback budget.
  • Memory-based and weight-based systems share one scoring scale.
  • Frozen learners are tested on fresh seen-task instances and on unseen tasks.
Explain this AI news with Vani AIWhat was announced, what is new and why it matters, in a new chat

What each outlet adds

beyond the shared facts
  • arXivAdds retry charging, imitation training, and no domain knowledge.
  • Hugging Face daily papersAdds that memory and weight systems share one scoring scale.

Coverage

oldest first · one row per article · UTC
  1. · 6d ago arXiv cs.AI / cs.LG / cs.CLFirst report

    Continual-ARC scores lifelong skill acquisition on ARC-AGI streams

    Harness persistence rarely beats a demonstration cache on memorization, and no harness method gains on unseen tasks, unlike weight updates.

  2. · 6d ago Hugging Face daily papers

    Continual-ARC measures lifelong skill learning on repeated ARC-AGI tasks

    Harness persistence rarely beat caching demonstrations; weight updates helped unseen tasks, harness methods did not.