Skip to content
NewA calmer, faster ZeroHour
StoryAI researchinfo

Continual Learning Exam finds agents rarely finish five-level tasks

2 sources · 2 articlesFirst reported Updated ()
What's new

October 25: arXiv reports the best setup cleared 47.7% of levels and that the 5.3-point gain was versus a single submission.

Merged summary · x-ai/grok-4.7 · rewritten as coverage arrivesSummaries by Vani AI

A new 600-level exam finds the best of 17 frontier LLM setups finishes all five levels on only 23.3% of tasks.

Researchers introduced Continual Learning Exam, a benchmark of 120 human-authored professional tasks built with more than 50 domain experts. Each task is a five-level ladder, totaling 600 levels across 89 roles and 12 occupational domains; agents must pass a level to unlock the next, and an evaluation agent scores work against human rubrics. Seventeen harness-model configurations of frontier LLMs were compared. The best cleared 47.7% of levels and completed every level of a task in 23.3% of cases. Diagnostic feedback plus one revision raised median completion by 5.3 percentage points versus a single submission. The authors say the set can also post-train models for continual learning.

Key facts
  • CLE has 120 tasks, 600 levels, 89 roles and 12 domains.
  • More than 50 domain experts authored the tasks and rubrics.
  • Best of 17 setups cleared 47.7% of levels.
  • That setup finished all five levels on 23.3% of tasks.
  • Feedback and one revision lifted median completion 5.3 points.
  • Later levels build on experience from earlier assignments.
Explain this AI news with Vani AIWhat was announced, what is new and why it matters, in a new chat

Timeline

dated events in the reporting · oldest first
  1. Oct 24, 2026Hugging Face daily papers posts the Continual Learning Exam study. · Hugging Face daily papers
  2. Oct 25, 2026arXiv listing adds level-clearance and revision-baseline figures. · arXiv

What each outlet adds

beyond the shared facts
  • Hugging Face daily papersSays no tested frontier setup finishes most five-level tasks.
  • arXivAdds 47.7% of levels cleared and the single-submission baseline.
Products
Continual Learning Exam

Coverage

oldest first · one row per article · UTC
  1. · 6d ago Hugging Face daily papersFirst report

    Continual Learning Exam benchmarks long-horizon learning in LLM agents

    Feedback plus one revision lifts median completion by 5.3 points, but no tested frontier setup finishes most five-level tasks.

  2. · 6d ago arXiv cs.AI / cs.LG / cs.CL

    Continual Learning Exam shows frontier agents stall on multi-level tasks

    The best of 17 setups cleared 47.7% of levels and finished all five levels on 23.3% of tasks.