Language models err more on composed tasks via implicit token budgets
Original title: On the Fragility of Tasks under Composition: Implicit Budgeting in Language Models
Across open-weight models and Claude Opus 5, grouping fragile tasks cut tokens 43% without losing accuracy.
Models that solve tasks reliably alone make substantially more errors when several tasks share one prompt. The paper says the drop is not just accumulated independent errors: under composition, models spend fewer tokens per task even inside the context window. Longer isolation traces are more fragile. Grouping by fragility reduced token use by 43% on math and linguistic tasks while preserving accuracy.
Unconfirmed: The fragility effect and 43% token cut are reported by the paper, not independently confirmed.
- The effect appears in open-weight models and Claude Opus 5.
- Under composition, models use substantially fewer tokens per task.
- Tasks with longer isolation solution traces are more fragile.
- Non-fragile tasks waste compute by over-reasoning in isolation.
- Grouping by fragility cut token usage 43% while preserving accuracy.
From the original article
1 min read at huggingface.co
Modern language models are increasingly asked to handle several tasks within a single prompt. Doing so requires solving each task correctly while integrating the resulting answers into a coherent response. In this paper, we identify a fundamental limitation in this process, where models make substantially more errors when tasks are composed, even when they can solve every constituent task reliably in isolation.
By Chenpan Li, Thies Brockmoeller, Stefanie Jegelka · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.