DISCO: Distributed Long Context Scaling with Grounding-Reasoning Disaggregation
DISCO assigns long-context grounding to worker LLMs so a driver model avoids context rot.
DISCO disaggregates contextual grounding from reasoning to counter context rot in advertised million-token windows. Worker LLMs perform parallel localized extraction, while a central driver LLM trained with GRPO plans atomic tasks and synthesizes a final answer from gathered evidence. On RULER-QA at 1 million tokens it maintains 78.4% accuracy where standard baselines collapse, and it outperforms full-context models by up to 9.8 points on LongBench v2. It matches Gemini-3-Pro-Preview while reducing inference cost by more than 80%.
- Worker LLMs ground long context in parallel; a GRPO-trained driver synthesizes the answer.
- Holds 78.4% accuracy on RULER-QA at one million tokens as baselines collapse.
- Beats full-context models by up to 9.8 points on LongBench v2.
- Matches Gemini-3-Pro-Preview while cutting inference cost by more than 80%.
From the original article
1 min read at huggingface.co
While Large Language Models (LLMs) advertise million-token context windows, reasoning quality often collapses as inputs grow -- a phenomenon termed context rot. This failure stems from a structural entanglement in monolithic architectures, where the massive search burden of contextual grounding exhausts the representational capacity needed for complex reasoning.
By Guanzheng Chen, Viet Dac Lai, Subhojyoti Mukherjee · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.