Skip to content
NewA calmer, faster ZeroHour

Nvidia researchers argue huge AI context windows are a dead end

The Decoderby Jonathan Kemper ()1 min read1
AI summary · x-ai/grok-4.7Summaries by Vani AI

In many tests, periodic summaries and targeted lookups matched million-token windows while costing less, the researchers say.

Nvidia researchers say expanding language-model context windows to novel length is often unnecessary and costly. A context window is the working memory a model uses when generating a reply. Windows have grown from GPT-4's 8,192 tokens in March 2023 to about one million tokens in GPT 5.6 Sol. The team reports that periodic summaries plus targeted lookups performed similarly in many tests.

Unconfirmed: The article does not name the paper, the tests, or exact performance figures.

Key facts
  • One million tokens is roughly enough for Tolstoy's War and Peace.
  • GPT 5.6 Sol's window is about 120 times GPT-4's 8,192 tokens.
  • The researchers call full-document context often overkill and expensive.
Explain this AI news with Vani AIWhat was announced, what is new and why it matters, in a new chat

From the original article

1 min read at the-decoder.com

Nvidia researchers argue that language models often don't need their full context windows—which are slow, expensive, and prone to errors—and propose an alternative. In tests, periodic summaries and targeted keyword search usually matched full-context results at a fraction of the cost. The team even proposes a new class of models that works with short notes instead of long context.

The Decoder

By Jonathan Kemper · the-decoder.com

Read the full article at The Decoder

A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.