Periscope: Extending Frozen Language Models Beyond Their Context Window
Periscope lets frozen models answer million-token contexts via a training-free chunked evidence map.
Periscope is a training-free inference method that factorizes long-context decisions by scoring local and strided spans of chunked text with a frozen model and building an evidence map. On LongBench v2, reading only the highest-ranked chunks, about 9k tokens, matches the same model's best window read from 32k to 1M tokens. On InfiniteBench, where the median context is 150k tokens, it leads the best window read by 5 points, and it also ranks BRIGHT corpora with the best NDCG@10 of six methods. A 27B model can read 4.5M-token contexts on one 80GB GPU, versus an estimated 296GB cache for a single pass.
- Training-free method scores local and strided spans of chunked text.
- On LongBench v2, 9k selected tokens match the best full-window read.
- On InfiniteBench it leads the best window read by 5 points.
- A 27B model reads 4.5M-token contexts on one 80GB GPU.
From the original article
1 min read at huggingface.co
A language model reads long text in one quadratic forward pass, stops at the context window, and loses accuracy with length before reaching it. We ask whether the read can be factorized when deciding over a finite set: which document is relevant, which option is supported, which passage is the evidence.
By Mohamed Eltahir, Anas Obayd, Raed Rashid · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.