The Copy Ceiling: An Input-Exposure Control for Ontology-Grounded Generation over Curated Corpora
A copy-ceiling control shows corpus-grounded model gains mostly reflect exposed answers, not extra reasoning.
The paper proposes exposure accounting for ontology-grounded generation over curated corpora, classifying each gold item by whether the shown context exposes it and whether the answer recovers it. Its reference, the copy ceiling, is the recall a verbatim copy of the context achieves. Across ten models, unaided recall averages 0.26 and grounded recall 0.92, yet gain over copy is uniformly negative, from -0.067 to -0.022. Of 11,360 gold-item observations, only three unexposed items receive lexical credit, and a judged audit finds all three fail relational adjudication.
- Copy ceiling equals recall from verbatim copying of shown context.
- Grounded recall averages 0.92 but stays below the copy baseline.
- Only three of 11,360 observations credit items the context did not expose.
- Rephrasing questions cuts exposure from 0.964 to 0.328.
From the original article
1 min read at arxiv.orgWhen a language model answers from a curated corpus via graph-based retrieval, a large grounding uplift does not establish reasoning over the retrieved structure: the context may already expose the gold answers. We propose exposure accounting, which classifies each gold item by whether the shown context exposes it and whether the answer recovers it.
By John J. O'Hare · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.