Does an Agent's History Tell You When Compaction Will Hurt? A Modest, Bounded Effect on the TRACE Paired-Replay Corpus
On TRACE's 590 AppWorld boundaries, recent agent history only weakly predicts harmful context compaction.
The paper asks whether a long-horizon agent's recent behavior predicts when context compaction will harm later actions. TRACE's public corpus of 590 harness-triggered AppWorld compaction boundaries replays each boundary from a re-executed prefix under the original context and under the summary, scoring next-action burden as errors or repeated calls. Pre-boundary history predicts harm only weakly: the best extension-protocol trigger reaches held-out AUROC 0.66 against a same-boundary replicate of 0.72. The best frozen interpretable trigger avoids 21% of harmful boundaries while keeping 84% of compaction opportunities, but does not beat random expectation on burden mass, and a matched comparison with a token-budget rule cannot be evaluated on the release.
- Corpus covers 590 AppWorld compaction boundaries with paired replays.
- Best trigger reaches held-out AUROC 0.66 versus a 0.72 replicate baseline.
- Interpretable trigger avoids 21% of harmful boundaries and keeps 84% of opportunities.
- A matched comparison with a token-budget rule cannot be run on the release.
From the original article
1 min read at arxiv.orgMany long-horizon agents compact their context on a global rule, usually a token budget, blind to what the agent was doing. We ask whether the agent's recent behaviour predicts when a compaction will hurt.
By Egor Pakhomov · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.