Decision-Sufficient State Representations: Measuring and Reducing Write-Time Regret
Training LLM agent writers on reader loss cuts write-time regret, but gains fade as facts must be remembered longer.
The paper studies LLM agents that rewrite a short state each step and act from that state alone, splitting reader loss into unavoidable budget loss and write-time regret from the writer's choices. In TextWorld cooking games, a 128-token state holding the needed facts wins nearly every game, while prompted language-model writers win at most 17%. Decision-sufficient state representations train the writer from the reader's forward-rolled loss, and that score predicts outcomes (correlation 0.48) unlike fixed-context hindsight scoring. On a pre-registered split, training adds 7.0 success points when facts are needed soon, significant only at the shortest delay.
- Prompted language-model writers win at most 17 percent of TextWorld games.
- A 128-token state that keeps needed facts wins nearly every game.
- Forward-rolled DSSR scores predict outcomes with correlation 0.48.
- Training adds 7.0 success points, significant only at the shortest delay.
From the original article
1 min read at arxiv.orgLong tasks produce more history than an LLM agent can hold in its context, and more than it uses reliably even when the history fits. A growing line of work therefore has agents carry a short written state instead: at every step a writer rewrites the state, and a reader acts from the state alone. Steps stay cheap, but anything the writer drops is lost before later decisions reveal that they need it.
By Bingyu Shen · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.