THEATRON benchmark tests persona persistence in long LLM decisions
Original title: THEATRON: A Benchmark for Evaluating Persona Persistence in Long-Horizon LLM Decision-Making
Four LLMs show persona-driven shifts that often fail to become stable, context-sensitive policies over multi-month financial trajectories.
Researchers introduce THEATRON, a controlled benchmark that uses financial replay to test whether fixed behavioral-finance personas persist during long sequential decisions. Models see shared daily market evidence and account-state feedback, then act over multi-month trajectories. Zero-shot, one-shot, and chain-of-thought persona injection are compared across four unnamed LLMs. Conditioning shifts behavior beyond a no-persona baseline, but separation can come from stereotyped action collapse or noisy switching rather than coherent policies. Code and data are public.
Unconfirmed: The paper does not name the four LLMs or report independent replication.
- Evaluations cover Zero-Shot, One-Shot, and Chain-of-Thought persona injection.
- Metrics include cumulative return and action trajectories.
- Persona separation generally exceeds variation from equivalent wording.
- Stable separation can reflect stereotyped action collapse.
- Code and benchmark data are on GitHub.
From the original article
1 min read at arxiv.orgExisting evaluations often test persona expression rather than whether persona information persists in extended decision-making. We introduce THEATRON, a controlled benchmark that uses financial replay to evaluate persona persistence in long-horizon sequential decision-making.
By Yuyan Wang · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.