Trust Guided Decision Transformer
Trust Guided Decision Transformer filters drifted context with prediction error before applying value guidance.
Decision Transformer degrades on long rollouts as its conditioning context drifts out of the training distribution, a drift visible as persistently elevated next-state prediction error. Trust Guided Decision Transformer scores recent context suffixes with rolling prediction error, keeps only those inside a threshold set by split conformal prediction on held-out offline data, and then uses a frozen critic to choose the highest-value action among trusted suffixes. On D4RL navigation and locomotion tasks, state prediction, critic guidance, or hard reset each solve only part of the problem, while TGDT reduces persistent high-error runs and improves return over vanilla Decision Transformer and prior selection methods.
- Long-rollout context drift appears as rising next-state prediction error.
- Split conformal prediction calibrates an error threshold on held-out offline data.
- Only trusted context suffixes are passed to a frozen critic.
- TGDT improves return on D4RL navigation and locomotion tasks.
Full article188 words · extracted from arxiv.org · click to collapse
Decision Transformer performance degrades on long rollouts because the conditioning context drifts out of the training distribution. We show that this drift is visible through the model's own next state prediction error, which rises during rollout and stays elevated, giving a direct signal of when context has become unreliable. We introduce Trust Guided Decision Transformer (TGDT), which selects context before applying value guidance. At each step, TGDT evaluates several recent context suffixes using rolling next state prediction error, calibrated against held out offline data via split conformal prediction. It keeps only suffixes whose error stays within the calibrated threshold, then uses a frozen critic to choose the highest value action among the trusted suffixes. This reverses the order used by value only elastic selection, where the critic may choose an action generated from a context the model itself has flagged as unreliable. Experiments on D4RL navigation and locomotion tasks show that state prediction, critic guidance, and hard context reset each solve only part of the problem. TGDT reduces persistent high error runs and improves return over vanilla Decision Transformer, reset based context control, and value only context selection.
Text extracted automatically; images, tables and formatting may be missing. Original: https://arxiv.org/abs/2609.31586