AlphaBet uses reinforcement learning to train LLM event-forecasting agents
Original title: AlphaBet: Reinforcement Learning for LLM Event Forecasting Agents
On GLM-4.5-Air, Brier score fell from 0.731 to 0.626 on Forecast-Dojo and from 0.612 to 0.528 on FutureX.
AlphaBet is an agentic reinforcement learning framework that trains one LLM policy to search for evidence, reason, and report outcome probabilities on several dates before an event resolves. Each dated forecast is scored with a proper scoring rule against the eventual outcome, so one resolved event supervises decisions under different information states. Ablations credit the gains to multi-date training and to better evidence acquisition and probability reporting. The authors also report that GRPO standard-deviation normalization can distort multi-class proper-score targets while top-outcome accuracy stays largely unchanged.
- Forecasting regret is split into an evidence-acquisition gap and a probability-reporting gap.
- Training covers search, reasoning, and probability reports across multiple pre-resolution dates.
- GRPO standard-deviation normalization degraded proper-scoring-rule performance in their tests.
- Top-outcome accuracy stayed largely unchanged under that normalization.
From the original article
1 min read at huggingface.co
Forecasting an unresolved event requires finding relevant evidence and deciding how strongly it supports each outcome. Yet outcome-based training of Large Language Model (LLM) forecasters often begins with preassembled evidence at a single prediction date, leaving evidence acquisition outside the learning loop.
By Liqin Ye, Rongzhi Zhang, Xin Liu · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.