Skip to content
NewA calmer, faster ZeroHour

Correlated reasoning splits self-consistency certificates into two nulls

Original title: Realized versus Population Reliability: Anytime-Valid Self-Consistency Certification under Correlated Reasoning

arXiv cs.AI / cs.LG / cs.CLby Terence D. Agbeyegbe ()1 min read
AI summary · x-ai/grok-4.7Summaries by Vani AI

On AIME, a single-unit rule crossed on 24% of prompts at nominal epsilon 0.05; a prompt-level test did not.

A paper argues that language-model self-consistency certificates assuming independent rollouts fail when reasoning streams are correlated. Correlation splits a realized null about the drawn belief from a population null about average model behavior, and the standard betting e-process remains anytime-valid only for the realized null. On Claude Haiku 4.5, GSM8K shows a dormant gap, while AIME shows the single-unit rule crossing on 24% of prompts at nominal epsilon 0.05. A prompt-level e-process restores population validity, optimally with one rollout per prompt under a pure rollout budget.

Unconfirmed: Reported rho, theta, and crossing rates are estimates from the authors' simulations and rollouts.

Key facts
  • Claude Haiku 4.5 rollouts are used in the experiments.
  • GSM8K estimates are rho 0.435 and theta +0.96.
  • AIME estimates are rho 0.480 and theta -0.15.
  • Within-unit effective sample size saturates at 1/rho.
  • The prompt-level procedure records a crossing rate of 0.00 on AIME.
Explain this AI news with Vani AIWhat was announced, what is new and why it matters, in a new chat

From the original article

1 min read at arxiv.org

Self-consistency certification treats repeated sampling from a language model as sequential evidence accumulation and stops when an anytime-valid e-process crosses a threshold, guaranteeing finite-sample Type I error control by Ville's inequality. Prior constructions assume sampled rollouts are conditionally independent.

arXiv cs.AI / cs.LG / cs.CL

By Terence D. Agbeyegbe · arxiv.org

Read the full article at arXiv cs.AI / cs.LG / cs.CL

A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.