Astral requires attested snapshots for LLM web-claim verdicts
Original title: Who Verifies the Verifiers? Accountable Semantic Consensus for AI-Mediated Web Information Verification
With two of seven assessors adversarial, attack success fell to 21.5-53.1% versus 54.7-77.8% for majority voting.
The paper argues that protocols deciding only from assessor outputs cannot reliably reject invalid web claims when errors are correlated. Astral pins assessors to one committed snapshot of the evidence and accepts a verdict only when 2f+1 assessors attest above a confidence threshold and no signed challenge stands. On five public benchmarks, it lowers pooled attack success but leaves many claims disputed. Without attack, 16.0-61.7% of claims receive a binary verdict.
Unconfirmed: Attack-success ranges are the authors' benchmark results with a fixed adversarial-assessor setup.
- Up to f faulty assessors are tolerated via a 2f+1 attestation rule.
- Remaining claims are disputed for audit rather than forced to a verdict.
- Authors prove output-only agreement cannot reject invalid claims with positive probability.
- Code is posted at an anonymous repository.
From the original article
1 min read at arxiv.orgLarge language models (LLMs) increasingly decide whether Web evidence supports online claims, often by voting over several assessors whose errors are presumed independent and benign. On the open Web, retrieved pages can steer the assessors that read them, and assessors from one model family fail together.
By Yanna Jiang · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.