When AI Reviews Train AI Reviewers: Scientific-Judgment Collapse and Mitigation
Training LLM reviewers on model-generated reviews compresses rating distributions and diversity, a failure the authors name scientific-judgment collapse and mitigate with TrustReviewer.
The authors fine-tune Llama 3.1 8B as a reviewer on official ICLR reviews from 2018-2023, then train successor models on ICLR 2024 data with varied mixtures of official and synthetic reviews. Synthetic reviews compress rating distributions and reduce same-paper and corpus-level semantic diversity, a pattern termed scientific-judgment collapse. They introduce TrustReviewer, an open-source reviewer system combining curated single-stage training with test-time paired activation steering to preserve judgment diversity.
- Recursive AI peer review training compresses ratings and reduces semantic diversity
- Experiments use Llama 3.1 8B fine-tuned on ICLR 2018-2024 reviews
- TrustReviewer mitigates collapse via curated training and activation steering
From the original article
1 min read at huggingface.co
Large language models (LLMs) increasingly participate in scientific evaluation, both as automated reviewers and as assistants to human reviewers. As model-generated reviews enter public data and future training corpora, AI peer review can become recursive: later reviewers learn from judgments produced by earlier models. We study one step of this feedback loop in a controlled setting.
By Sy-Tuyen Ho, Minghui Liu, Furong Huang · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.