Benchmark plants 100 errors in psychology papers to test AI review
The best system found 71 errors and the pooled systems found 93; seven missed errors were all omissions.
Stories per day · 30 days
2 RSS
10 stories, newest first · one card per story
Stories tagged #peer-review
PremiseCheck fact-checks peer reviews at about half frontier cost
An LLM rewrites comments as premises, then a decision model checks each premise against the paper.
Google Deepmind's AlphaProof Nexus solved dozens of open math problems and actually got peer reviewed
DeepMind's AlphaProof Nexus, built on Gemini 3.1 Pro, solved long-open math problems in a peer-reviewed Science study.
Sakana AI’s LLM Peer Review System Catches 73% of Core-Claim Errors
Sakana AI's three-agent Claude reviewer caught 73.43% of core-claim errors, versus 14.81% for the best prior system.
Science Utopia? Closed-Loop LLM Simulation of Academic Research Ecosystems
SciUtopia simulates 61 research worlds with 40,000+ LLM-agent researchers, finding rejection-driven resubmission amplifies reviewer burden and inequality emerges endogenously.
A Missing Piece for Trustworthy AI Reviewers: From Benchmarking Rhetorical Robustness to SciCore Review
Researchers benchmark AI reviewers' rhetorical robustness and propose SciCore, averaging full-text and science-core judgments.
AI conference ICLR is drowning in abstracts, with roughly 50,000 submissions before the deadline
ICLR 2027 received roughly 50,000 abstracts before its deadline, far above ICLR 2026's 19,500 valid submissions, straining peer review.
When AI Reviews Train AI Reviewers: Scientific-Judgment Collapse and Mitigation
Training LLM reviewers on model-generated reviews compresses rating distributions and diversity, a failure the authors name scientific-judgment collapse and mitigate with TrustReviewer.
ActReview: Rebuttal-Guided Training Data and Rubric Rewards for Actionable Peer Review Generation
ActReview post-trains Qwen3-8B-Base on 40K rebuttal-derived instances with rubric rewards to generate actionable, grounded peer-review feedback, plus a 1,000-instance benchmark.
ActReview: Rebuttal-Guided Training Data and Rubric Rewards for Actionable Peer Review Generation
ActReview post-trains Qwen3-8B-Base on OpenReview rebuttals to generate actionable peer-review feedback with grounded revision suggestions, benchmarked on 1,000 curated instances.