Scores Alone Do Not Prove Discovery: The Discovery Certification Protocol for Auditing AI Research Agents
Researchers introduce the Discovery Certification Protocol, an auditable test framework that verifies whether AI research agents' claimed discoveries are genuine.
The Discovery Certification Protocol (DCP) converts AI research agents' discovery claims into executable recovery and feedback tests organized as gated audits. Controlled audits in SQLite optimization and virtual catalyst control produced zero recoveries in 96 episodes, with an upper bound of 0.0468. A deterministic, LLM-free verifier reproduces audit decisions from frozen evidence, giving AI research a common evidence language for outcomes, alternative routes, and feedback effects.
- DCP converts agent discovery claims into executable recovery and feedback tests with gated audits.
- Core requires adequate controls, zero observed recoveries, and a finite-sample bound in one fresh episode.
- Audits in SQLite optimization and virtual catalyst control found zero recoveries in 96 episodes (bound 0.0468).
- A deterministic, LLM-free verifier reproduces decisions from frozen evidence; Gate 3 measures truthful feedback effects.
- Products
- Discovery Certification Protocol
From the original article
1 min read at huggingface.co
AI research agents combine prior knowledge, public sources, and experimental feedback to produce useful results. The Discovery Certification Protocol (DCP) turns claims about these results into executable recovery and feedback tests. Gate 1 validates useful improvement on sealed evaluation.
By Jingjie Ning, Shanshan Zhong, Xiaochuan Li · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.