Paper proves no finite rulebook can certify every safe program
Original title: No finite rulebook can certify every safe program: an impossibility theorem for gatekeeping AI agents
Watching can catch every prohibition violation, but no safe program can be cleared for good.
The paper proves that two finite, machine-checkable rulebooks—one listing allowed programs and one listing forbidden ones—cannot both be complete for an AI assistant that writes small programs. Finite evidence never settles “never” or “eventually.” Prohibitions can be caught by watching but not cleared permanently; confidentiality and responsiveness fail on both sides. Any sound n-bit rulebook omits a compliant program of about n bits. Simulations on 2.2 million random programs and self-referential adversaries find the obstruction is adversarial.
- A time limit makes rules checkable only by running nearly the whole limit.
- A gatekeeper can keep a checkable list of what it must allow and nothing it must refuse.
- The error a longer test never removes is the liveness part of confidentiality.
- Rules considered are no forbidden action, no dependence on a secret, and eventual answers.
From the original article
1 min read at arxiv.orgAn AI assistant writes small programs, and a gatekeeper must decide, from the code alone, whether each one may run. Suppose the rules are fixed: never perform a forbidden action, never let the output depend on a secret, always answer eventually.
By Tomas Perez-Acle · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.