Study finds LLM reviews miss flaws in post-quantum FPGA design
Original title: From FIPS to FPGA with LLM Agents: Measuring What Tests and Reviews Miss in Post-Quantum Cryptographic Hardware
Module simulation killed 76% of 399 mutants; formal checks flagged planted key-custody flaws that scoped review found in 3 of 20 sessions.
A cs.CR paper describes a seven-stage verification-gated workflow used while porting an ML-KEM-768/ML-DSA-65 accelerator to a 300 MHz Zynq UltraScale+ key-custody card. Board tests were byte-exact for all six operations and passed 85 interface-applicable NIST ACVP cases of 285, plus 484 ML-DSA known answers. Mutant, replay, and formal audits showed passing tests and LLM reviews overstated assurance, especially for custody invariants.
Unconfirmed: Detection rates come from the authors' pre-registered single-model replays; the model name is not stated.
- Existing simulation gates killed 76% of 399 tool-selected netlist mutants across three modules.
- Survivor-guided tests raised fresh-draw decapsulation kill rate from 48% to 74%.
- Undirected review caught two verifier flaws in 3/6 and 1/6 sessions.
- Module-scoped review caught every known flaw in all 54 sessions.
- Scoped review detected planted custody flaws in 3/20 sessions; formal checks flagged all.
- Organizations
- NIST
From the original article
1 min read at arxiv.orgAs LLM agents increasingly generate RTL alongside the tests and reviews that validate it, self-generated evidence is a weak foundation for trustworthy hardware. We present our seven-stage verification-gated workflow, evolved during a post-quantum design port, in which a gate counts only if shown able to fail on negative controls and fault mutants, and reviewer agents audit work before commit.
By Jungmin Park · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.