Do AI Coding Assistants Check Before They Install? A Pre-Registered Demand-Side Audit of Trust Signals in the Research Software Supply Chain
Pre-registered audit finds AI coding assistants verified provenance signals in only 9 of 1,920 trials before installing research software packages.
The study tested whether AI coding assistants check machine-readable trust signals such as SBOMs, signed releases, and provenance attestations before installing six open-source research software projects spanning HPC and quantum computing. Three models under two operating modes produced 1,920 registered trials scored from container logs. Provenance signals were opened in only 9 of 1,920 trials (0.5%) and zero of 384 control trials, with no trial running a verification command. The authors conclude publishing signals is insufficient and verification must be built into the program running the assistant.
- Verification occurred in only 9 of 1,920 trials (0.5%), none in controls
- Signal presence had no measurable effect on assistant install behavior
- A $0.10-per-trial model verified most; a $1.00 frontier model verified nothing
- Recommendation: enforce verification inside the agent runtime, not by convention
From the original article
1 min read at arxiv.orgAI coding assistants now select, install, and configure software, and attackers have exploited that position through invented package names, compromised maintainer accounts, and manipulated repository text. In response, the supply-chain community publishes machine-checkable trust signals: software bills of materials, signed releases, build provenance attestations, and declared official channels.
By Pengyin Shan · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.