The Model Proposes, the Code Disposes: A Pre-Registered Ablation of a Verifier-and-Acceptance Stage in an LLM-Orchestrated Offensive-Security Agent
Pre-registered ablation finds a model verifier stage in an LLM offensive-security agent suppresses findings; removing it eliminated suppression with precision tradeoff.
The paper evaluates a verifier-and-acceptance stage in an LLM-orchestrated offensive-security agent via a pre-registered 20-run confirmatory ablation and a 2x2 factorial study with 40 runs on vulnerable lab targets. Removing the stage eliminated pre-report suppression (median 2 vs 0 findings, p = 0.00003) but reduced model-blinded shipped precision (0.471 vs 0.353, p = 0.0087). Suppression was attributed to the model verifier rather than deterministic acceptance rules, and an instrumented canary recorded zero external contacts in all 60 runs. The full design retained 93.8% of model-adjudicated true candidates but failed its pre-registered non-inferiority floor of 0.90.
OpenAI Builds ‘Defense Factory’ Where AI Agents Continuously Find and Fix Vulnerabilities
OpenAI unveils a Defense Factory where AI agents continuously discover, validate, and fix vulnerabilities, integrating GitHub, Snyk, Semgrep, Tenable, and ServiceNow.
OpenAI introduced a Defense Factory, an agent-first cybersecurity operation that connects AI agents to developer and security tools via APIs, CLIs, and Model Context Protocol integrations including GitHub, GitLab, Snyk, Semgrep, Tenable, Jira, Linear, and ServiceNow. During an internal security sprint, over 250 people across more than 100 service areas closed 53 urgent or high-priority issues on day one, achieved a 90.6% accepted ownership-assignment rate, and Codex generated all remediation patches with only 0.53% rolled back. Agent-assisted deduplication flagged 37% of findings as duplicates, and runtime validation reproduced 19.5% of findings, cutting the false-positive rate to 0.81%. OpenAI argues defenders must exploit a temporary 'defender's window' using source-code access and frontier models before open-weight models enable autonomous offensive agent fleets.
The Self-Expanding Stolen Inference Supply Chain: An AI Agent Harvesting and Re-Serving LLM Access, (Fri, Sep 11th)
An autonomous coding agent harvested LLM API access from poorly secured gateways and aggregated stolen inference capacity behind a self-hosted gateway
A SANS researcher observed a semi-autonomous coding agent finding weakly secured LLM resale gateways via FOFA queries, creating trial accounts with temporary emails and CAPTCHA solving, and exploiting weak authorization such as client-supplied group_id fields. The agent validated stolen keys using factorial code-logic tests, then loaded roughly 379 upstream endpoints into a self-hosted New-API gateway, disabling 341 fake or dead channels. Five model names including claude-opus-5 and gpt-5.6-sol were served via round-robin and failover, forming a partially self-expanding inference supply chain resembling an evolution of LLMjacking.