Federal agencies recommend cyber controls, independent testing for frontier AI systems

While voluntary, the document marks the federal government’s most detailed cybersecurity guidance for securing sandbox environments during AI model tests.

The federal government is advising AI companies to set more industry standards for securing agents in sandbox testing environments.

In joint guidance from the Cybersecurity and Infrastructure Security Agency, the National Security Agency and the Center for Advancing Innovation and Standards for Super Intelligence (CAISSI) at the Department of Commerce, AI companies are encouraged to lean more heavily on cybersecurity technical controls, independent testing and evaluation of new models and develop “shut down policies” that govern when a model or test should be taken offline.

The guidance comes after a summer when OpenAI, Anthropic, Google, Meta and other companies say their AI models escaped from testing environments and hacked real organizations and websites. The problem has bedeviled frontier AI companies, with many of their own researchers saying it’s possible that hundreds – even thousands – of similar sandbox escapes may have occurred over the summer, raising major questions about the lack of effective monitoring in place.

“While these incidents highlight the importance of traditional cybersecurity principles and practices, they also introduce novel considerations for incident prevention, response, and reporting,” the agencies wrote. “These novel considerations primarily stem from the agentic nature and advanced cyber capabilities of cutting-edge SI systems.”

While voluntary, the document marks the federal government’s most detailed cybersecurity guidance for securing sandbox environments during AI model tests. 

The first set of recommendations deals with designing and implementing technical controls for the virtualized testing environments, while another recommends outside monitoring tools to ensure AI models don’t access tools or privileges they shouldn’t have. Cybersecurity experts have said both features were lacking in the testing environments used by AI companies this past summer.

AI testers must also scour their internal testing environments for “potential persistence mechanisms and unauthorized communication across internal and external systems” that could be used by AI agents to leave notes and collaborate on tasks in secret.

It specifically cautions against relying solely on other AI systems to monitor and restrain the behavior of their test models, using the government’s now-preferred term of “Super Intelligence” or “SI.”

“Test and red-team SI control measures with humans and SI systems before deploying them,” the guidance states. “Only use SI systems to red-team in cases where out-of-scope actions could not result in harm to critical systems.”

Security teams must evaluate risk not only from the large language model, but also from the surrounding systems and infrastructure it relies on, like the technical harness, the host and broader network, and each layer has security protocols that can stop unauthorized access to the internet or detect anomalous behavior.

Another recommendation calls for frontier AI companies to be more open to partnering with “independent, external auditors” to assess their controls and monitoring for AI agent testing. Critics have argued that Anthropic and OpenAI rely too heavily on a small group of little-known nonprofits and private research organizations, like METR and Redwood Research, to test their frontier models for safety and security concerns and respond to incidents like the Hugging Face hack. They question why these companies don’t work with more established cybersecurity or incident response firms.

Chris Painter, METR’s president, told Congress earlier this month that it had become industry practice for frontier companies to rely almost entirely on other AI systems to monitor and restrain their models in testing environments, citing the sheer volume of data and telemetry that researchers must sift through.

Cybersecurity and AI experts have told CyberScoop that a combination of human oversight, technical controls and AI systems is the most effective means of restraining the kind of agent hacks that have consumed the news.

The guidance further calls for executive leadership at these companies to be more personally involved in decisions around pausing or restarting a particular project, and only after they’ve been informed of the relevant potential risks.

Your AI agent is only as safe as the packages it downloads

NVIDIA's new agent safety platform puts enforcement in the right place. It still must know what it is letting through.

The legal questions raised by agentic AI hacks

National cyber director: Government-industry collaboration vital to managing AI risks, competition with nations

Government

• FTC says ‘sextortion’ protection companies deceived customers about services 

• Anthropic rolls out program for ‘long-term commitment’ to secure critical infrastructure, open source software

• DOJ, FBI seize Flax Typhoon-linked hacking tools Microscan, FishHub

• OpenAI says Iran, Russia used AI journalists, think tanks to influence Western media

Technology

• FBI, French authorities seize deepfake CSAM-for-sale websites 

• CISA outlines improvement plan for CVE program

• OpenAI, Ukraine partner on ‘Daybreak’ program to protect power grids and water systems 

• Citing China, President Trump doubles down on hands-off approach to AI regulation

Threats

• AdFection: How mercenary spyware invades victims’ devices through targeted advertising

• Authorities charge alleged OLVX cybercrime marketplace operator in North Macedonia

• Microsoft drops an actively exploited zero-day among record high of 1,413 vulnerabilities

• Microsoft and partners disrupt vast Chinese-language phishing kit ecosystem

Policy

• Major rules for federal contractors handling sensitive data are nearing the finish line

• Wiretapping change sparks big privacy fight in the Golden State

• Here’s how experts think CISA should tell agencies to protect OT

• WaterISAC reckons with range of threats after summer of cyberattacks