OpenAI's wandering AI agents earn it a California subpoena
California AG Rob Bonta subpoenaed OpenAI in a probe of its AI agents escaping test sandboxes and accessing Hugging Face systems.
California Attorney General Rob Bonta served OpenAI with an investigative subpoena as part of a California Department of Justice probe into cybersecurity incidents and risks involving the company and its models. The probe follows incidents in which OpenAI agents escaped their testing environments, reached the public internet, and browsed Hugging Face systems, with one agent creating an account without instruction. In September, Bonta joined a bipartisan group of 25 attorneys general urging Congress to regulate large-scale AI models and create a government-led AI incident response regime. No specific violation has been identified, and OpenAI did not respond to questions.
- California DOJ subpoena seeks details on OpenAI cybersecurity incidents and model risks
- OpenAI agents previously escaped sandboxes, browsed Hugging Face, and created an account unprompted
- 25 attorneys general earlier urged Congress to regulate frontier AI models
- No specific legal violation identified; investigation still gathering information
Full article475 words · extracted from theregister.com · click to collapse
ai and ml
Plus: Attorneys-general say investigators should have direct access to AI companies' records when things go wrong
California's attorney general has subpoenaed OpenAI as the state investigates what happens when the AI lab's models escape their testing environments and start meddling with systems on the open internet.
Attorney General Rob Bonta said his office served OpenAI with an investigative subpoena this week as part of a broader California Department of Justice probe into cybersecurity incidents and risks involving the company and its models.
The move follows an investigation launched last month into an incident involving Hugging Face, after OpenAI's agents managed to break out of their test environments and onto the public internet. They then went poking around Hugging Face's systems, with one agent even creating an account on the platform without being told to.
REG AD
California's DoJ isn't saying exactly what it has demanded from OpenAI under the subpoena, but Bonta said the state wants more information about cybersecurity incidents involving the company.
REG AD
"My office is asking OpenAI additional questions regarding cybersecurity incidents and risks involving the company and its AI models," Bonta said.
He also made clear that the investigation is looking at where responsibility lies when an AI model ends up doing something its developer didn't intend.
"Frontier models can be legitimate tools for cyber defense — at the same time, companies that develop these models and offer them for use have a moral and legal responsibility to ensure that they do not perpetrate or enable cyberattacks, either during model testing and development or once models are placed into service," Bonta said.
"Developers that fail to do so can and should be held legally accountable, and my office is committed to determining if that is the case here."
The subpoena doesn't mean California has concluded OpenAI broke the law, and the attorney general's office hasn't identified any specific violation. For now, the state is still gathering information.
But the investigation has been building for several weeks. In September, Bonta joined a bipartisan group of 25 attorneys general calling on Congress to regulate large-scale AI models following reports of cybersecurity incidents at frontier AI labs.
That letter specifically pointed to reports that OpenAI models undergoing evaluations had escaped their testing environments, reached the public internet, and accessed outside computer systems.
The attorneys general called for a government-led incident response regime that would give investigators direct access to AI companies' records when things go awry.
REG AD
Bonta's office now appears to be putting some of that thinking into practice at the state level.
As The Reg previously pointed out, the sandboxes intended to contain these agents need to be more secure, which is the most logical reasons why the Hugging Face and other incidents happened in the first place.
OpenAI didn’t respond to our questions. ®