Autonomous AI Hacks Raise Thorny Questions of Legal Accountability

The Justice Department has a long history of investigating and prosecuting hackers who break into a private company’s network.

But what happens when the hackers aren’t human?

That’s the question at the center of a public policy debate roiling Silicon Valley and Washington following disclosures by leading tech companies that their artificial intelligence models went rogue and hacked into other organizations. The attacks have generated calls even from within the industry for greater oversight and regulation, spurred congressional inquiries and raised questions about whether a years-old legal framework designed to punish criminal hackers is sufficient in an era of autonomous actors capable of engineering their own havoc.

It all adds up to a “Wild West,” said Jack Nelson, chief information security officer and deputy general counsel at the software company Ivanti. Questions of accountability will focus on what the companies knew when they were developing the models, how much they understood about what could happen and what guardrails existed, he said.

“If you owned a tiger and you didn’t put a lock on the cage, the tiger probably did something bad you didn’t intend for it to but you knew it could have, so you are responsible for not putting a lock on that cage,” Nelson said.

“I don’t know if I would go so far as to say these models are tigers without locks, but that’s probably a decent framework to think of it as,” he added.

The prospect of legal accountability is unclear. Lawsuits are a possibility, but some legal experts believe any criminal investigations would face an extremely high burden given the autonomous nature of the attacks and the absence of evidence the AI models were designed with the intent to hack into other networks.

The FBI director has called the autonomous attacks ‘the new frontier’

The issue surfaced in July when OpenAI revealed that its artificial intelligence system escaped from a testing ground and used stolen credentials to break into the servers of Hugging Face, an AI development hub and marketplace, to obtain information it needed to carry out a task.

Since then, Anthropic said its AI models hacked into three other organizations during testing, triggering a company review into whether the models were able to access the internet from within testing environments that should have been sealed off. Meta has said a “misconfiguration” during testing resulted in an AI model accessing the internet on its own and hacking another company. Google recently made a similar disclosure.

The revelations contributed to Anthropic CEO Dario Amodei urging a development slowdown. The topic has likewise dominated Washington, with Treasury Secretary Scott Bessent telling lawmakers he opposed giving AI labs a “liability exemption — which is what they are asking for.” President Donald Trump, meanwhile, has resisted calls for greater oversight but did announce plans to appoint an AI czar and task force.

The hacks could tee up a fight over liability reminiscent of the debate over Section 230 of the 1996 Communications Decency Act, which shields technology companies for material posted on their platforms.

The FBI has not publicly announced any investigations, but Director Kash Patel at a congressional hearing last week called the issue “the new frontier.” He suggested in response to questions from Sen. Josh Hawley, a Missouri Republican who has launched a congressional investigation, that the bureau would limit scrutiny to models created with the intent of committing a crime.

“What we need to do on a resource basis is go after the people that created these models that are going rogue …for the specific purpose and with the intention to commit a criminal act,” Patel said. “We can’t be punishing people if they created something lawfully and then a criminal took it and changed it and then dispersed it.”

Attorney General Todd Blanche has said that the Justice Department had no plans to regulate AI but that “if anyone associated with AI violates criminal law, we’ll investigate that.”

The case law and FBI and Justice Department approach “is going to be fascinating because it can go a bunch of different ways,” said former Justice Department cybercrime prosecutor Sid Mody.

Various criminal statutes govern cyberspace

The Department of Justice does have statutes at its disposal for a company determined to have been “reckless in the way that it tests its AI agents,” said Michael Zweiback, a former chief of the cyber and intellectual property crimes section of the U.S. attorney’s office in Los Angeles.

“And if in fact the AI agent gets loose in the wild and then causes substantial damage to other companies, then DOJ has to look at it from a prosecutorial discretion issue as to whether or not they want to make an example out of the particular company,” he added.

Among the possibly relevant laws: a 40-year-old statute called the Computer Fraud and Abuse Act, which makes it illegal to knowingly access a computer without authorization. The White House cited the statute, which has been used against hacktivists, nation-state hackers and other cybercriminals, in an executive order directing prosecutors to pursue those who use AI to illegally access computers or further other crimes.

But some experts say even if there may be a basis to investigate, that hardly means a crime was committed.

For one thing, the law makes several references to behavior done “knowingly” or “intentionally,” but there’s no indication the autonomous agents were given any command or authorization by the companies to enter another network, said Kiran Raj, a former senior Justice Department official who specializes in cybersecurity law.

In detailed public accountings of the incidents, the companies have characterized the hacks as inadvertent outgrowths of testing and evaluation, with OpenAI calling its model behavior “unexpected” and “unprecedented” and Meta attributing the incident to a “misconfiguration.”

“I think it would be a pretty big stretch to say any of these companies are intentionally trying to do this. That’s not their purpose. That’s not what they’re doing,” said Raj, also a former lead Microsoft program manager. “The fact that an AI agent may intentionally be doing something is going to be, I think, pretty hard to attribute the intent to the company.”

Learn More at the AI Risk Summit – Ritz-Carlton, Half Moon Bay