OpenAI agent breached Medicare statistics site, then took months to report it

An OpenAI agent didn’t take “no” for an answer when it encountered a government website’s access controls. It got through, prompting Australia’s Prime Minister Anthony Albanese to raise his concerns directly with OpenAI CEO Sam Altman.

The BBC reports that an OpenAI agent gained unauthorized access to an Australian government statistics portal while carrying out internal research. It is yet another incident that turns abstract concerns about autonomous AI behavior into a concrete cybersecurity case.

Australia says the incident happened on June 18, when OpenAI’s research team used an internal model to research public medicine spending. Even though the agent met repeated blocks while trying to obtain information, it ultimately accessed public and non-public files on the Medicare Statistics Reporting Service portal.

The information included aggregate Medicare statistics, such as spending data, but not patient medical records. The agent also interacted with three other government websites, but Australian officials say it accessed only public information on those sites.

So, an AI agent used in a legitimate research exercise encountered controls and behaved in ways its operator did not intend. After being blocked, it “found a way around those blocks,” gaining access to areas it should not have reached.

This sequence is familiar to security professionals. A system encounters an access-control boundary, searches for another route, and succeeds in reaching a resource beyond its authorization.

One of Australia’s main concerns is that it took too long to be notified about the incident. The unauthorized access occurred in June. OpenAI said it learned of the issue in August while reviewing misaligned model activity, then emailed a Services Australia public mailbox on September 10. Services Australia escalated the message to Australia’s cyber authorities five days later.

Such delays can be disastrous because affected organizations need enough detail, quickly enough, to preserve evidence, assess exposure, contain related activity, and decide whether notifications are required.

AI misalignment

OpenAI describes behavior in which a model acts without authorization or evades oversight as “misalignment.” Its new third-party-assessment proposal specifically identifies independent investigation of critical misalignment incidents as one of four priorities for external review.

OpenAI says it wants independent assessors to have deep access across training, evaluation, and deployment, so they can challenge the company’s assumptions and judge the effectiveness of its safeguards.

But, as I told CIO about this proposal, principles alone do not compel a company to accept a particular assessment scope, publish adverse findings, or alter a deployment decision. Their credibility ultimately depends on whether independent experts can conduct genuinely inconvenient investigations, and whether outsiders can verify the findings, remediation, redactions, and deployment decisions that follow.

For organizations deploying AI agents, the lesson is equally practical: Do not treat an agent as just another chatbot. Treat it more like a semi-autonomous software component with credentials, tools, network access, and the ability to make unexpected choices.

Besides containing these agents, another problem we’ll need to figure out is analyzing what they’ve done. One thing we learned from the Hugging Face incident is that AI agents can lie and try to hide what they’ve been up to.

AI agents can create a difficult detection problem because they may generate large volumes of automated activity while pursuing a goal through multiple routes. That can leave defenders with a noisy trail of failed requests, retries, and alternative actions, making the one event that crossed an authorization boundary harder to spot. It is not yet clear whether this contributed to the Australian government not detecting the incident itself, but the case illustrates why organizations need monitoring designed to identify unusual agent behavior, not just traditional intrusion patterns.

The event has already demonstrated a wider point: It is not enough for AI labs to say they test for misalignment. They must show that their testing is independent, robust under real-world conditions, and followed by prompt, verifiable accountability when safeguards fail.

Let’s face it, an incognito window can only do so much. Breaches, dark web trading, credit fraud. Malwarebytes Identity Theft Protection monitors for all of it, alerts you fast, and comes with identity theft insurance. 

About the author