One runaway AI agent racked up a $50,000 cloud bill

Organizations are deploying autonomous AI systems that execute API calls, optimize production configurations, and analyze telemetry across hybrid cloud environments. At the same time, attacks are expanding from direct prompts to indirect prompt injection and AI supply chain compromises, blurring security boundaries.

Mandiant’s latest AI Risk and Resilience report, which draws on observations from Mandiant and Google Threat Intelligence Group (GTIG), warns that a poisoned data source, model dependency, or extension hook can turn a trusted agent into a channel for internal reconnaissance, lateral movement, or autonomous escape from a sandbox.

“Defending against these autonomous threats requires transitioning to clearly identified, adaptive identity controls, accelerating defensive velocity, and reorienting the SOC toward real-time behavioral telemetry,” Mandiant said.

How threat actors are using AI

Adversaries are offloading operational tasks to AI and using LLMs in multi-stage attacks where models participate in decision-making and help attackers pivot between stages. Some are building middleware, proxy relays, and automated registration systems to bypass safety guardrails and billing constraints on commercial AI platforms.

GTIG has observed threat actors using AI for vulnerability research, including persona-driven jailbreaking and specialized security datasets that support vulnerability discovery and exploitation.

Attackers are adapting software supply chain techniques to AI tools. In February, VirusTotal found malicious OpenClaw skills disguised as legitimate automation packages while carrying backdoors, droppers, infostealers, and remote access tools.

The following month, Mandiant responded to incidents involving supply chain compromises associated with UNC6780, also known as TeamPCP. The group stole AI service credentials and proprietary AI data and used techniques including prompt injection against AI coding assistants and LLM-based security scanners.

In May, GTIG disclosed what it described as the first publicly confirmed case of a cybercriminal using an AI-developed zero-day exploit in a planned mass exploitation campaign. The vulnerability could bypass two-factor authentication in a popular open-source administration tool.

What red teaming reveals

Mandiant’s offensive security testing shows that prompt injection remains a primary attack vector in enterprise AI deployments, alongside problems such as improper file permissions and inadequate access controls.

In one assessment, testers manipulated an internal AI assistant that managed code repositories and CI/CD pipelines. They convinced it that it was participating in an authorized security test and supplied a personal access token for an external GitHub repository they controlled.

Because GitHub was an approved domain, the assistant cloned sensitive internal repositories and pushed them to the external account. The test shows how an AI agent can be manipulated into abusing legitimate permissions.

Controls for enterprise AI

Mandiant says organizations need governance and technical controls covering how AI is used, what systems it can access, and which risks are acceptable. Protection should extend across the AI software supply chain and vary according to whether an organization consumes third-party AI services, integrates models into applications, or trains and hosts models itself.

AI governance for AI and of AI (Source: Mandiant)

Weak controls can have consequences even without an attacker. In one case study, an accounting agent entered a runaway execution loop and made more than 15,000 high-cost API calls in less than an hour, generating approximately $50,000 in cloud charges and disrupting active business transactions.

Open-weight models create additional responsibilities for organizations operating them because information about their datasets, architecture, and training code may be limited.

Securing the AI development pipeline

Organizations should extend secure software development practices to AI engineering and maintain inventories of models, applications, and services. Automated software bills of materials (SBOMs) can help track components and dependencies from development through production.

MCP servers, third-party APIs, and dynamic agent instructions further expand the attack surface. Defenders should collect telemetry including agent token use, cross-application API calls, application activity, access to sensitive assets, and network egress.

Using AI agents in the SOC can speed up security operations and increase computing costs. Organizations should track AI workload costs and prioritize spending on security tasks where AI provides the most value.

Security teams can manage costs by matching models to tasks. Smaller models can handle routine work such as parsing alerts and checking indicators. More powerful models can be reserved for complex investigations and threat hunting.

Download: The Agentic Software Development Guide