Hackers build AI frameworks for widescale credential theft

Threat actors are increasingly switching from AI-powered coding assistants to multi-agent frameworks that automate every stage of an attack.

Drawing on telemetry from Mandiant's incident response engagements, threat actor tracking, and live platform defenses, the Google Threat Intelligence Group (GTIG) observed AI agents coordinating multiple attack tasks, troubleshooting failures, and adapting their actions with minimal human intervention.

“Over the past quarter, threat actors have moved beyond simple prompt-based LLM interactions to integrate AI capabilities into multiple stages of an attack lifecycle,” GTIG notes.

“While traditional script-based automation has long been a staple of threat actor operations, groups are increasingly upgrading these workflows, creating highly autonomous systems capable of reasoning through complex tasks and making dynamic decisions without the need for human oversight.”

In one such incident, a financially motivated attacker compromised an organization’s cloud infrastructure and deployed an autonomous multi-agent framework.

In less than six hours, the threat actor planned, built, and deployed a mass credential-harvesting campaign using an AI coding chatbot, a prompt, and markdown agent instructions, Google says.

The AI agents managed the vulnerability-scanning pipeline, harvested thousands of third-party credentials, troubleshot problems in real time, rotated IP addresses, and routed attack traffic through legitimate, compromised cloud environments to evade detection.

This approach dramatically reduced “human-in-the-loop” latency and the response windows for defenders.

In another incident, the researchers found an exposed command-and-control (C2) server hosting an automated reconnaissance and credential-management framework called “Recon.”

Its files included instructions for AI agents, knowledge files, and OpenClaw artifacts related to the framework that managed in real-time more than 23,800 harvested secrets, such as API keys.

GTIG's report notes other examples where China-linked cyberespionage actors experimented "with AI-powered development tools to build an AI-assisted, automated exploitation and post-exploitation pipeline."

The researchers say that other espionage groups, such as the Russia-based UNC5792, integrated AI models to automate monitoring bots searching Telegram channels for information of interest to the government.

However, GTIG underlined that fully autonomous hacking has not become widespread yet, and did not observe threat actors deploying fully autonomous pipelines for zero-day discovery and network exploitation against real-world targets.

The company also noted that Gemini, its AI model, caught many of these abuses early and responded in accordance with its safety protocols, allowing Google to take additional action, disrupt the campaigns, and ban the associated accounts.

AI tool abuse has also been observed in supply-chain attacks conducted by UNC6780 (TeamPCP), Gemini AI distillation operations involving 100 million prompts, and a growing market for stolen AI account credentials and API keys.

Also, state-backed groups continue to use AI for reconnaissance, phishing, malware development, exploitation, post-exploitation, data processing, and propaganda.