How Threat Actors Are Turning Trusted AI Platforms Into an Attack Surface

As AI platforms become part of daily workflows, attackers have found a new way in: the platforms themselves. The Huntress Security Operations Center (SOC) says the bigger day-to-day risk comes from threat actors abusing the AI features people already trust and rely on, rather than attacks on the AI companies or models themselves.

Over the past nine months, Huntress has tracked incidents in which attackers weaponized shareable AI content, public mini-apps, and sponsored search placement to target AI users and deliver malware.

Legitimate features, hijacked

Huntress has observed threat actors abuse a handful of real AI platform features, including:

• Claude Artifacts: content Claude generates and displays in a chat preview pane, which users can publish and share via a public link.

• claude.ai/share links: shareable URLs created when someone publishes a Claude conversation; these can surface in search engines when posted to crawlable spots like forums or social media.

• ChatGPT and Grok conversations: shared, indexable conversations hosted on chatgpt.com and grok.com that can rank for troubleshooting searches.

Each of these sits inside a trust boundary. Users recognize the platform, the branding, and the surrounding content, so malicious instructions or downloads look legitimate. These campaigns often only run for hours or days before a provider pulls the content down, but that's enough time to trick victims before getting caught.

FakeAgent: malvertising through a Claude Artifact

In July, Huntress saw a campaign called FakeAgent hit more than 29 organizations. It started with a malicious Claude Artifact hosted on the real claude.ai domain.

Since public Artifacts are meant for lightweight demos and get minimal vetting from Anthropic beyond a generic disclaimer, attackers built a convincing fake Claude Desktop download page.

Victims searching Bing for the Claude desktop app landed on the fake page and clicked what looked like a legitimate download link. Instead, they were redirected to an external domain that delivered the SectopRAT malware.

Huntress reported the Artifact and Anthropic removed it by July 22, but incidents tied to the same redirect domain continued into August.

A fake install guide hiding in claude.ai/share

In a separate incident, a victim searching Google for "Claude on Mac" clicked a sponsored result that led to a claude.ai/share link posing as an Apple Support install guide. Because the page lived on Anthropic's own domain, it carried none of the usual red flags: no lookalike URL, no certificate warning.

The fake guide instructed the victim to paste a curl command into Terminal, kicking off a six-stage chain that deployed the MacSync stealer. It harvested cookies, credentials, keychain secrets, Telegram sessions, and SSH and cloud keys.

AI poisoning via ChatGPT and Grok

A third pattern targets AI-generated troubleshooting advice itself. In December, a routine search for "clear disk space on macOS" surfaced high-ranking ChatGPT and Grok conversations that gave ClickFix-style instructions instead of real fixes.

Attackers had crafted the conversations, hit "share" to generate a public URL on the platform's trusted domain, and used SEO poisoning to push the link to the top of Google's results.

Because the links lived on real chatgpt.com and grok.com domains, victims trusted the advice and ran the suggested Terminal commands, which delivered the AMOS stealer. 

What defenders should do

None of these attacks broke through the AI platform security. They exploited the trust users place in familiar brands and real domains. 

Defenders should treat clipboard-driven execution and AI-assisted troubleshooting as security risks. Restrict script execution from the clipboard and enforce application allow-listing. Watch for new scheduled tasks and antivirus exclusion changes, and train users to spot ClickFix-style lures. Report suspicious AI-hosted content to the platform vendor quickly.

These campaigns tend to be short-lived, but fast reporting and layered controls can shrink the window attackers get to exploit them.

If you’re interested in this kind of tradecraft and exploring how attackers evolve their tactics, join our experts at Tradecraft Tuesday, where we break it all down every month. 

Sponsored and written by Huntress Labs.