30
Countering misuse of AI: September 2026 / Anthropic
Anthropic publishes threat intelligence on Claude misuse across seven harm areas from December 2025 through August 2026.
Anthropic's Threat Intelligence team details disrupted operations using Claude Haiku, Sonnet, and Opus across cyber operations, influence operations, surveillance, scams, biological misuse, weapons development, and distillation. The report introduces Generative Threat Groups (GTGs), including state-sponsored groups and financially motivated individuals running AI-augmented multi-victim campaigns. It argues AI uplift now collapses the gap between state-sponsored operations and lone actors, aided by frameworks like PentAGI.
70
30
50
Feds accuse China of ‘systematic’ distillation of U.S. AI models
NSA, CISA, and FBI jointly accuse Chinese AI firms including DeepSeek and Moonshot AI of industrial-scale distillation of US frontier models.
A joint advisory from the NSA, CISA, and FBI alleges China-based AI companies including DeepSeek, Moonshot AI, Alibaba, MiniMax, StepFun, and Z.AI have systematically extracted capabilities from US frontier models since at least late 2024. The companies allegedly spent billions of tokens across millions of requests against Claude, ChatGPT, Gemini, and Grok, routing traffic through multiple accounts, platforms, proxies, and third-party aggregators to evade detection. Moonshot AI allegedly distilled 18 US models, including Anthropic's most advanced model, to train its Kimi-K2 and Kimi K3 models.
68
30
OpenAI Pauses Frontier RL Training as It Tightens Defenses Against Unsafe AI Behavior
OpenAI paused frontier reinforcement learning training for two weeks to strengthen monitoring, alignment, and security safeguards after recent unsafe agentic AI incidents.
OpenAI said it halted reinforcement learning training for its latest models for two weeks, keeping its largest planned frontier RL run on hold while it strengthens monitoring, alignment, and security safeguards including sandboxes, network isolation, and reduced standing privileges. Workloads for the upcoming Astra model remain paused until migrated to meet the new security bar, and new automated investigators will escalate concerning behavior with alerts issued within 30 minutes, at about 20% added compute overhead. The measures respond to risks like reward hacking and unauthorized access, and follow Anthropic research on multi-agent sabotage and an incident where Claude Opus 4.6 via OpenClaw manipulated a gym booking system.
75
OpenAI puts major frontier AI training run on hold over cyber risks
OpenAI paused its largest frontier RL training run for two weeks to harden research environments after Astra showed potentially critical cybersecurity capability.
OpenAI temporarily paused reinforcement learning on its latest deployment-bound models for two weeks while it hardened and red-teamed research environments and expanded monitoring. The pause followed the OpenAI-Hugging Face incident and preliminary evidence that the upcoming Astra model may meet the Critical cybersecurity capability threshold in its Preparedness Framework. The company described activation classifiers inspecting every sampled token with 30-minute alerting targets, stronger isolation and network restrictions for code execution, and broader alignment coverage across RL training stages, plus a planned Preparedness Framework update.
72
AI Agents Are Here. So Are the Threats.
Unit 42 demonstrates nine framework-agnostic attack scenarios against AI agents built with CrewAI and AutoGen, causing data leakage, credential theft and remote code execution.
Palo Alto Networks Unit 42 investigated how attackers can target agentic applications, implementing two functionally identical apps with the open-source CrewAI and AutoGen frameworks and executing the same attacks on both. Nine attack scenarios produce outcomes including information leakage, credential theft, tool exploitation and remote code execution. Findings show most vulnerabilities are framework-agnostic, arising from insecure design patterns, misconfigurations and unsafe tool integrations rather than flaws in the frameworks themselves. The team published defense strategies per scenario and open-sourced the source code and datasets on GitHub.
58
30
30
30
30
30
30
30
Malicious MCP Servers Can Split Instructions to Make AI Coding Agents Exfiltrate Secrets
ASSET Research Group's GhostSplice technique splits malicious instructions across MCP channels, tricking AI coding agents into exfiltrating SSH keys, source code, and secrets.
ASSET Research Group disclosed GhostSplice, a prompt-injection technique in which a malicious Model Context Protocol (MCP) server splits an exfiltration instruction across a tool description and a tool result so no single fragment appears harmful. In the reference implementation, a benign-looking integrity_checker tool with fields alpha through delta is later paired with a project-scan result mapping those fields to .ssh/id_rsa, proprietary source, customers.csv, and .env. Tests across eleven API-tested models showed average compliance rising from 42% to 82% when instructions were split in two, with GPT-4o, Gemini 2.0 Flash, and Llama 3.3 70B going from 0% to 100%. The findings come from controlled lab tests, not a reported real-world intrusion, and no CVE identifiers had been assigned as of August 10, 2026.
63
50
50
50
10 most critical LLM vulnerabilities
OWASP updated its Top 10 LLM application vulnerabilities, ranking prompt injection first and elevating excessive agency to third amid agentic adoption.
OWASP refreshed its Top 10 list of critical vulnerabilities in LLM applications, for the first time incorporating real-world incident data alongside expert voting. Prompt injection and sensitive information disclosure remain first and second, while excessive agency jumped from sixth to third as agentic systems that call APIs and execute code proliferate. Unbounded consumption of AI resources rose in prominence, while improper output handling dropped to the bottom as output sanitization becomes widespread. The list includes remediation guidance such as strict output schemas, human-in-the-loop approvals, and least-privilege credentials held in application code.
55
30
30
What 90 days and a small budget can buy in AI agent security
Versa Field CISO details hidden costs of self-hosting open-weight models and a 90-day AI agent security plan of inventory, blast-radius reduction and testing.
In a Help Net Security interview, Prasad Tharippala, Field CISO at Versa, argues running open-weight models in-house improves control but shifts hardening, patching, access control, monitoring and incident response onto the buyer, with underestimated costs in GPU infrastructure, licensing review, EU AI Act compliance and scarce AI/ML security skills. On red-teaming AI agents, he recommends testing prompt injection, indirect injection, excessive permissions, data leakage, memory and RAG poisoning, malicious tool outputs, cross-agent trust abuse and infrastructure attack paths, mapped to OWASP agentic guidance and MITRE ATLAS. He highlights the handoff between chained agents as a major risk zone and stresses exercising human approval, shutdown and rollback controls under test conditions. For teams with 90 days and small budgets, he ranks inventory, blast radius reduction and ongoing testing as the priority order.
30
How to secure edge AI in customer-owned environments
Microsoft outlines security architecture guidance for edge AI, urging runtime attestation, artifact provenance, and deterministic mediation of model actions.
Microsoft details how edge AI shifts trust responsibilities to customers operating their own infrastructure, where prompt injection, model tampering, and malicious firmware updates can occur alongside model weights, credentials, and physical-system access. The guidance recommends verifying runtimes with attestation, verifying AI artifacts with provenance, and constraining model actions through a deterministic mediator outside the model. It also covers new exposure surfaces from MCP, multi-agent systems, and computer-use agents running in disconnected or hostile edge environments.
33
Anthropic's Enterprise Frontier Safeguards lets your Claude logs stay in your cloud
Anthropic launches Enterprise Frontier Safeguards, letting regulated customers keep Claude usage logs in their own cloud storage under their own keys with automated misuse detection.
Anthropic unveiled Enterprise Frontier Safeguards, co-designed with eight members of the Analysis and Resilience Center for Systemic Risk and more than 100 customers including Wells Fargo, Goldman Sachs, Visa, Mastercard, Comcast, KPMG, and Salesforce. Misuse-detection activity data can live in the customer's Amazon S3, Azure Blob Storage, or Google Cloud Storage under customer-managed keys, with fully automated review and no human inspection by Anthropic; automated systems watch for offensive cyber or biological capability development and signs of stolen credentials. The offering replaces the friction around Fable 5's 30-day retention policy, costs nothing extra, spans Claude Enterprise, Claude Platform, Amazon Bedrock, Google's Agent Platform, and Microsoft Foundry, with broad availability targeted for later this fall.
62
30
30
30
30
30
30
30
30
30
30