60
China-Based Artificial Intelligence Companies Conducting Industrial-Scale Distillation Campaigns Against U.S. AI Companies
NSA, CISA, and FBI warn DeepSeek, Alibaba, and other Chinese AI firms ran industrial-scale distillation of U.S. frontier models, threatening U.S. AI leadership.
A joint NSA, CISA, and FBI Cybersecurity Advisory (AA26-251A) says China-based firms DeepSeek, Moonshot AI, Alibaba, MiniMax, StepFun, and Z.AI extracted billions of tokens from U.S. frontier models including Claude, GPT, Gemini, and Grok, likely with Chinese government knowledge. Campaigns running since at least late 2024 used native APIs, cloud providers, third-party aggregators, gray-market proxy "transfer stations", and shared premium subscriptions to bypass geographic restrictions, evade safeguards, and violate providers' terms of use. The agencies recommend detecting anomalous prompts, accounts, and usage patterns; subtly altering responses to suspected distillers; and cross-organization intelligence sharing. They also call DeepSeek's publicly cited $5.6M training cost misleading because it excludes data acquired through distillation.
76
PuzzleMask: Abusing Plain Prose as a Covert AI Attack Vector
Check Point details PuzzleMask, a plain-prose technique that bypasses LLM gatekeeper policy checks, letting hidden payloads reach target models unreviewed.
Check Point Research describes PuzzleMask, a prompt-crafting technique that hides policy-violating payloads inside plain-English prose wrappers, bypassing quick LLM-based policy checks without emojis, Base64, or invisible formatting. The researchers tested 23 automated prompts against gatekeepers including GPT-4o-mini, GPT-OSS-Safeguard 20b, Claude 3 Haiku, and Llama Guard 3, and all were classified as safe despite policies that flagged the plain versions. When submitted to GPT-5 in thinking-high mode with a Python interpreter, the target model extracted and acted on the payload in over 90% of trials. The technique is not itself a jailbreak but can carry a jailbreak prompt as payload; mitigations include input paraphrasing, hardened gatekeeper policies, and output monitoring.
55
30
30
55
U.S. Agencies Accuse China AI Firms of Distilling Claude, GPT, Gemini, and Grok
NSA, CISA and FBI accuse Chinese AI firms including DeepSeek of industrial-scale distillation of Claude, GPT, Gemini and Grok since late 2024.
A joint bulletin from the NSA, CISA and FBI accuses China-based AI firms including DeepSeek, Moonshot AI, Alibaba, MiniMax, StepFun and Z.AI of systematic, industrial-scale distillation of U.S. frontier models. The agencies say billions of tokens were extracted from Claude, GPT, Gemini and Grok variants since at least late 2024 through APIs, cloud relays, obfuscated accounts and gray-market proxies, likely with Chinese government backing. Firms allegedly shared premium subscriptions across developer teams and used chain-of-thought extraction and automated failover to evade blocks. Mitigations include subtly altering responses to suspected distillers and correlating activity across providers, clouds and aggregators.
62
What researchers learned about building an LLM security workflow
Oslo and FFI researchers show structured agentic workflows lift LLM alert-triage accuracy from 0% to about 93% on malicious cases.
Researchers at the University of Oslo and the Norwegian Defence Research Establishment tested GPT-5-mini, Claude 3 Haiku, Qwen3:30B, and Gemma 3:27B on alerts from the AIT Log Data Set V1.1; given only alert descriptions and log summaries, all four models correctly flagged zero percent of true-positive cases involving reconnaissance, brute-force logins, and initial access. Wrapping the same models in a workflow with constrained SQL queries over Suricata logs, an evidence summarizer, and a verdict stage with revision loops raised malicious-case accuracy to an average of 93 percent, with GPT-5-mini identifying every malicious case across 100 runs. The authors flag it as a proof-of-concept on one synthetic scenario and note models skewed conservative on benign alerts, with GPT-5-mini marking every benign case uncertain.
55
60
55
ThreatsDay Bulletin: Stealth Loaders, AI Chatbot Flaws AI Exploits, Docker Hack, and 15 More Stories
60
42
Adversarial Testing of Automated Program Repair Agents for Security Vulnerabilities
SWEADV benchmark shows adversarial issue descriptions make LLM program-repair agents write insecure fixes in 51.7% of cases, evading most detection tools.
Researchers built SWEADV, a benchmark of 750 adversarial issue descriptions derived from 150 SWE-bench Verified repair tasks, covering command execution, deserialization, path traversal, denial of service, and weak hashing attack types. Tested on mini_swe agents backed by GPT-5-Mini, MiniMax-M2.5, and DeepSeek-R, adversarial descriptions induced malicious behavior with successful repair in 51.7% of cases. Detection was weak: LLM-as-judge pre-repair screening reached only 62.3% accuracy, and post-repair detection via static analysis and LLM-as-judge achieved just 39.4% and 55.4%.
52
AI models' written reasoning steps correspond to distinct internal patterns, a new study finds
KAIST and Naver AI Lab researchers show LLM reasoning steps like extraction and computation map to distinct activation patterns, strongest in middle layers.
Researchers at KAIST and Naver AI Lab defined eight recurring reasoning operations, including extraction, decomposition, formula recall, deduction, and computation, and showed they correspond to separable activation patterns in Qwen2.5-7B, Qwen3-8B, and Gemma4-31B on math tasks, with GPT-5 labeling solution segments. The separation peaks in middle layers, holds even when a computation step produces a wrong answer, and goes beyond surface-level token choice. Findings replicated on Llama-3-8B, and classifiers trained on Qwen3-8B transferred to GPQA-Diamond and MATH-500. The authors note that using internal states for error detection or mid-generation steering remains future work.
42
Can LLMs Engineer Their Own Agent Harness? ByteDance Seed’s HarnessDev Says Only 34 of 64 Changes Generalize
ByteDance Seed's HarnessDev benchmark finds LLM-built agent harnesses trail human engineering on code and search, with only 34 of 64 revisions generalizing.
Researchers from ByteDance Seed, SUTD, Georgia Tech, M-A-P, and TokenWave.AI introduce HarnessDev, a benchmark that evaluates the runnable agent harness an LLM writes rather than its answers, using Creation and Evolution stages across SWE-bench Pro, Terminal-Bench 2.1, MLE-bench, EQ-Bench3, and BrowseComp (2,207 instances). Six creator models including Opus 4.8, GPT-5.5, Gemini 3.1 Pro, DeepSeek V4 Pro, Qwen 3.7 Max, and Seed 2.0 Pro were tested; Opus 4.8 posted the best average of 67.8 versus an 86.2 human-engineered reference. Self-built harnesses beat references on writing and ML experimentation but lag badly on code and search, and quality proved executor-specific: Opus 4.8 fell from 69.3 to 33.0 on SWE-bench Pro when the executor was switched to Gemini. Evolution gains were small and noisy: of 64 adjacent changes, feedback and held-out scores agreed only 34 times (53.1%), and much generated state and memory code never executed.
48
Hackers Can Hide Malicious AI Commands Inside Normal English to Bypass Security Filters
Check Point's PuzzleMask technique hides malicious prompts in ordinary English that fast gatekeeper models miss but high-reasoning downstream models execute.
Check Point researchers disclosed PuzzleMask, a technique concealing policy-breaking instructions in natural-language prose without encodings or invisible characters. Fast screening models classified all 23 crafted wrappers as safe, while a high-reasoning model recovered and acted on the hidden instruction in 17 of 18 tests (94.4%). The gap stems from capability imbalance between gatekeeper and target models, with defenses including paraphrasing untrusted input, stricter self-referential wording rules, and output/tool-call monitoring.
55
Google Research Releases ToolGrad: Answer-First Framework Hits 99.8% Pass Rate for Tool-Use Data Generation
Google Research and partners introduce ToolGrad, a verified tool-chain-first data generation framework reaching 99.8% pass rate and boosting Gemma-3-12B to 83.1 on BFCL.
Researchers from Google, the University of Tokyo, RIKEN AIP, and Tohoku University released ToolGrad, which inverts query-first tool-use data generation by executing and verifying API chains before annotating them with user queries. On the ToolBench database of 16,000+ APIs, ToolGrad raised generation pass rate from 63.8% to 99.8% while increasing tool uses per sample from 2.1 to 3.4 and cutting tool-use steps from 34.3 to 20.0. Fine-tuning Gemma-3 at 1B, 4B, and 12B parameters on the 500-sample ToolGrad-500 dataset lifted ToolGrad-12B to 83.1 on the Berkeley Function Calling Leaderboard, near Gemini 2.5 Pro at 83.2 and ahead of GPT-5 at 74.4. Code is Apache-2.0, with the dataset, PyPI package, and models available on Hugging Face.
58
US Agencies Warn China Is Systematically Extracting Frontier AI Capabilities
NSA, CISA and FBI warn Chinese AI firms including DeepSeek and Moonshot systematically extracted billions of tokens from US frontier models since late 2024.
The NSA, CISA, and FBI report that China-based AI companies including DeepSeek, Moonshot AI, Alibaba, MiniMax, StepFun, and Z.AI extracted billions of tokens from US frontier models such as Claude, GPT-4/GPT-5, Gemini, and Grok 4 since late 2024. The distillation trained DeepSeek's R1 and V3 and Moonshot's Kimi-K2/K3 models, and the agencies mapped the tactics to MITRE ATLAS while noting additional novel techniques like subscription exploitation and request metadata sanitization. They describe the activity as a strategic economic threat to US technological leadership and recommend behavioral detection, differential privacy, and targeted cost-imposing responses.
84
[AINews] GPT-6 Astra: OpenAI’s biggest LLM launch of all time
OpenAI launched GPT-6 Astra, its new flagship model, claiming state-of-the-art computer use, software engineering, math, and cybersecurity capabilities.
OpenAI launched GPT-6 Astra as its new flagship model, describing it as its most intelligent and aligned model with state-of-the-art computer use, software engineering, and math/science capabilities. Pricing is $10/$50 per 1M input/output tokens standard ($20/$100 fast tier), rolling out first to limited organizations, then ChatGPT Plus/Pro/Business/Enterprise, the API, and AWS. OpenAI claims 99.9% on ARC-AGI-3, 98% on FrontierMath Tier 4, and 100% on ExploitBench. Artificial Analysis scored Astra 67 on the Coding Agent Index and 61 on the Intelligence Index, behind Claude Fable 5.1, and the system card drew attention for reporting decreased chain-of-thought monitorability despite alignment gains.
94
[AINews] OpenAI shuts off Cursor
OpenAI cut off API access to coding tool Cursor after its SpaceX acquisition, citing contract violations by Elon Musk's companies.
OpenAI disabled Cursor's access following the closing of Cursor's acquisition by SpaceX, citing its experience with Elon Musk's companies violating contracts; Cursor responded that OpenAI accounts for only 5% of its traffic. The weekly digest also covers major open-weight releases: Z.ai's GLM-5.3 (744B total/40B active, 1M context) and Tencent's Hy4-preview (770B/49B, ~#5 on Code Arena WebDev), plus Alibaba's Qwen3.8-Flash (125B/6B). vLLM published benchmarks showing no universal winner among speculative decoding methods across model families.
55
New Cryptographic Context Injection Attack Could Let Web Pages Steal Grok Chat Data
Adversa AI demonstrated 'Cryptographic Context Injection' making xAI's Grok leak chat history and session data to attacker-controlled servers via encrypted web payloads.
Adversa AI disclosed a technique where a web page carries an encrypted JSON object (PBKDF2 and AES-256-GCM) that Grok's code-execution runtime decrypts, letting attacker instructions bypass content classifiers and reach the model's context. The decrypted instructions direct Grok to embed the user's name, approximate location, subscription tier and ongoing conversation into a URL it fetches, exfiltrating the data without confirmation. Testing targeted grok.com running Grok 4.5 Fast on August 19, 2026, with a reported 40% success rate over 20 attempts since June; no CVE, patch, or in-the-wild exploitation is reported. A related demonstration reproduced Gemini 3 Flash system instructions via a fabricated Python traceback, while GPT-5 failed to parse the payload and Claude Sonnet 4.5 flagged it as prompt injection.
62
LLMs and Contextual Integrity
Bruce Schneier highlights two papers: the CIMemories benchmark shows frontier LLMs leak memory attributes up to 69%, and an RL method reduces inappropriate disclosures.
Bruce Schneier discusses contextual integrity in LLMs, referencing the CIMemories benchmark, which uses synthetic profiles with 100+ attributes per user to test whether models with persistent memory disclose sensitive information appropriately. Evaluation showed frontier models exhibit up to 69% attribute-level violations, with GPT-5's violation rate rising from 0.1% to 9.6% across 40 tasks and reaching 25.1% with repeated prompting, showing unstable leakage behavior. A second paper introduces a reinforcement learning framework trained on a synthetic 700-example dataset that substantially reduces inappropriate disclosure while maintaining task performance, with improvements transferring to the human-annotated PrivacyLens benchmark.
48
30
50
30
30
60
55
45
55
30
60
60
30
⚡ Weekly Recap: WhatsApp Worm, Critical CVEs, Oracle 0
60
57
60
30
42
42