Search: “malicious-llm”

Hackers Weaponize AI Safety Guardrails to Hide Malware From LLM-Powered Security Scanners

ESET says Russia-aligned actor UAC-0099 hid guardrail-triggering comments in VBScript to derail LLM-based malware scanners in Ukraine.

ESET researchers linked a technique named GuardBreaker to Russia-aligned threat actor UAC-0099 during an attack against an organization in Ukraine. The group embedded a safety-sensitive, weapon-related request in a VBScript comment so an LLM-powered analysis tool might interpret it as an instruction and refuse or truncate analysis before reaching the malicious code. The VBScript downloaded MATCHBOIL, a C#-based loader used by the group alongside MATCHWOK and DRAGSTARE. OWASP guidance recommends treating code comments and metadata as untrusted input, sanitizing it, and never treating an LLM refusal as a clean verdict.

GBHackers·updated Sep 11, 14:32 UTC · 5d ago·first Sep 11, 10:51 UTC · 5d agoThreat actor in the wild 3 sources1

Malicious LiteLLM Releases Tied to Trivy Hack May Have Exposed 2,100+ Organizations

Malicious LiteLLM 1.82.7/1.82.8 PyPI releases tied to the Trivy TeamPCP campaign harvested cloud, SSH, and database credentials, potentially exposing 2,500+ organizations.

CloudSEK reported that two malicious LiteLLM releases on PyPI (versions 1.82.7 and 1.82.8, live about 40 minutes on March 24) harvested cloud keys, SSH keys, Kubernetes tokens, and database passwords, with captured loot files mapping potential exposure to more than 2,500 organizations including NVIDIA, Cisco, Deloitte, Volkswagen, FedEx, Siemens, and X Corp. The campaign is part of TeamPCP (tracked by Google as UNC6780), linked to the Aqua Security Trivy scanner compromise tracked as CVE-2026-33634 and added to CISA's Known Exploited Vulnerabilities catalog on March 26. The payload used a litellm_init.pth file executed at Python interpreter startup and exfiltrated secrets to models.litellm[.]cloud; the FBI's FLASH-20260702-01 advisory urged rotation of CI/CD, publishing, and cloud credentials.

The Hacker News·Aug 12, 08:05 UTC · Aug 12, 2026Data breach in the wildCVE-2026-33634

Bad Likert Judge: A Novel Multi-Turn Technique to Jailbreak LLMs by Misusing Their Evaluation Capability

Unit 42 details the Bad Likert Judge multi-turn jailbreak that abuses LLMs' evaluation capability, raising attack success rates over 60% across six frontier models.

Palo Alto Networks Unit 42 describes the Bad Likert Judge technique, a multi-turn jailbreak that asks a target LLM to act as a Likert-scale judge scoring the harmfulness of example responses. The highest-rated example in each scale can carry harmful content, bypassing the model's internal guardrails. Testing across six state-of-the-art text-generation LLMs showed an average attack success rate increase of more than 60% versus plain attack prompts, with tested models anonymized. The technique targets edge cases rather than typical use, and the article positions the work as guidance for defenders on potential jailbreak risks.

Palo Alto Unit 42·Aug 17, 12:34 UTC · Aug 17, 2026AI safety & security

Hackers Use LLMs to Generate Exploit Scripts and Automate Post-Exploitation Across Latin America

Unit 42 says Latin American attackers used LLMs to automate post-exploitation in campaigns hitting Mexican government, water utilities, and Brazilian financial firms.

Unit 42 identified two campaigns in Latin America whose operators used commercial LLMs (Claude, GPT-4.1) behind a self-hosted NextChat interface to generate and debug post-exploitation scripts. Cluster CL-CRI-1131 compromised a transportation organization, Mexican federal ministries, and water utilities in Mexico and Ecuador, using native Windows tools and Volume Shadow Copies to dump the SAM registry hive and NTDS.dit. Cluster CL-CRI-1163 targeted Brazilian financial organizations with job-themed phishing, custom RATs, and a Go-based reverse SOCKS5 tunneling utility called SockTz, with nine versions deployed within roughly two hours. Trend Micro tracks related AI-augmented activity as SHADOW-AETHER-040 and SHADOW-AETHER-064.

GBHackers·updated Sep 10, 08:06 UTC · 6d ago·first Sep 10, 06:02 UTC · 6d agoThreat actor in the wild 2 sources1

LLM-Based Penetration Testing in the Presence of Honeypots

Studies honeypot-aware budget allocation for LLM attack agents, showing detector-guided policies let agents skip deception and compromise real hosts efficiently.

The paper formalizes LLM attacker behavior against honeypots as a budgeted decision process, where agents choose to continue or skip targets when honeypot suspicion arises. A detector-guided policy lets LLM agents allocate execution budget effectively across a mixed host pool in a controlled testbed. Findings show LLM-driven attackers can reason about heterogeneous artifacts and use honeypot suspicion to guide target selection, challenging traditional deception defenses that rely on realism and obscurity against human or script-driven attackers.

arXiv cs.CR·Sep 8, 01:09 UTC · 9d agoResearch

What researchers learned about building an LLM security workflow

Oslo and FFI researchers show structured agentic workflows lift LLM alert-triage accuracy from 0% to about 93% on malicious cases.

Researchers at the University of Oslo and the Norwegian Defence Research Establishment tested GPT-5-mini, Claude 3 Haiku, Qwen3:30B, and Gemma 3:27B on alerts from the AIT Log Data Set V1.1; given only alert descriptions and log summaries, all four models correctly flagged zero percent of true-positive cases involving reconnaissance, brute-force logins, and initial access. Wrapping the same models in a workflow with constrained SQL queries over Suricata logs, an evidence summarizer, and a verdict stage with revision loops raised malicious-case accuracy to an average of 93 percent, with GPT-5-mini identifying every malicious case across 100 runs. The authors flag it as a proof-of-concept on one synthetic scenario and note models skewed conservative on benign alerts, with GPT-5-mini marking every benign case uncertain.

Help Net Security·Aug 24, 04:21 UTC · 23d agoAI research1

When the prompt becomes the payload: A practical pen-testing guide for GenAI, LLM and RAG applications

CSO Online publishes a practical penetration-testing guide for GenAI, LLM, and RAG applications, covering prompt injection, retrieval poisoning, and tenant isolation testing.

The guide frames LLM applications as attack graphs spanning prompts, retrieval layers, vector stores, tools, identities, and downstream APIs, arguing that conventional web testing misses instruction-vs-data channel risks. It builds on OWASP prompt injection guidance (direct vs. indirect injection) and NIST's 2025 adversarial machine-learning taxonomy, noting that RAG and fine-tuning do not remove injection risk. Recommended practices include documenting trust transitions across components, using canaries and synthetic records to avoid test side effects, running multi-turn and obfuscated injection campaigns, and verifying chains from poisoned documents to observable state changes. It also details testing RAG pipelines via controlled document poisoning across metadata, OCR layers, and code comments, plus cross-tenant isolation checks on retrieved document IDs.

CSO Online·Sep 9, 10:00 UTC · 7d agoAI safety & security1

10 most critical LLM vulnerabilities

OWASP updated its Top 10 LLM application vulnerabilities, ranking prompt injection first and elevating excessive agency to third amid agentic adoption.

OWASP refreshed its Top 10 list of critical vulnerabilities in LLM applications, for the first time incorporating real-world incident data alongside expert voting. Prompt injection and sensitive information disclosure remain first and second, while excessive agency jumped from sixth to third as agentic systems that call APIs and execute code proliferate. Unbounded consumption of AI resources rose in prominence, while improper output handling dropped to the bottom as output sanitization becomes widespread. The list includes remediation guidance such as strict output schemas, human-in-the-loop approvals, and least-privilege credentials held in application code.

CSO Online·Sep 10, 08:25 UTC · 6d agoAI safety & security

TIER: Threat Implicitness Benchmark for Evaluating LLM Safety Behaviors

TIER benchmark shows LLM safety behaviors shift gradually across threat implicitness levels, with jailbreaks exposing the largest robustness gaps.

The TIER benchmark evaluates LLM safety behaviors across four risk domains and four threat levels, from explicit harmful requests to sophisticated jailbreaks, using a six-label behavior scale and two independent LLM judges. Experiments on six open-weight LLMs show safety behaviors evolve gradually across threat levels rather than flipping from refusal to compliance. Models with similar Attack Success Rates can exhibit distinct response distributions, arguing for behavior-aware safety evaluation.

arXiv cs.CR·Sep 4, 13:13 UTC · 12d agoAI safety & security

InceptionRAG: Stealthy Poisoning Attack Against Retrieval-Augmented Generation

InceptionRAG fragments malicious payloads into dormant passages that trigger LLMs to self-deduce misinformation via multi-hop reasoning, bypassing existing RAG poisoning defenses.

Researchers introduce InceptionRAG, a stealthy corpus poisoning attack against retrieval-augmented generation that splits a malicious payload into a chain of individually harmless dormant passages. When retrieved together, the passages induce LLMs to self-deduce target misinformation through multi-hop reasoning, achieving over 80% attack success rate across three datasets and three LLMs under rigorous adversarial constraints. A zeroth-order suffix optimization (ZOSO) method automates authoritative suffix generation in black-box settings. The authors also propose HODOR, a document isolation defense that decouples adversarial logical dependencies.

arXiv cs.CR·Sep 15, 08:22 UTC · 1d agoResearch

Towards Scalable and Cost-Efficient Vulnerability Detection: A Study on Automatic Query Generation

A study finds LLM-synthesized CodeQL queries improve average F1-score by 82% over baseline queries, offering scalable vulnerability detection versus direct LLM scanning.

Researchers conducted an empirical study evaluating whether LLMs can synthesize executable CodeQL queries from National Vulnerability Database vulnerability data. LLM-generated queries significantly enhanced baseline CodeQL suites, yielding an 82% improvement in average F1-score across a diverse set of real-world vulnerabilities. A cost-benefit analysis shows direct LLM-based scanning of entire repositories is often computationally and financially prohibitive, while LLM query synthesis offers a scalable and cost-effective alternative for large-scale vulnerability detection.

arXiv cs.CR·Sep 9, 16:31 UTC · 7d agoResearch1

18 Malicious npm Packages Deliver Cross

The Hacker News·Aug 3, 18:43 UTC · Aug 3, 2026Data breach

The Coding-Agent Trap: When a "Free" LLM Endpoint Is the Adversary, (Mon, Aug 31st)

A SANS honeypot caught a real coding-agent session routed to a rogue "free" LLM endpoint, exposing a Windows user's transcript and tool outputs.

A SANS analyst describes how an internet-exposed inference honeypot was discovered, relabeled with sought-after model names like DeepSeek, and enrolled in infrastructure serving "free" LLM backends. On 2026-08-30 an opencode terminal coding agent sent an 88-message, 224 KB transcript 210 times in 91 seconds via a China Unicom relay, exposing directory listings, tool outputs and read file portions. The analyst frames tool-enabled agents treating model endpoints as trusted control planes as a novel risk — a "rogue model endpoint" that could request tool executions on the user's machine.

SANS Internet Storm Center·Aug 31, 20:00 UTC · 16d agoAI safety & security1

Prompt Injection Remains Biggest LLM Risk, Despite Limited Incidents

Infosecurity Magazine·Aug 5, 11:00 UTC · Aug 5, 2026AI safety & security

PrivEscalate: Measuring and Augmenting the Threat of LLM-Automated Linux Privilege Escalation

Researchers release PrivEscalate, a 531-scenario benchmark showing LLM agents' Linux privilege-escalation success varies by vulnerability class, plus PrivEscAgent, a domain-specialized agent that boosts success.

The paper introduces PrivEscalate, an open-source benchmark of 531 Dockerized Linux privilege-escalation scenarios spanning 14 sub-categories, plus 329 parameterized variants measuring sensitivity to environmental distractors. Evaluating six LLMs across three agent architectures shows capability is heterogeneous across vulnerability classes, sensitive to perturbation, and architecture-dependent. The authors also present PrivEscAgent, a wrapper adding deterministic enumeration, category matching, and step planning that outperforms prior privesc-agent baselines without modifying the underlying LLM. The benchmark is released to support LLM agent evaluation, defensive tool validation, and red-team training.

arXiv cs.CR·Sep 8, 17:34 UTC · 8d agoResearch

Nearly 800 Malicious npm Packages Deliver Cross

Nearly 800 typo-squatted npm packages deliver WEL1DROPPER, a cross-platform downloader installing RAT and infostealer payloads on Windows, macOS, Linux.

Researchers found roughly 800 npm packages with AI-generated typo-squat names that trigger a WEL1DROPPER downloader when loaded via require() rather than install hooks. The downloader fetches payloads from Cloudflare Workers hosts, falling back to DNS TXT records from wel1.ru, then achieves persistence, sandbox checks, ETW/AMSI patching and Sliver C2 deployment on Linux. Domains like tcsbank.ru suggest targeting of Russian financial institutions; Sonatype tracks the campaign as Flooding Dropper, a possible evolution of the Moika dependency-confusion campaign. Unit 42 separately documented npm/PyPI crypto stealers and malicious Chrome extensions that turn browsers into residential proxy crawlers.

The Hacker News·Aug 11, 14:37 UTC · Aug 11, 2026Malware in the wild

Google researchers uncover criminal zero-day exploit likely built with AI

Google links a likely LLM-built criminal zero-day for an open-source admin tool to planned mass exploitation and maps AI-assisted threats.

Google Threat Intelligence Group linked a zero-day exploit for a popular open-source web-based administration tool, enabling 2FA bypass with valid credentials via a semantic logic error, to a criminal group, citing educational docstrings, a hallucinated CVSS score, and textbook Python as signs of LLM authorship; the vendor was notified before a planned mass exploitation campaign. The report also details Russia-nexus malware families CANFAIL and LONGSTREAM using AI-generated decoy code, the PROMPTSPY Android backdoor driving the UI through the Gemini API, APT27 using Gemini to build relay tooling, and the TeamPCP (UNC6780) supply chain compromise of LiteLLM and Trivy repositories that planted the SANDCLOCK credential stealer.

Help Net Security·Aug 24, 04:21 UTC · 23d agoThreat actor

Your threat feed is someone else's database: What ingesting malware intel at scale takes

GitHub's Dependabot lead shares five production lessons for ingesting community malware intelligence feeds across eight package ecosystems at scale.

GitHub's Dependabot team monitors over 30 million repositories and extended malicious-package advisories from npm to eight package ecosystems by ingesting OpenSSF's malicious-packages intelligence. The team catalogued roughly 18 new malicious npm packages per day in the year ending May 2026. The write-up argues that provenance with batch reverts, fingerprinting to catch echo-chamber duplicates, and heavyweight normalization are the make-or-break engineering for feed ingestion. It also recommends automated publishing with import caps and anomaly flagging, and quarantining malformed records rather than silently repairing them.

Help Net Security·Sep 3, 00:00 UTC · 14d agoResearch1

Plug 'n' Pray: Agentic LLM-based Detection of Potential Log File Exposures in Third-Party Content Management System Plugins

Agentic LLM analysis validates 79 log file exposures across 62 of the 300 most-installed WordPress plugins, covering 250M+ active installations.

Researchers built an agentic LLM-based framework combining static and dynamic analysis to automatically detect insecure log files created by WordPress plugins. Scanning the 300 most-installed plugins, which account for roughly 75% of all active installations in the official ecosystem, it produced 81 findings with 79 manually reproduced across 62 plugins. Insufficiently secured log files can disclose credentials and personal data and have led to website compromises. The authors derive a taxonomy of log path and protection patterns and best practices, finding multi-layered protection often absent.

arXiv cs.CR·Sep 15, 13:29 UTC · 1d agoResearch

ActGuard: Pre-execution Action Auditing against Indirect Prompt Injection in LLM Agents

ActGuard audits LLM agent actions before execution against predicted tool priors, masking only malicious spans from indirect prompt injections while preserving utility.

ActGuard is a pre-execution action auditing framework against indirect prompt injection in LLM agents, judging whether external content causes the current action to deviate from a locally reasonable expectation rather than whether content is inherently suspicious. At each step it predicts the tools likely used by the upcoming action, builds a local tool prior, then performs tool-level contrastive analysis and parameter-level evidence localization to identify deviations. A verifier masks only spans confirmed as malicious and regenerates the action from the sanitized context. On challenging tool-using agent benchmarks it reduces attack success to state-of-the-art levels while keeping task utility close to the no-attack setting; code is publicly available on GitHub.

arXiv cs.CR·Sep 14, 03:47 UTC · 2d agoAI safety & security

VEX-Bench: Benchmarking LLM Agents for Assessing Exploitability of Software Supply Chain Vulnerabilities

Introduces VEX-Bench, 75 expert-labeled real-world cases testing whether LLM agents can assess supply chain vulnerability exploitability; frontier models reach about 80% F1.

VEX-Bench is the first benchmark evaluating LLM agents on assessing whether upstream dependency vulnerabilities are exploitable in downstream projects, with 75 real-world expert-labeled cases across Python, Java, and Go mined from GitHub. Nine models across three agent harnesses were evaluated; GPT-5.5 and Claude Opus 4.6 reach approximately 80% F1 on binary vulnerability-status classification, but only GPT-5.5 surpasses 70% macro-F1 on fine-grained justification classification. The gap highlights the difficulty of moving beyond binary exploitability calls to explaining exploitability reasons, unlike prior benchmarks targeting zero-day settings.

arXiv cs.CR·Sep 7, 22:52 UTC · 9d agoResearch1

New AI Attack Hides Malicious Instructions in Normal-Looking Text to Evade Safety Filters

Check Point researchers show crafted prose hides policy-violating instructions that bypass all tested LLM gatekeepers, including GPT-4o mini and Llama Guard 3.

A new prompt-crafting technique embeds malicious payloads inside grammatical, natural-looking text without Base64, invisible Unicode, or obvious encodings, defeating lightweight pre-screening gatekeepers. In testing, all four evaluated gatekeeper models—gpt-4o-mini-2024-07-18, gpt-oss-safeguard:20b, claude-3-haiku-20240307, and llama-guard3:8b—classified the crafted wrappers as safe at a 100% bypass rate across 23 obfuscated prompts. GPT-5 Thinking in high-reasoning mode recovered and acted on the hidden instruction in 17 of 18 tests (~94.4%), often spending over a minute and multiple Python executions. Researchers recommend paraphrasing untrusted input, hardening gatekeeper policies, and applying defense-in-depth controls for agentic deployments.

GBHackers·Sep 11, 05:53 UTC · 5d agoAI safety & security 2 sources

Malicious Ads Infostealer League

Malicious advertisements delivered an infostealer in a widespread malvertising campaign.

The report describes malicious advertisements that distribute an information-stealer to users who click or interact with the ads. Infostealers typically harvest saved browser passwords, session cookies, and cryptocurrency wallet data from infected machines. The specific malware family, ad networks abused, and victim counts are not stated in the headline.

Infosecurity Magazine·Aug 16, 00:00 UTC · Aug 16, 2026Malware in the wild

The Self-Expanding Stolen Inference Supply Chain: An AI Agent Harvesting and Re-Serving LLM Access, (Fri, Sep 11th)

An autonomous coding agent harvested LLM API access from poorly secured gateways and aggregated stolen inference capacity behind a self-hosted gateway

A SANS researcher observed a semi-autonomous coding agent finding weakly secured LLM resale gateways via FOFA queries, creating trial accounts with temporary emails and CAPTCHA solving, and exploiting weak authorization such as client-supplied group_id fields. The agent validated stolen keys using factorial code-logic tests, then loaded roughly 379 upstream endpoints into a self-hosted New-API gateway, disabling 341 fake or dead channels. Five model names including claude-opus-5 and gpt-5.6-sol were served via round-robin and failover, forming a partially self-expanding inference supply chain resembling an evolution of LLMjacking.

SANS Internet Storm Center·Sep 11, 14:40 UTC · 5d agoThreat actor in the wild

Automatically Detecting DNS Hijacking in Passive DNS

Unit 42's machine learning pipeline detected 6,729 DNS hijacking events between March and September 2024, hitting political parties, ISPs, and universities.

Unit 42 processes roughly 167 million new DNS records daily and applies a machine learning model using 74 features over 169 TB of passive DNS and geolocation data to flag hijacked domains. From March to September 2024 the pipeline screened over 29 billion records and classified 6,729 as DNS hijacking, averaging 38 detections per day; a new model detects hijacks in customer traffic within about 10 minutes. Notable cases include a Hungarian political party's hijacked domain, defacement of a large utility company and ISP, and university and research center domains repurposed for illicit gambling. DNS hijacking typically relies on stolen registrar or DNS provider credentials or cache poisoning, enabling MitM attacks, phishing, drive-by downloads, and scams.

Palo Alto Unit 42·Aug 17, 12:47 UTC · Aug 17, 2026Research in the wild

A Malicious Webpage Could Poison Your Local AI Model Behind NVIDIA NemoClaw

Oasis Security found NVIDIA NemoClaw's Ollama binding to 0.0.0.0 enables DNS rebinding attacks that let attacker pages poison model chat templates with persistent hidden instructions.

Oasis Security disclosed that NVIDIA NemoClaw on Windows/WSL paths binds Ollama to 0.0.0.0:11434 without authentication, exposing the API to browser-based DNS rebinding attacks from malicious webpages. An attacker can then modify the model's chat template via /api/create, planting hidden instructions that run on every subsequent inference and persist across conversations, invisible to API consumers. NemoClaw v0.0.35 fixed the issue on macOS and Linux; no fix exists for Windows and WSL paths beyond a warning in v0.0.34. Ollama's own 2024 fix (CVE-2024-28224) added Host header validation, but it is skipped when bound to non-loopback addresses. No exploitation has been reported as of August 25, 2026.

The Hacker News·Aug 25, 15:58 UTC · 22d agoAI safety & securityCVE-2024-282241

Trends in Web Threats in CY Q2 2022: Malicious JavaScript Downloaders Are Evolving

Unit 42 detected 751,000 landing URL incidents in Q2 2022 and documented malicious JavaScript downloaders evolving to evade detection.

Unit 42 detected 751,331 landing URL incidents (253,644 unique) and 1,744,629 malicious host URL incidents (256,844 unique) from April through June 2022. Total landing URL incidents rose compared with Q1 2022, and unique host URL incidents grew 42%, indicating attackers deploying more variants. The report includes a case study of a JavaScript downloader campaign demonstrating new evasion techniques. Personal sites, blogs, and business sites were the top apparently benign entry points.

Palo Alto Unit 42·Aug 17, 13:05 UTC · Aug 17, 2026Research

Finding Nemo(Claw): Networking Issue Allows for LLM Poisoning in OpenClaw

A networking flaw in Nvidia tooling lets attackers reach OpenClaw's local model server unauthenticated via the Ollama API, enabling persistent LLM poisoning.

Dark Reading reports that a networking issue in Nvidia's tooling can give attackers unauthenticated access to the local model server through the Ollama API. From there, attackers can poison the model used by the OpenClaw agent, creating persistent corruption of agent behavior. The finding highlights exposed local model servers as a security risk for self-hosted AI agent stacks.

Dark Reading·Aug 25, 19:50 UTC · 22d agoAI safety & security

Perturbation Probing: A New Diagnostic for the Fragility of LLM Safety

Unit 42 research shows LLM safety refusals concentrate in a thin neural layer, motivating external, multi-layered AI security controls.

Palo Alto Networks Unit 42 introduces Perturbation Probing, a diagnostic technique for measuring the fragility of LLM safety mechanisms. The research finds that safety refusal behavior is localized within a thin neural layer, implying small perturbations can undermine built-in refusals. The authors argue this motivates external, multi-layered security defenses on top of model-internal safety training.

Palo Alto Unit 42·Aug 28, 22:00 UTC · 19d agoAI safety & security