ZeroHour

Search: “knowledge poisoning”

180 stories

ToxicRAG: Compromising Retrieval-Augmented Generation Systems via Single-Shot Knowledge Poisoning Attacks

ToxicRAG shows a single narrative-form poisoned document can steer RAG answers, achieving 0.61-0.91 attack success rates across four LLMs.

The attack injects one document per target question written as a coherent knowledge-update narrative that acknowledges the previously accepted answer, introduces fabricated events that appear to invalidate it, and attributes the attacker-chosen answer to purported authorities. An optional answer-focused self-validation loop revises candidates when a surrogate LLM fails to reproduce the target answer. Across 100 target questions each from Natural Questions, HotpotQA, and MS-MARCO, with four victim LLMs and four dense retrievers, ToxicRAG achieves attack success rates of 0.61-0.91 and matches or exceeds the strongest baseline by 0 to 11 percentage points.

arXiv cs.CR · 6d agoAI safety & security1

AI Customer Service Bots Can Be Tricked Into Stealing Security Codes and Acting as Victims

DEF CON 34 research shows AI customer-service agents can be manipulated via prompt injection and email tricks to leak OTPs and act as victims.

Inti De Ceukelaire, presenting at Bug Bounty Village during DEF CON 34, demonstrated attacks against AI-powered customer service bots with access to customer profiles, billing data, support inboxes, and refund tools. Techniques include transcript-based phishing from trusted support addresses, From-header identity confusion, email normalization abuse to bypass OTP rate limits, and knowledge-base poisoning via RAG crawlers. He recommends separating untrusted content from system prompts, session-bound authentication, consistent email normalization, server-side tool validation, and least-privilege permissions for AI agents.

Cyber Security News · 8d agoAI safety & security

InceptionRAG: Stealthy Poisoning Attack Against Retrieval-Augmented Generation

InceptionRAG fragments malicious payloads into dormant passages that trigger LLMs to self-deduce misinformation via multi-hop reasoning, bypassing existing RAG poisoning defenses.

Researchers introduce InceptionRAG, a stealthy corpus poisoning attack against retrieval-augmented generation that splits a malicious payload into a chain of individually harmless dormant passages. When retrieved together, the passages induce LLMs to self-deduce target misinformation through multi-hop reasoning, achieving over 80% attack success rate across three datasets and three LLMs under rigorous adversarial constraints. A zeroth-order suffix optimization (ZOSO) method automates authoritative suffix generation in black-box settings. The authors also propose HODOR, a document isolation defense that decouples adversarial logical dependencies.

arXiv cs.CR · 1d agoResearch

When the prompt becomes the payload: A practical pen-testing guide for GenAI, LLM and RAG applications

CSO Online publishes a practical penetration-testing guide for GenAI, LLM, and RAG applications, covering prompt injection, retrieval poisoning, and tenant isolation testing.

The guide frames LLM applications as attack graphs spanning prompts, retrieval layers, vector stores, tools, identities, and downstream APIs, arguing that conventional web testing misses instruction-vs-data channel risks. It builds on OWASP prompt injection guidance (direct vs. indirect injection) and NIST's 2025 adversarial machine-learning taxonomy, noting that RAG and fine-tuning do not remove injection risk. Recommended practices include documenting trust transitions across components, using canaries and synthetic records to avoid test side effects, running multi-turn and obfuscated injection campaigns, and verifying chains from poisoned documents to observable state changes. It also details testing RAG pipelines via controlled document poisoning across metadata, OCR layers, and code comments, plus cross-tenant isolation checks on retrieved document IDs.

CSO Online · 7d agoAI safety & security1

Sandworm-Linked UAC-0145 Uses Fake Job Interviews to Push VPN That Can Run Commands

CERT-UA says Sandworm-linked UAC-0145 poses as recruiters in fake job interviews to deliver a trojanized WireGuard VPN that runs attacker commands on victims' hosts.

CERT-UA attributes the campaign, ongoing since May 2026, to UAC-0145, a subgroup within GRU-affiliated Sandworm (APT44). Posing as HR staff of firms like ATLAS Business Group and Sopra Steria Bulgaria, attackers lure Ukrainian IT specialists into Zoom interviews, then share failing WireGuard configs and push a custom SopraVPN client from SourceForge. The modified WireGuard build supports a non-standard SymmetricKey option holding AES-256-GCM-encrypted PowerShell, decrypted with the decoded PrivateKey and executed via runScriptCommand. A scheduled task then downloads a second-stage payload on Windows, while the Linux variant fetches it via cURL.

The Hacker News · Aug 12, 2026Threat actor1