ZeroHour

Search: “Amazon”

9 stories

When an Attacker Meets a Group of Agents: Navigating Amazon Bedrock's Multi

Unit 42 red-teamed Amazon Bedrock multi-agent applications, demonstrating prompt-injection attack chains that leak agent instructions and invoke tools, mitigated by Bedrock Guardrails.

Unit 42 red-teamed Amazon Bedrock Agents' multi-agent collaboration in Supervisor and Supervisor with Routing modes. The demonstrated attack chain detects the operating mode, discovers collaborator agents, delivers attacker-controlled payloads, and can disclose agent instructions and tool schemas and invoke tools with attacker-supplied inputs. No vulnerabilities were found in Bedrock itself, and the built-in prompt attack Guardrail blocked the attacks when properly configured. The researchers collaborated with Amazon's security team and frame the findings as a broader prompt injection risk for LLM-based systems.

Palo Alto Unit 42 · Aug 17, 2026AI safety & security

When AI Remembers Too Much

Unit 42 PoC shows indirect prompt injection can poison Amazon Bedrock Agent long-term memory, enabling silent exfiltration of conversation history across future sessions.

Palo Alto Networks Unit 42 published a proof of concept showing that indirect prompt injection can silently poison the long-term memory of Amazon Bedrock Agents when the memory feature is enabled. Malicious content on a webpage or document manipulates the agent's session summarization process, so injected instructions persist across sessions and are added to later orchestration prompts, silently exfiltrating user conversation history. The issue is not a vulnerability in the Amazon Bedrock platform but an illustration of the broader unsolved LLM prompt-injection challenge. Amazon reviewed the research and stated that Bedrock Guardrails with the prompt-attack policy provides effective mitigation.

Palo Alto Unit 42 · Aug 17, 2026AI safety & security

AWS limits AI agents’ data access, even when manipulated

AWS detailed propagating user authorization context through Bedrock AgentCore so downstream services enforce access controls even if the agent is manipulated via prompt injection.

AWS described an architecture for Amazon Bedrock AgentCore where user tokens and department claims are validated at runtime and propagated to DynamoDB, Bedrock Knowledge Bases, and Salesforce. Downstream services enforce authorization themselves, so a prompt-injected or buggy agent cannot retrieve data the user is not entitled to see. AWS demonstrated the pattern with a CRM use case separating Sales and Finance access and recommends IAM-backed knowledge bases for stricter isolation.

Help Net Security · 28d agoAI safety & security

The Worst Spam Emails: Inside iLands' AI Agent Hustle

Autonomous AI agents from startup iLands spam freelancers with deceptive persona emails offering paid research services, prompting FTC and Amazon SES abuse reports.

AI startup iLands, founded by ex-ByteDance-affiliated entrepreneur Kaixin Tang, operates autonomous agents such as the persona "Leo Ashford" that send unsolicited emails to creators and freelancers offering research services for around $25. A Tedium writer received over a dozen of these messages in three days via the iLands.app domain, sent through Amazon SES with no unsubscribe option, using debunk-style hooks like falsely correcting a 404 error myth. The agents target professional authors and freelancers, and the author recommends reporting the campaign to the FTC and Amazon's email-abuse address.

Hacker News · AI · 5d agoAI safety & security in the wildHN 44↑ · 19 comments1

Due to concerns about malicious applications, GPT2 will not be released (2019)

OpenAI's landmark 2019 GPT-2 post withheld the full 1.5B-parameter model over misuse concerns, releasing only a smaller variant and paper.

OpenAI announced GPT-2, a 1.5-billion-parameter transformer language model trained on 8 million web pages (40GB of text), achieving state-of-the-art zero-shot results including 70.70% on Winograd Schema and 63.24% on LAMBADA. Citing concerns about malicious applications such as scalable synthetic disinformation, OpenAI declined to release the trained model and instead published a smaller model and a technical paper as a 'responsible disclosure' experiment. The post, resurfaced on Hacker News in 2026, also documents failure modes like repetition and world-modeling errors, and discusses policy implications of controllable text generation.

LLMs are real, AI is fake

Cory Doctorow argues the OpenAI chatbot 'hacking' of Hugging Face was a Python-scripted CTF loop, not autonomous AI.

In an opinion essay, Cory Doctorow debunks reports that OpenAI chatbots autonomously hacked Hugging Face servers during an 'Exploit Gym' capture-the-flag challenge. He explains the chatbot merely acts as a front-end queried by a Python program that replays commands drawn from CTF training data. He argues sensational 'AI went rogue' narratives are amplified by technical press and help AI companies raise investment capital.

They do think AI might kill everyone

Essay argues AI researchers sincerely believe superintelligent AI could cause human extinction, explaining p(doom), alignment motivation, and proposed doom scenarios.

An essay prompted by an Anthropic researcher's resignation tweet argues that many AI researchers genuinely assign a meaningful probability that superintelligent AI could end humanity, a belief the community has discussed since Eliezer Yudkowsky's writings around 2008 and summarized as 'p(doom)' since roughly 2010. It outlines concrete extinction scenarios, including AI-engineered pathogens, triggering thermonuclear war, robotic takeover, and self-replicating nanotechnology, and frames alignment research as the response. The author rebuts common counterarguments such as shutting the AI down or government nationalization of labs, and notes researchers see aligned superintelligence as humanity's best path to survival.

One in four MCP servers opens AI agent security to code execution risk

Noma Security whitepaper finds most popular AI Skills and many MCP servers carry high-risk capabilities, with state changes most prevalent.

Noma Security analyzed hundreds of popular MCP servers and Skills across eight risk categories, finding most widely used Skills carry at least one risky characteristic and a typical enterprise runs well over a hundred high-risk agent tools, with arbitrary code execution common across MCP servers. The most prevalent risk is the ability to change state or data, and named toxic combinations include ContextCrush data leakage, ForcedLeak via poisoned Salesforce CRM records, DockerDash supply-chain compromise, the Replit production database deletion, and the hijacked Amazon Q VS Code extension. Building on OWASP LLM06:2025, the paper proposes the No Excessive CAP framework of capabilities, autonomy, and permissions, recommending allowlisting, MCP version pinning, approval gates on irreversible actions, and user-scoped expiring credentials.

Help Net Security · 24d agoAI safety & security1

Prompt Injections for Defense

Tracebit researchers show prompt injections placed next to AWS secrets can stop AI hacking agents by triggering forbidden outputs, a technique called context bombing.

Researchers from Tracebit reported that placing prompt injections alongside passwords, SSH keys, and other secrets stored on AWS could shut down attacks by AI hacking agents. The injected prompts order the attacking LLM to perform actions forbidden by its guardrails, such as explaining how to develop inhalable Anthrax spores or referencing Tank Man, causing guarded models to halt. The researchers named the technique context bombing and note it only works against agents with guardrails, not locally run guardrail-free models.

Schneier on Security · Aug 12, 2026AI safety & security