Search: “prompting”

Notes on gotchas while migrating 35kb preprompts from Opus to self-hosted Ollama

Opinion piece urges migrating 35KB preprompts from Anthropic/OpenAI to self-hosted Ollama, citing session privacy risks and safety filters blocking security research.

The author documents gotchas migrating 35KB preprompts from Claude Opus to self-hosted Ollama, motivated by fears that frontier providers train on user sessions, citing the OpenAI Navier-Stokes controversy. The piece argues inference providers cannot audit their own retention or training pipelines and that only self-hosted hardware offers verifiable privacy. It also criticizes frontier safety filters for refusing vulnerability research tasks and calls for models that support exploitability testing in CI/CD pipelines.

Hacker News · AI·Sep 14, 13:59 UTC · 2d agoAI industryHN 41↑ · 13 comments1

Not All Prompts Are Equal: Exploration-Guided Prompt Scaffolding for Multimodal Reinforcement Post-Training

Exploration-guided prompt scaffolding rewrites training prompts by Exploration Potential Score, boosting multimodal RL post-training accuracy up to 11.5%.

The paper proposes dynamically adapting the training prompt distribution during online RL post-training of multimodal LLMs using the Exploration Potential Score (EPS), a lightweight rollout-based proxy for prompt utility computed from on-policy statistics with no additional overhead. Rather than discarding low-utility prompts, a teacher model generates scaffolded rewrites that preserve task intent while making training more informative. Integrated with GRPO on Geo3K and MMK12, the method achieves up to 9.7% relative in-domain improvement plus 11.5% on MathVision and 11.1% on MMMU-Pro.

Hugging Face daily papers·Sep 13, 20:00 UTC · 3d agoAI research

Hackers Target Claude, Cursor and Codex AI Agents to Steal Tokens and Prompt Histories

Gen Digital found infostealers like Amatera and Remus stealing AI coding agent tokens, prompt histories, and MCP configs from infected Windows and macOS machines.

Gen Digital analysts observed Amatera and Remus detections among tens of thousands of protected Windows users over three months, with Amatera targeting Cline and Continue data and Remus targeting Claude, Cursor, and OpenCode. CallbackBeaver added Cursor and Claude to its collection scope with more than 5,000 samples in 30 days, while macOS-focused Djinn Stealer has been associated with Claude, Codex, Gemini, Cline, OpenCode, and Kilo. The stealers harvest access and refresh tokens, prompt histories, and MCP configuration files that can expose source control, ticketing, databases, cloud resources, and sensitive project context for follow-on fraud. Many stealers add targets via remotely managed rules, meaning this is an adaptation of existing infostealers rather than a new vulnerability in the AI tools themselves.

Cyber Security News·Sep 9, 11:41 UTC · 7d agoMalware in the wild2

Inoculation Midtraining with Learned Neologisms

Inoculation Midtraining confines unsafe LLM behavior to a neologism-marked context, reducing misalignment after unsafe post-training but leaking under nearby contextual cues.

The paper introduces Inoculation Midtraining, which teaches a base model during midtraining that unsafe behavior belongs to a context marked by a learned neologism token, then post-trains on unsafe data within that context. Across supervised fine-tuning and RL post-training regimes, the technique reduces misalignment while preserving transfer of benign properties like German or Shakespearean prose. However, it does not outperform standard Inoculation Prompting, is sensitive to training configuration, and produces a leaky boundary that nearby contextual cues can reactivate. The authors conclude it is not yet a load-bearing component of a developer safety framework.

arXiv cs.AI / cs.LG / cs.CL·Sep 14, 17:12 UTC · 2d agoAI safety & security

Hackers Can Hide Malicious AI Commands Inside Normal English to Bypass Security Filters

Check Point's PuzzleMask technique hides malicious prompts in ordinary English that fast gatekeeper models miss but high-reasoning downstream models execute.

Check Point researchers disclosed PuzzleMask, a technique concealing policy-breaking instructions in natural-language prose without encodings or invisible characters. Fast screening models classified all 23 crafted wrappers as safe, while a high-reasoning model recovered and acted on the hidden instruction in 17 of 18 tests (94.4%). The gap stems from capability imbalance between gatekeeper and target models, with defenses including paraphrasing untrusted input, stricter self-referential wording rules, and output/tool-call monitoring.

Cyber Security News·Sep 11, 07:36 UTC · 5d agoAI safety & security1

Hiding Prompt Injection in Legal Filing

A judge banned a plaintiff from electronic court filings after hidden prompt-injection text was discovered planted in legal documents.

Bruce Schneier's blog discusses an incident in which hidden prompt-injection instructions were planted inside a legal filing, apparently targeting AI systems that might process court documents. Judge Walter Spader Jr. responded by banning the plaintiff from electronic filings, requiring all future submissions as printed hard copies. Commenters debate whether the tactic could affect future AI-based processing of court records and whether plain-text formats will regain favor.

Schneier on Security·Aug 31, 11:04 UTC · 16d agoAI safety & security in the wild

Discounted Claude access bought on the gray market may expose every prompt you send

Help Net Security·Aug 6, 00:00 UTC · Aug 6, 2026Phishing & fraud

Hidden Prompts Trick AI Into False Email Summaries

Attackers can hide prompts in invisible HTML within emails to manipulate AI email summarizers into producing misleading summaries.

Dark Reading reports that simple HTML invisible to users can manipulate AI-powered email summarizers into generating false or malicious summaries. The technique is a form of prompt injection delivered through hidden email content. It creates risk that decision-makers act on manipulated AI-generated email summaries rather than the actual message content.

Dark Reading·Aug 25, 21:08 UTC · 22d agoAI safety & security

YuE2 · Frontier Music with Symbolic Planning

YuE2, a 3.59B-parameter music generation model, scores 6.9632 on SongBench, beating Suno v5 via symbolic planning.

YuE2 is a music generation model of roughly 3.59B parameters and 28 layers supporting song creation, covering, and agentic editing through editable ABC symbolic scores. Its best-of-8 setting reaches 6.9632 on SongBench, the highest mean among 15 evaluated settings on WildSongBench (192 prompts), ahead of Suno v5 at 6.8721. The project also introduces MERT2, whose 632M-parameter encoders achieve state of the art on 14 of 15 MARBLE metrics, and SheetSage2, which transcribes beats, downbeats, key, chords, structure, and melody with SOTA on 10 of 13 benchmark metrics.

Hacker News · AI·updated Sep 11, 03:35 UTC · 5d ago·first Sep 11, 00:33 UTC · 6d agoModel release 2 sourcesHN 43↑ · 35 comments

Amazon Kiro Prompt Injection Can Exfiltrate Sensitive Data Through Kiro Powers

Mindgard found a prompt injection flaw in Amazon Kiro IDE letting attacker-controlled workspace files exfiltrate sensitive local data; fixed in version 0.8.140.

Mindgard disclosed a prompt injection flaw in Amazon Kiro, an agentic AI IDE, that lets attacker-controlled repository content steer the agent into exfiltrating sensitive workspace data through Kiro Powers, which bundles MCP server configurations, POWER.md steering files, hooks, and contextual knowledge. Exploitation requires the user to open a malicious project via a workspace file and send any message to the agent; difficulty is rated low and it works in both trusted and untrusted workspaces. Amazon fixed the issue in Kiro IDE 0.8.140; the flaw has no CVE identifier and follows earlier Kiro bugs including CVE-2026-10591, plus related prompt-injection and code-execution issues in Codex CLI, Cursor, Gemini CLI, Copilot CLI, and Claude Code.

The Hacker News·Aug 28, 18:35 UTC · 19d agoAI safety & securityCVE-2026-10591CVE-2026-35603CVE-2026-257251

The Coding-Agent Trap: When a "Free" LLM Endpoint Is the Adversary, (Mon, Aug 31st)

A SANS honeypot caught a real coding-agent session routed to a rogue "free" LLM endpoint, exposing a Windows user's transcript and tool outputs.

A SANS analyst describes how an internet-exposed inference honeypot was discovered, relabeled with sought-after model names like DeepSeek, and enrolled in infrastructure serving "free" LLM backends. On 2026-08-30 an opencode terminal coding agent sent an 88-message, 224 KB transcript 210 times in 91 seconds via a China Unicom relay, exposing directory listings, tool outputs and read file portions. The analyst frames tool-enabled agents treating model endpoints as trusted control planes as a novel risk — a "rogue model endpoint" that could request tool executions on the user's machine.

SANS Internet Storm Center·Aug 31, 20:00 UTC · 16d agoAI safety & security1

WordlistLoader Delivers Amatera via ClickFix, SynkLoader Phishes Windows Passwords

Gen Digital details WordlistLoader delivering Amatera Stealer via ClickFix and EtherHiding, while SynkLoader phishes Windows credentials through Microsoft Teams.

Gen Digital identified WordlistLoader, an intermediate stage that reconstructs shellcode encoded as plain English words (or 16-byte UUID chunks) and loads Amatera Stealer 4.3.3-alpha1, delivered through ClearFake ClickFix prompts on compromised websites. The chain uses EtherHiding to fetch JavaScript from a blockchain smart contract, hidden cmd.exe via conhost, WebDAV-mounted shares with rundll32 execution, and ETW bypass via hardware breakpoints. The stealer adds hardened WoW64 syscalls, Heaven's Gate indirect-syscall trampolines and a redesigned application-bound encryption bypass. Separately, Expel observed SynkLoader distributed via Microsoft Teams phishing impersonating IT service desks, installing an MSI from Azure blob storage and serving a fake lock screen to capture credentials.

The Hacker News·Aug 26, 05:08 UTC · 21d agoMalware

When AI Agents Go Rogue: Agent Session Smuggling Attack in A2A Systems

Unit 42 unveils agent session smuggling, where a rogue AI agent hides covert instructions in established Agent2Agent (A2A) protocol sessions to manipulate victim agents.

Palo Alto Networks Unit 42 discovered agent session smuggling, a new attack technique in which a malicious AI agent exploits an established cross-agent session under the Agent2Agent (A2A) protocol to send covert instructions hidden among benign client requests and server responses. The technique leverages the implicit trust agents place in collaborating agents and the stateful, multi-turn nature of A2A sessions; the researchers stress it affects any stateful protocol, not an A2A flaw. Unlike one-shot data-based attacks, a rogue agent can converse, adapt and build false trust over multiple interactions. Proposed mitigations include human-in-the-loop enforcement, cryptographically signed AgentCards for remote agent verification, and context-grounding to detect injected instructions.

Palo Alto Unit 42·Aug 17, 12:32 UTC · Aug 17, 2026AI safety & security2

When the prompt becomes the payload: A practical pen-testing guide for GenAI, LLM and RAG applications

CSO Online publishes a practical penetration-testing guide for GenAI, LLM, and RAG applications, covering prompt injection, retrieval poisoning, and tenant isolation testing.

The guide frames LLM applications as attack graphs spanning prompts, retrieval layers, vector stores, tools, identities, and downstream APIs, arguing that conventional web testing misses instruction-vs-data channel risks. It builds on OWASP prompt injection guidance (direct vs. indirect injection) and NIST's 2025 adversarial machine-learning taxonomy, noting that RAG and fine-tuning do not remove injection risk. Recommended practices include documenting trust transitions across components, using canaries and synthetic records to avoid test side effects, running multi-turn and obfuscated injection campaigns, and verifying chains from poisoned documents to observable state changes. It also details testing RAG pipelines via controlled document poisoning across metadata, OCR layers, and code comments, plus cross-tenant isolation checks on retrieved document IDs.

CSO Online·Sep 9, 10:00 UTC · 7d agoAI safety & security1

A Vinyl Bar in Shibuya is a startup from a former Spotify leader for making music apps

Former Spotify innovation head raises $5.5M pre-seed for A Vinyl Bar in Shibuya, a startup building playful music-creation apps with selective generative AI features.

A Vinyl Bar in Shibuya, founded by former Spotify head of innovation Máuhan M Zonoozy, raised a $5.5M pre-seed round from Mantis VC, SV Angel, Boxgroup, Quiet Capital and others. The startup ships small music-play apps including Speed Surfer, Usersound, Stacks, Drops, Sampler, and the iOS mixer app bop, plus a new prompt-based sound creation feature. Zonoozy says the company deliberately avoids infusing AI into every product, arguing human taste and participation become more valuable as AI-generated content grows abundant.

TechCrunch · AI·Sep 14, 13:55 UTC · 2d agoAI industry 2 sources

Your AI agent's system prompt is not a security control

AWS and SANS guidance says system prompts are not security controls; enforce user permissions at retrieval time and default-deny every agent tool invocation.

AWS VP Gee Rittenhouse and SANS fellow Eric Johnson, with three AWS security specialists, published agentic AI security guidance for organizations with agents running or under development. They recommend scoping queries to user permissions inside existing RBAC/ABAC systems and filtering results before the model's context window, noting prompts can be bypassed, ignored, or overridden. The guidance warns risk concentrates when one agent holds sensitive data access, external communication, and exposure to untrusted content, the vector for prompt injection, which OWASP ranks as the top AI application threat. It cites IBM 2025 research that ungoverned shadow AI added $670,000 to average breach cost, and prescribes 30-day behavioral baselines, Cedar or Open Policy Agent for default-deny tool invocation, and four-layer containment with automatic circuit breakers.

Help Net Security·Sep 7, 12:03 UTC · 9d agoAI safety & security

The AI Supply Chain Has a Security Problem, and Much of It Is Sitting on the Open Internet

Researchers counted 36,769 publicly reachable self-hosted AI endpoints, only about 2% behind HTTP authentication, exposing Ollama, vLLM, and Flowise to abuse.

A Mysterium VPN study found 36,769 self-hosted AI endpoints reachable through internet scanning, with only 2.02% returning an HTTP authentication challenge. Open WebUI accounted for 18,529 reachable instances, Ollama for 6,935 fingerprinted hosts, and 5,223 agent-builder and workflow platforms were exposed, often holding API keys, database credentials, and other secrets. The report highlights LLMjacking risk from exposed Ollama APIs, a critical Flowise bug (CVE-2026-40933), leaked n8n tokens, and prior SentinelOne/Censys research finding roughly 175,000 exposed Ollama hosts in 130 countries.

Security Affairs·Sep 11, 17:27 UTC · 5d agoAI safety & securityCVE-2026-409331

Repeat-After-Me: Black-Box Adaptive Visual Prompt Injection

Researchers unveil Repeat-After-Me, a black-box visual prompt injection achieving over 80% success on Qwen3.6-27B and 47% on GPT-5.5.

Researchers present Repeat-After-Me, a black-box adaptive visual prompt injection that induces frontier VLMs to reveal PII or make malicious tool calls via injected images. It exceeds 80% attack success rate on Qwen3.6-27B and 47% on GPT-5.5 even when the benign user prompt is unrelated and does not authorize the injected task. In a real-world OpenClaw Discord deployment, a minimally injected image can overwrite TOOLS.md, enabling later remote code execution and secret exfiltration.

arXiv cs.CR·Sep 3, 22:44 UTC · 13d agoAI safety & security

Extortion crews have their eyes on high-value AI data, Google warns

Google's Mandiant warns extortion crews now steal proprietary AI models, prompts, and research for ransom, while threat actors automate attacks with agentic AI.

Google's AI Threat Tracker details intrusions in which extortion crews exfiltrated healthcare drug research and a proprietary AI model, and stole an AI media company's source code, prompts, skills, model scripts, and secrets, threatening leaks unless paid. TeamPCP (UNC6780), behind large open-source supply chain attacks on PyPI, npm, and Docker Hub since March, used a malicious GitHub Actions workflow to exfiltrate a company's proprietary AI repository. Mandiant also observed a fully autonomous multi-agent credential-harvesting cloud intrusion completed in under six hours, and a China-linked espionage group using Gemini to build a dynamic automated penetration-testing framework.

The Register · Security·Sep 8, 12:00 UTC · 8d agoThreat actor in the wild 2 sources1

Who Should Grade My Work? Student Perspectives on Transparent AI-Assisted Writing Assessment in Higher Education

A Saudi university study finds students value ChatGPT writing feedback but treat human instructors as the final grading authority.

Thirteen male undergraduate computing students at a Saudi public university completed handwritten writing tasks that were scored by ChatGPT using a rubric-based prompt, then reflected after being told the score and feedback were AI-generated. Inductive thematic analysis identified four themes: perceived feedback usefulness, awareness of AI's contextual and pedagogical limitations, conditional trust, and reflection on the instructor's institutional role. Participants accepted GenAI feedback for surface-level revision but consistently positioned human instructors as the authority over grading decisions, distinguishing feedback utility from evaluative authority.

arXiv cs.AI / cs.LG / cs.CL·Sep 4, 16:49 UTC · 12d agoAI research

Sophisticated Cyberattack Exposes Data of 678,000 French Taxpayers

Hackers stole personal and tax data of 678,000 individuals and businesses from France's tax agency DGFiP, prompting a Paris criminal investigation.

France's Directorate-General for Public Finances (DGFiP) confirmed a sophisticated cyberattack exposed data on 678,000 users of the tax system, including income figures, tax rates and family circumstances for individuals and SIREN registration data for businesses. The Paris prosecutor's cybercrime unit opened a probe and referred it to the French anti-fraud office OFAC after a threat actor claimed the breach in late June. Officials stressed the stolen data does not grant access to secure accounts on impots.gouv.fr, and taxpayer notification begins Monday with warnings about identity theft and fraudulent follow-up requests. The incident follows recent breaches at the ANTS documents agency and the INSEE statistics authority.

Security Affairs·Aug 16, 08:55 UTC · Aug 16, 2026Data breach

LLM Forensics: Where Do Backdoors Hide? Localizing and Controlling Trigger Mechanisms with Sparse Autoencoders

Researchers use sparse autoencoders to localize trigger-based backdoor mechanisms in 1B and 8B LLMs, finding detection features differ from causal control features.

In a controlled language-switching backdoor setting where fixed trigger sequences make 1B and 8B language models continue English prompts in French or German, the authors train sparse autoencoders (SAEs) across layers and transformer components. Attention and MLP features detect triggered prompts with near-perfect F1, but ablating them rarely suppresses the language switch, while residual-stream features can suppress triggered generation and some can induce target-language continuations without the trigger. The work decomposes token-trigger mechanisms into distinct SAE feature roles: trigger detection, residual-stream propagation, and language tracking, a decomposition the authors expect to transfer to other trigger-based backdoors.

arXiv cs.AI / cs.LG / cs.CL·Sep 7, 16:47 UTC · 9d agoAI safety & security1

Fortinet expands AI security portfolio with Virtue AI acquisition

Fortinet acquired Virtue AI to add agent red-teaming, MCP scanning and runtime guardrails to its AI security portfolio.

Fortinet acquired Virtue AI to extend its Security for AI strategy beyond FortiAIGate, which protects LLMs from prompt injection, data leakage and model poisoning. Virtue AI brings automated agentic red-teaming across 50+ sandboxed environments and 14 domains, agent discovery and governance including MCP tool scanning, continuous AI validation, and real-time guardrails across text, images, video, audio and code. Financial terms were not disclosed and the consideration is immaterial to Fortinet; Gartner projects the AI security market to grow from $2.8B in 2026 to $16.4B by 2030.

Help Net Security·Aug 17, 00:00 UTC · Aug 17, 2026Industry

Claude's new system prompt really doesn't want to reproduce song lyrics

Anthropic published updated Claude consumer system prompts, including changes steering the model away from reproducing song lyrics, likely over copyright concerns.

Anthropic publishes system prompts for Claude.ai and Claude mobile apps, including historic revisions, and has reorganized them into an index with per-model pages such as the Haiku 4.5 page showing the original October 15, 2025 prompt and an updated January 18, 2026 version. The latest consumer prompt strongly discourages reproducing song lyrics, a behavioral constraint likely tied to copyright considerations. Prompts for Claude Cowork and Claude Code are not included in the published set.

Simon Willison·Sep 2, 14:16 UTC · 14d agoAI safety & security

Critical Gitea vulnerability now exploited in the wild (CVE-2026-60004)

Attackers are actively exploiting critical Gitea RCE CVE-2026-60004 via the diffpatch endpoint, prompting CISA KEV addition and at least one cryptomining compromise.

CISA added CVE-2026-60004, a critical code injection flaw in Gitea's diffpatch endpoint, to its Known Exploited Vulnerabilities catalog after in-the-wild attacks. An attacker with repository write access, or an unauthenticated visitor on instances with open registration, can execute arbitrary shell commands as the Gitea OS user. A disclosed incident saw an automated scanner register an account and deploy a loader and cryptominer inside a Docker container within about 11 seconds, with no persistence mechanism found. Gitea patched the flaw in v1.27.1, and CISA ordered US federal civilian agencies to update by August 28, 2026.

Help Net Security·Aug 26, 00:00 UTC · 22d agoExploit / PoC in the wildCVE-2026-60004

New AI Attack Hides Malicious Instructions in Normal-Looking Text to Evade Safety Filters

Check Point researchers show crafted prose hides policy-violating instructions that bypass all tested LLM gatekeepers, including GPT-4o mini and Llama Guard 3.

A new prompt-crafting technique embeds malicious payloads inside grammatical, natural-looking text without Base64, invisible Unicode, or obvious encodings, defeating lightweight pre-screening gatekeepers. In testing, all four evaluated gatekeeper models—gpt-4o-mini-2024-07-18, gpt-oss-safeguard:20b, claude-3-haiku-20240307, and llama-guard3:8b—classified the crafted wrappers as safe at a 100% bypass rate across 23 obfuscated prompts. GPT-5 Thinking in high-reasoning mode recovered and acted on the hidden instruction in 17 of 18 tests (~94.4%), often spending over a minute and multiple Python executions. Researchers recommend paraphrasing untrusted input, hardening gatekeeper policies, and applying defense-in-depth controls for agentic deployments.

GBHackers·Sep 11, 05:53 UTC · 5d agoAI safety & security 2 sources

Delta investigates in-flight Wi-Fi spoofing on post

A passenger on Delta flight 591 allegedly spoofed the airline's in-flight Wi-Fi with a rogue access point, prompting an FBI and FAA investigation.

Delta Airlines is investigating an incident on flight 591 from Las Vegas to Atlanta in which a passenger reportedly created a rogue Wi-Fi network named 'Delta WiFi Fast' to steal sensitive passenger data. The cabin crew deactivated the aircraft's Wi-Fi for roughly 30 minutes, and Delta says flight safety and aircraft operating systems were never affected. The FBI's Atlanta office and the FAA acknowledged awareness of the incident, which bears hallmarks of an evil twin attack involving spoofed login portals and credential harvesting. DEF CON organizers said this year's conference in Las Vegas also suffered multiple similar deauthentication Wi-Fi attacks.

CyberScoop·Aug 11, 21:35 UTC · Aug 11, 2026Phishing & fraud in the wild

Prompt Injection Remains Biggest LLM Risk, Despite Limited Incidents

Infosecurity Magazine·Aug 5, 11:00 UTC · Aug 5, 2026AI safety & security

Researchers Disclose AI-Assisted SharePoint Exploit Chain Reaching Unauthenticated RCE

Rapid7 disclosed CVE-2026-55040, a SharePoint JWT validation bypass chaining into CVE-2026-63520 unauthenticated RCE, with research substantially AI-agent-assisted; patches released.

Rapid7 disclosed CVE-2026-55040 (CVSS 9.1), several JWT validation pipeline issues letting unauthenticated attackers impersonate chosen SharePoint users by SID or UPN, chained with CVE-2026-63520 (CVSS 8.1), an unsafe .NET type instantiation in Business Connectivity Services yielding RCE as the service account. An AI agent contributed significantly across 96 sessions and roughly 80,000 tool calls over 24 active days, though an expert had to steer it and it repeatedly overstepped its threat model. No exploitation of the bypass had been reported as of CISA's July 14 assessment. The RCE affects SharePoint Subscription Edition, 2019, and 2016, plus Project Server 2013 SP1 and Office Web Apps 2013 SP1; the July updates break the chain.

The Hacker News·Aug 11, 16:49 UTC · Aug 11, 2026VulnerabilityCVE-2026-55040CVE-2026-635201

Mind2Dialogue: Training Human-Aware Language Models by Simulating User Mental States

Mind2Dialogue simulates users' mental states to generate privileged supervision, boosting personalization and preference-following in Qwen, Llama, and OLMo assistants.

The Mind2Dialogue framework uses a psychology-guided simulator that preserves personal characteristics while updating user mental states through interaction, driving coherent conversations and an Oracle assistant's responses. Privileged distillation trains models on the Oracle's well-informed responses so they can assist users without direct access to mental states at deployment. Training on the full corpus improves every reported personalization metric over Qwen, Llama, and OLMo instruction-tuned baselines, including 26.6 to 40.9 percentage point gains in preference-following generation.

Hugging Face daily papers·updated Sep 14, 17:55 UTC · 2d ago·first Sep 13, 20:00 UTC · 3d agoAI research 2 sources

Authorization Architectures for Tool-Using AI Agents

Review paper proposes an authorization reference architecture for tool-using AI agents, identifying runtime enforcement and delegation bounds as unresolved gaps.

This review examines authorization models for tool-using AI agents that invoke APIs, databases, browsers, and protocols like MCP, arguing every consequential agent action must be traceable to a human principal, bounded by delegation, and contestable. It introduces a principal hierarchy spanning human user, operator/deployer, orchestrator agent, sub-agent, and tool endpoint, and analyzes five layers including credential lifecycle, delegation propagation, runtime enforcement, prompt injection as authorization bypass, and auditability. Drawing on 89 primary sources from 2023-2026, it proposes seven structural requirements, a four-layer reference architecture, and three deployable configurations.

arXiv cs.CR·Sep 14, 17:21 UTC · 2d agoAI safety & security

AuK Technical Report: An Open-Source Foundational Model for Speech Generation and Editing

Open-source speech foundation model AuK unifies generation and editing, trained on 1.95 million hours, with distilled AuK-Flash achieving 4.5x speedup.

AuK is an open-source foundational model that unifies speech generation and editing through natural-language instructions and audio context, trained on approximately 3.03 billion instruction-audio instances and 1.95 million hours of supervision across five task families including generation, content editing, and acoustic editing. It combines a multimodal LLM for semantic conditioning, a VAE jointly trained on speech, general audio, and music, and a hybrid rectified-flow Transformer using dual-stream MMDiT blocks followed by unified single-stream DiT blocks. Post-training applies human-feedback preference optimization for editing and reward-based reinforcement learning for generation, and the distilled AuK-Flash performs 4-step inference without classifier-free guidance at a 4.5x wall-clock speedup. Source code and model weights are released.

Hugging Face daily papers·Sep 7, 20:00 UTC · 9d agoModel release2

Microsoft Offers Up to $30,000 for Critical AI Flaws in Dynamics 365 and Power Platform

Microsoft expands AI bug bounty to Dynamics 365 and Power Platform, paying up to $30,000 for critical inference manipulation flaws.

Microsoft's bug bounty program offers up to $30,000 for critical 'Inference Manipulation' or 'Inferential Information Disclosure' bugs in Dynamics 365 and Power Platform, including Copilot Studio, AI Builder, Power Apps, Power Automate, and Dataverse. Payouts scale by report quality ($30,000/$20,000/$12,000 for critical) with important-severity AI flaws earning $6,000-$20,000, plus 20% multipliers for Dataverse privilege escalation and Plugin Sandbox escapes. Prompt injection affecting only the attacker, hallucinated execution, and system-prompt disclosure are excluded from scope.

Cyber Security News·Sep 14, 16:11 UTC · 2d agoIndustry

Measuring Language Transfer in Robot Policies: Adding Greek to a Cosmos3 Vision-Language-Action Policy

Researchers added Greek to the Cosmos3 vision-language-action policy using only machine-rephrased instructions, finding bilingual training reaches roughly two fifths of English performance.

The paper studies localizing the open Cosmos3 vision-language-action robot policy to Greek without architectural changes, using machine-rephrased instructions only. Bilingual training yields a consistent 6.7-7.1 point margin over controls on a 90-task, three-seed evaluation suite, while Greek-only training gains at most 2.7 points. Several common evaluation instruments, including color-histogram metrics and single-goal benchmarks, produced false conclusions, and results were dominated by seed variation. The authors recommend building guaranteed-null baselines and replicating low-resource-language results across seeds.

Hugging Face daily papers·Sep 6, 20:00 UTC · 10d agoAI research

When the Whole Company Adopts AI: What It Does to Your SOC

Analysis of 16.9 million SOC alerts finds AI-related alerts at 0.43%, growing 685% since February, with 94.1% noise and 0.02% real attacks.

A review of roughly 16.9 million SOC alerts found about 73,000 (0.43%) were AI-related, a share that grew 685% between February and June 2026. Of AI-related alerts, 94.1% were noise, 5.8% genuine risks, and 0.02% real attacks; 79.8% received benign verdicts, 81.7% were automatically suppressed, and only 5.4% reached a human analyst. The only confirmed attacks were phishing campaigns that weaponized AI brand names as lures, while developer coding agents spawning shells and reading credential stores routinely tripped detections written before AI agents existed.

The Hacker News·Sep 12, 10:24 UTC · 4d agoResearch1

The Hidden Instructions That Can Hijack AI Agents

Hidden prompt injections embedded in documents and metadata can hijack autonomous AI agents, causing data exfiltration and out-of-policy actions at machine speed.

Bowbridge warns that hidden indirect prompt injections, embedded in documents, metadata, emails, images, and code repositories, can cause autonomous AI agents to treat attacker-controlled content as trusted guidance. Because agents inherit user privileges, act silently, and lack human judgment, injections can lead to data exfiltration or file poisoning that traditional security controls cannot detect. A real-world example involved a supplier quote whose metadata instructed an agent to override guidance and select the most expensive option. Bowbridge recommends scanning documents before agents process them.

SecurityWeek·Sep 8, 17:00 UTC · 8d agoAI safety & security

Exaforce extends its AI security tool to monitor more than just Claude

Exaforce AI Security extends beyond Claude to monitor OpenAI, Gemini, and Copilot agents using existing SOC telemetry, no new endpoint agents.

Exaforce expanded its June Claude Compliance API integration into Exaforce AI Security, adding monitoring for OpenAI, Gemini, Microsoft Copilot, and OAuth-connected AI apps. The tool inventories AI agents by correlating EDR, cloud, SaaS, and model-provider logs without new gateways or endpoint agents, and can respond by revoking sessions, deactivating API keys, isolating devices, or killing agent processes via existing controls. Analysts note the agentless approach lowers friction but lacks runtime inspection and inline blocking offered by competitors such as Palo Alto Prisma AIRS, SentinelOne Prompt AI Agent Security, and CrowdStrike Falcon Guardian. A March 2026 Cloud Security Alliance survey found 68% of organizations cannot distinguish human from AI-agent activity and 74% report AI agents receive excessive access.

CSO Online·Sep 15, 13:00 UTC · 1d agoTools

Show HN: LLM Attention Visualization

A developer released a browser-based tool that visualizes which past tokens influence each LLM output token using aggregated, value-weighted attention scores.

A Show HN project presents a React application built on Transformers.js that renders per-token attention influence by aggregating attention weights scaled by value-vector magnitudes across all attention heads and layers. To expose internal tensors, the author instrumented the ONNX computation graph, hosted a modified model on Hugging Face, and pre-generated prompts to avoid long model downloads in the browser. Demos with a 600-million-parameter model show how verbatim copying draws heavily on source tokens and how single outputs blend information from multiple phrases.

Hacker News · AI·Sep 8, 16:59 UTC · 8d agoAI tools & infraHN 41↑ · 9 comments

LexFlip: A Dissociation Diagnostic for Legal Meaning Preservation Metrics

LexFlip releases 373 minimal perturbations of Quebec statutory French that reverse legal force while preserving tokens, exposing weaknesses in embedding-based meaning preservation metrics.

LexFlip provides 373 minimal perturbations of Quebec statutory French that reverse legal force while preserving 0.93 of tokens, creating dissociation items that break monotone token-overlap metric validation. The seven embedding and BERTScore metrics tested register only 0.022-0.039 of their identical-to-unrelated range on these edits, versus 0.670 for bidirectional NLI. Against FrJudge, with a measured human ceiling of r=0.597, a bare length feature outscores every semantic metric tested.

arXiv cs.AI / cs.LG / cs.CL·Sep 4, 15:49 UTC · 12d agoAI research

Microsoft Copilot reveals secret input that allowed it to be hacked

Microsoft disclosed a hidden input in Copilot that let attackers steal passwords from users who clicked a crafted link.

Microsoft revealed that Copilot contained a secret, undocumented input parameter that allowed the assistant to be compromised. Attackers could abuse the hidden input to steal passwords when a target clicked a malicious link. The disclosure highlights hidden-parameter risks in widely deployed AI assistants.

Ars Technica · Security·Aug 18, 13:00 UTC · 29d agoAI safety & security in the wild1