ZeroHour
DashboardNewsCVEsExploitsIncidentsAIBriefAnalyticsEntitiesWatchlistIOCsAsk the feedSourcesFeeds & APIKiosk mode

Source: arXiv cs.CR

55 stories in the last 30d

Sortlatesthotimportance
DomainAllCyberAIWindow24h3d7d30dAllIn the wildImportant onlyShow duplicates
CategoryAllAI safety & security 51AI research 3Model release 1AI policy 1
SeverityAllcritical 0high 0medium 1low 0info 55source: arXiv cs.CR ✕

AgentLSD: Evaluating AI Security Agents Under Adversarial Task Contamination

AgentLSD benchmark shows deceptive CTF artifacts like fake flags and decoy endpoints steer AI security agents wrong, inflating turns and tokens.

The paper defines adversarial task contamination, where deceptive artifacts in agent environments, including non-instructional evidence beyond prompt injection, influence AI security agents. AgentLSD injects trap artifacts such as fake flags, misleading hints, decoy endpoints, and hidden cues into 11 web CTF challenges, evaluating six models with paired clean and trap-augmented runs. Clean-condition agents capture 41% of flags, and even successful captures see roughly +20 turns and +2k reasoning tokens, with heterogeneous solve-rate effects. The framework, configurations, and traces are released.

arXiv cs.CR · 18h agoAI safety & security

Characterizing Network Centralization and Observability in the Remote MCP Ecosystem

A measurement study of 179 remote MCP servers finds heavy infrastructure concentration (HHI 0.736) and a security-observability tradeoff in platform OAuth.

The paper introduces a three-tier observability framework (catalog metadata, passive compliance signals, live vulnerability analysis) applied to a stratified sample of 179 remote Model Context Protocol (MCP) endpoints from two public registries. The Herfindahl-Hirschman Index over ASN distribution is 0.736, well above the 0.25 high-concentration threshold, and 95% of commercial PaaS-hosted servers enforce gateway-level OAuth 2.1 with PKCE. Authentication correlates strongly with hosting platform choice rather than operator configuration, creating a security-observability tradeoff that constrains automated scanning for tool-poisoning vectors without prior credentials.

arXiv cs.CR · 18h agoAI safety & security

Securing quantum error correction against misleading advice from AI agents

Researchers design calibration-based certified checks that let quantum error-correction systems safely reject harmful recovery updates proposed by compromised AI advisers.

The paper shows that opposite coherent X rotations in an odd-distance square toric code yield identical passive syndrome histories, creating ambiguity an AI adviser could exploit to recommend harmful recovery updates. It introduces terminal logical measurements on calibration states plus an independent evaluator that accepts updates only when calibration uncertainty and drift bounds certify improvement. Simulated advice attacks showed calibration-confidence checks reject harmful proposals while retaining most beneficial updates, and the authors derive sufficient limits on calibration age.

arXiv cs.CR · 18h agoAI safety & security

ASLEval: Measuring Privacy Exposure Displacement in LLM Agent Sessions

ASLEval benchmark shows local privacy proxies miss 46.9% of LLM agent session exposure recovered by measuring all visible exits.

Researchers introduce privacy exposure displacement, the mismatch between local evaluation proxies and target-grounded exposure across full LLM agent sessions, and ASLEval, an authorization-aware framework that pre-registers hidden target sets and measures all declared visible exits. Across enterprise-style environments and independently implemented runtimes, expected-outlet-only views missed 46.9% of exposure recovered by the visible-exit union, and attacker self-reports combined omissions with high false discovery. Schema-aligned internal evidence usually preceded visible exposure at the request/probe level. The authors argue benchmarks should declare the complete visible boundary and report privacy alongside task utility.

arXiv cs.CR · 20h agoAI safety & security

Epsilon-Nash Equilibria in History-Dependent SA-MDPs

Researchers give the first algorithm for computing epsilon-approximate history-dependent equilibria in state-adversarial Markov decision processes with observation-perturbing adversaries.

The paper studies state-adversarial Markov decision processes (SA-MDPs) where an adversary knowing the true state perturbs observations within state-dependent proximity sets each step. The authors prove universal history-dependent equilibrium policies do not exist and reduce SA-MDPs to a strategically equivalent constrained zero-sum one-sided partially observable stochastic game, enabling the first algorithmic route to epsilon-approximations of initial-state dependent equilibria. The algorithm is validated on small analytically verifiable games and scales to larger benchmarks, including Atari Freeway rollouts with a 12-period-ahead horizon.

arXiv cs.CR · 20h agoAI safety & security

CaMeLoT: CaMeL orchestrated with Temporal logic for static verification and liveness

Researchers present CaMeLoT, extending CaMeL with CTL model checking that statically rejects unsafe LLM agent plans before any tool executes.

CaMeLoT adds a static verification layer to CaMeL, a runtime defense against prompt injection in tool-using LLM agents. It translates a generated plan into a finite-state transition system, labels it with tool calls, provenance, and taint information, and checks it against CTL temporal policies using the nuXmv model checker before any tool is invoked. Failed checks return counterexamples for plan repair, avoiding LLM calls, tool calls, and sandbox teardown. Evaluation covers policies derived from AgentDojo, SOC workflows, and prompt-extraction experiments.

arXiv cs.CR · 22h agoAI safety & security

A Security Risk Assessment Framework for AI-Powered Development Tools

Researchers propose SRF, a framework showing AI-generated code from multiple development tools introduces vulnerabilities, worst in input and file handling tasks.

The paper presents the Security Risk Assessment Framework (SRF), combining threat modeling, security analysis, and quantitative risk evaluation based on vulnerability criticality for AI-generated code. Code generated by multiple AI-powered development tools was analyzed with Bandit and Semgrep across security-relevant programming tasks. All evaluated tools introduced vulnerabilities; risk varied mainly by task type, with input processing and file handling showing higher risk, while differences between tools were smaller than differences across task categories.

arXiv cs.CR · 22h agoAI safety & security

The Illusion of Local Privacy: Confidentiality Boundary Failures in Consumer LLM Serving Systems

Researchers show local LLM serving systems leak prompts via memory residue, plaintext persistence, a llama.cpp tenant-isolation flaw, and timing oracles.

A study of consumer local-LLM serving systems identifies four boundaries where prompt confidentiality fails: model loading, runtime memory, wrapper persistence, and the serving interface. Using the LLAnalyzer framework across four open-weight model families and two deployment platforms, the authors recover plaintext prompts from allocator-managed memory after inference and show wrappers extend prompt lifetime. They also uncover a previously undocumented llama.cpp authorization flaw letting one authenticated client restore another tenant's saved conversation state, succeeding in 200/200 trials, plus a remote timing oracle via shared prompt-prefix caching that works over WAN.

arXiv cs.CR · 1d agoAI safety & security

Robot Visions: Breaking reCAPTCHA at Zero Cost and Zero Shot

Researchers defeat Google reCAPTCHA using free local models CLIP and OWLv2, achieving 92.6% per-session success at zero cost.

The paper taxonomizes Google reCAPTCHA challenges into Type A (independent tiles) and Type B (4x4 grid) and builds zero-shot, training-free solvers from open-source local models. CLIP solves 58% of Type A challenges and OWLv2 43.5% of Type B, while an end-to-end automated solver succeeds on 92.6% of 500 real-world sessions. The authors also show a non-technical adversary can solve challenges using natural-language instructions to a commodity AI assistant, collapsing the attacker skill floor and suggesting visual challenge CAPTCHAs have reached the end of their useful life.

arXiv cs.CR · 1d agoAI safety & security

MiST: Mid-Training LLMs for Cybersecurity

MiST introduces 8B and 32B cybersecurity-specialized LLMs that outperform Qwen baselines by up to 13.1 points on public security benchmarks.

MiST (Mid-trained Security Transformer) applies mid-training as an intermediate adaptation stage, converting an expert-vetted seed corpus into high-quality synthetic domain data rather than continual pretraining on raw text. The 8B and 32B checkpoints improve mean cybersecurity accuracy by +13.1 and +8.6 absolute points over Qwen baselines (+27.0% and +15.8% relative). Ablations show gains arise in mid-training and supervised fine-tuning, and MiST provides stronger initialization for downstream fine-tuning and reinforcement learning.

arXiv cs.CR · 1d agoModel release

Collective Loss of Control in LLM Agent Systems: An Epidemic Account of Mutation, Contagion, and Recovery

Researchers model multi-agent LLM failure as an epidemic, showing injected unsafe strategies spread with 40-95% executed harm across routes.

The paper proposes an epidemic account of collective loss of control in LLM agent systems built on mutation, contagion, and recovery, motivated by reported OpenAI agent coordination incidents. A deployment audit found implicit communication paths between nominally independent evaluation runs transported via a default Docker backend. The RogueHandoff-20 benchmark of 20 executable scenarios injects unsafe trajectories from a modified Qwen-27B route, showing executed harm of 0-5% on normal tasks but 40-95% after injection, exceeding paired direct malicious requests by 5-45 percentage points.

arXiv cs.CR · 1d agoAI safety & security

The Verifiable Action Card: Trustworthy Human-in-the-Loop Control for Secure Autonomous Agents

Verifiable Action Card architecture blocks indirect prompt injection in agentic browsers, cutting attack success from 68-100% to 0%.

Researchers propose VAC, a browser-architecture defense that reconstructs approval prompts from the ground-truth pending action and trusted intent provenance, rendering them out-of-band in trusted browser chrome. On a 24-scenario benchmark covering confused-deputy attacks, dialog forging, and indirect prompt injection, attack success fell from 68-100% to 0% across evaluated LLMs, with 78% legitimate-task completion and a 0% false-block rate. Approval is bound to the exact action re-verified at dispatch.

arXiv cs.CR · 1d agoAI safety & security

When Agents See Differently: Exposing UI Desynchronization Threats in Mobile Agents

Researchers expose 'human-agent UI desynchronization' attacks where repackaged APKs invisibly mislead mobile AI agents into attacker-chosen actions.

The paper introduces human-agent UI desynchronization: agents ingest digital screenshots and accessibility metadata that reveal content human users cannot perceive due to occlusion and luminance-contrast limits. An automated framework embeds perturbations into repackaged APK clones that steer mobile agents toward attacker-designated actions without access to runtime user instructions or online adaptation. Evaluations across five mobile-agent frameworks and three backbone models on 546 tasks achieved average misleading rates of 77.9% and 66.9%. A questionnaire study with 186 participants found the visual perturbations difficult for humans to notice.

arXiv cs.CR · 2d agoAI safety & security

Toward Secure AI-Powered Penetration Testing Agents: Security Threats, Guardrails, and Architectural Perspectives

Paper proposes a threat taxonomy and guardrail analysis for LLM-powered autonomous penetration testing agents, covering lifecycle, architecture, and behavioral attacks.

The paper analyzes security threats to autonomous LLM-based penetration testing agents that independently perform reconnaissance, vulnerability identification, exploitation planning, and post-exploitation with minimal human supervision. It characterizes trust boundaries and attack surfaces of representative agent architectures and proposes a threat taxonomy spanning LLM lifecycle attacks, agent-architecture attacks, and cross-cutting behavioral attacks. The authors argue existing conversational-AI guardrails are insufficient for agentic, long-horizon offensive workflows and outline research directions for context-aware, architecture-aware guardrails.

arXiv cs.CR · 2d agoAI safety & security

Adversarial Testing of Automated Program Repair Agents for Security Vulnerabilities

SWEADV benchmark shows adversarial issue descriptions make LLM program-repair agents write insecure fixes in 51.7% of cases, evading most detection tools.

Researchers built SWEADV, a benchmark of 750 adversarial issue descriptions derived from 150 SWE-bench Verified repair tasks, covering command execution, deserialization, path traversal, denial of service, and weak hashing attack types. Tested on mini_swe agents backed by GPT-5-Mini, MiniMax-M2.5, and DeepSeek-R, adversarial descriptions induced malicious behavior with successful repair in 51.7% of cases. Detection was weak: LLM-as-judge pre-repair screening reached only 62.3% accuracy, and post-repair detection via static analysis and LLM-as-judge achieved just 39.4% and 55.4%.

arXiv cs.CR · 2d agoAI safety & security2

Authorization Architectures for Tool-Using AI Agents

Review paper proposes an authorization reference architecture for tool-using AI agents, identifying runtime enforcement and delegation bounds as unresolved gaps.

This review examines authorization models for tool-using AI agents that invoke APIs, databases, browsers, and protocols like MCP, arguing every consequential agent action must be traceable to a human principal, bounded by delegation, and contestable. It introduces a principal hierarchy spanning human user, operator/deployer, orchestrator agent, sub-agent, and tool endpoint, and analyzes five layers including credential lifecycle, delegation propagation, runtime enforcement, prompt injection as authorization bypass, and auditability. Drawing on 89 primary sources from 2023-2026, it proposes seven structural requirements, a four-layer reference architecture, and three deployable configurations.

arXiv cs.CR · 2d agoAI safety & security

RAPID: A Real-Time Defense Against Unauthorized Model Distillation for Text-to-Image Services

RAPID embeds defensive perturbations in a T2I model's shared VAE decoder to block unauthorized black-box distillation in real time.

The paper defends text-to-image services against model theft via black-box output-based distillation, where adversaries collect prompt-image pairs to train substitute models. RAPID integrates defensive perturbations into the shared VAE decoder using self-referenced latent maximization plus reconstruction-guided color regularization, avoiding costly sample-wise online optimization. Across four T2I models and four datasets versus five baselines, it consistently degrades substitute-model generation quality while preserving visual fidelity.

arXiv cs.CR · 2d agoAI safety & security

When the World Lies: Backdoor Attacks on Latent World Models for Downstream Control

A poisoned world-model checkpoint hijacks downstream controllers without an explicit trigger rule, passing clean-data evaluation while steering 100% of triggered actions.

Researchers show that a released pretrained world-model checkpoint acts as a supply-chain backdoor for downstream control. The poisoned model routes trigger-bearing observations into a chosen latent region and reshapes dynamics so the victim's own Dreamer-style actor training or MPC/CEM planning re-discovers attacker-targeted actions. The attack hijacks 100% of triggered steps in the strongest settings while retaining roughly 75% clean-task success and passing standard clean-data diagnostics. Moderate clean fine-tuning fails to remove the backdoor without substantially degrading clean control.

arXiv cs.CR · 2d agoAI safety & security

CiteShade: Citation Laundering in Multi-Source Retrieval-Augmented Generation and Its Counterfactual Defense

CiteShade attack makes RAG models cite trusted sources for attacker-chosen wrong answers, raising wrong-answer rate from 0.01 to 0.68.

CiteShade is presented as the first citation laundering attack against multi-source retrieval-augmented generation: an attacker controlling a single source induces a wrong answer falsely attributed to a trusted source, even while correct evidence remains in context. The attack is formalized via three necessary conditions (retrieval, generation, citation) constructible without any instructions, raising wrong-answer rate from 0.01 to 0.68 on multi-hop QA, with source deletion confirming the malicious source as causal driver. Vulnerability tracks a model's citation propensity rather than scale, reaching CLR 0.84 with explicit instruction and 0.64 without on the most citation-prone model. Perplexity filtering and citation-support checking prove insufficient; the authors propose a counterfactual defense verifying which source actually drove the answer.

arXiv cs.CR · 2d agoAI safety & security1

Approval Integrity and Recovery in LLM Answer Publication

Study measures approval integrity in Lightcap LLM answer publication, finding the 14B response-act checker accepts 291 of 302 unsupported answers.

The study evaluates exact-content binding, authorization freshness, and checkpoint recovery in Lightcap's publication enforcement using 3,600 assessments over 900 human-annotated RAGTruth responses from three Ministral models. The production 14B response-act checker accepts 291 of 302 unsupported answers versus 41 for a direct-grounding baseline, with supported-answer retention of 95.2% versus 66.9%. A stateful recheck-recovery policy increases exact-match error by 9.23 percentage points relative to initial checkpoints, and controlled evidence-fingerprint changes expose asymmetric freshness enforcement between publication and recovery. A separate BIPIA prompt-injection experiment records zero target insertions among 266 valid editor outputs.

arXiv cs.CR · 2d agoAI safety & security

Misleading the Planner through Deceptive Resumes: Registration-Time Injection in Centralized Multi-Agent Systems

Researchers demonstrate registration-time prompt injection in centralized LLM multi-agent systems, dropping GAIA task success from 84.31% to 37.25%, and propose DescGuard defense.

The paper identifies a registration-time injection channel in centralized LLM multi-agent systems where third-party worker agent descriptions are trusted by the planner before any user instruction arrives. Analyzing 32,000 descriptions from three public agent marketplaces, at least 23.35% contain content outside the four defined description fields. Eight description-manipulation attack strategies targeting task decomposition, capability grounding, and subtask specification cut GAIA task success from 84.31% to 37.25% and increased token consumption or execution time by over 111%, persisting across two MAS implementations, six planner LLMs, and four evaluators. The proposed DescGuard defense filters descriptions to worker-scoped interface information and restores metrics toward baseline without modifying workers, planner, or orchestration logic.

arXiv cs.CR · 2d agoAI safety & security

Empirical Evaluation of Task-Based Permission Scoping Architecture for AI Agents

Fine-tuned RoBERTa-large task permission classifier matches Claude Haiku 4.5 on access scoping for AI agents, cutting severity-weighted attack surface by 84.4%.

The paper evaluates a three-source task-based permission architecture for AI agents combining role-based permission ceilings, a task permission classifier, and policy-based prohibitions. A fine-tuned RoBERTa-large security gate matched few-shot Claude Haiku 4.5 on a 600-prompt dataset, with macro-F1 0.881 versus 0.886, precision 0.897 versus 0.842, and lower severity-weighted residual risk (0.63 versus 1.12). An attack-surface elimination metric shows the role ceiling alone closes 27.9% of the severity-weighted surface while adding the task classifier closes 84.4%. The work establishes task-granular access control as a measured, deployable mechanism for reducing attack surface in agentic deployments.

arXiv cs.CR · 3d agoAI safety & security

Divide, Consult, Conquer: Capability Laundering Through Aligned LLMs

Attack shows unaligned orchestrators can launder capabilities from aligned frontier LLMs via benign subtask consultation, raising Gemma-4-31B CBRN rubric score from 62.3 to 83.1.

The paper introduces capability laundering, where a weaker unaligned model decomposes a harmful task into benign-looking subproblems, queries a stronger aligned model on each, and recombines answers locally, bypassing per-interaction safety evaluations. Evaluation used GPT-5.5, Claude Opus 4.8, and Grok-4.3 as consultants to four local orchestrators on CyBench, BountyBench, and CBRN tasks. On CyBench, Gemma-4-31B recovered 8/14 candidate tasks with GPT-5.5 and 7/9 with Opus, while Muse-Glimmer-30B recovered none. Across an eight-step hypothetical bioweapon attack chain, consultation raised Gemma-4-31B's mean rubric score from 62.3 to 83.1, exposing a gap in defenses that only refuse complete harmful tasks.

arXiv cs.CR · 3d agoAI safety & security

CounterPersona: Append-Only Defense Against Unauthorized Persona Skill Distillation

CounterPersona appends targeted counter-persona evidence after data collection to block AI systems from distilling an individual's behavioral patterns into reusable skills.

CounterPersona defends against unauthorized persona skill distillation, where attackers extract recurring patterns from collected personal data to replicate an individual's behavior. Unlike perturbation-based defenses that require modifying data before collection, it works in an append-only setting where historical records cannot be altered or revoked. It constructs targeted counter-persona evidence, packs compatible behavioral states into compact realization units, and strengthens them via rationale-guided consistency rewriting. Experiments show strong effectiveness across lexical, semantic, and LLM-based measures, remaining robust across different distillers.

arXiv cs.CR · 3d agoAI safety & security1

Pick Your Poison: Learning to Select Poison Sets for Stronger LLM Backdoor Attacks

SAILS learns to select poison sets for LLM backdoor attacks, showing attack success ranges 3% to 80% at fixed poison counts across LLaMA-3-8B settings.

The paper shows existing backdoor evaluations that randomly sample a fixed number of poisoned examples severely underestimate worst-case vulnerability: across three LLaMA-3-8B settings, attack success ranges from 3% to 80% depending only on which poison set is chosen. SAILS formalizes poison selection as oracle-budgeted set optimization, learning a set scorer from a few hundred finetune-and-evaluate runs to rank millions of candidate sets and audit a small shortlist. It improves held-out attack success by 30 percentage points over the strongest influence baselines and transfers from small-scale to full-scale finetuning, extending to code-generation, agentic, and API-only backdoors.

arXiv cs.CR · 3d agoAI safety & security 2 sources1

PIDS-Bench: Evaluating Prompt-Injection Detectors Under Over-Defense, Obfuscation, and Distribution Shift

PIDS-Bench shows prompt-injection detectors scoring F1 above 0.98 still misclassify about one-third of external benign security-adjacent prompts, revealing provenance-sensitive over-defense.

PIDS-Bench is a frozen multi-axis benchmark that jointly evaluates prompt-injection detectors on attack detection and benign false-positive behavior at fixed thresholds, spanning in-distribution inputs, hard-benign prompts, obfuscated attacks, and domain/structural distribution shifts. It evaluates seven detectors plus a rule-based lower-bound reference. A detector exceeding F1 = 0.98 on held-out data still misclassifies roughly one-third of an externally-sourced benign security-adjacent subset, and no internal detector reaches F1 >= 0.95 with hard-benign FPR <= 0.10 on the stress distribution. Hard-negative augmentation nearly eliminates over-defense on curated stress inputs but leaves it intact on externally-sourced prompts, a pattern termed provenance-sensitive over-defense.

arXiv cs.CR · 3d agoAI safety & security

ActGuard: Pre-execution Action Auditing against Indirect Prompt Injection in LLM Agents

ActGuard audits LLM agent actions before execution against predicted tool priors, masking only malicious spans from indirect prompt injections while preserving utility.

ActGuard is a pre-execution action auditing framework against indirect prompt injection in LLM agents, judging whether external content causes the current action to deviate from a locally reasonable expectation rather than whether content is inherently suspicious. At each step it predicts the tools likely used by the upcoming action, builds a local tool prior, then performs tool-level contrastive analysis and parameter-level evidence localization to identify deviations. A verifier masks only spans confirmed as malicious and regenerates the action from the sanitized context. On challenging tool-using agent benchmarks it reduces attack success to state-of-the-art levels while keeping task utility close to the no-attack setting; code is publicly available on GitHub.

arXiv cs.CR · 3d agoAI safety & security

Evaluating Context Segmentation in Locally Deployable SLMs for Cybersecurity CTF Tasks

Context segmentation framework boosts memory-constrained gemma-4 agents on picoCTF, solving 18.52% of tasks standard execution fails, highlighting local SLM offensive risk.

The paper introduces context segmentation, a two-level agentic framework that divides long-horizon CTF exploitation tasks into contextually isolated sub-problems to counter context bloat and cognitive degradation from accumulated tool-call outputs. It evaluates memory-constrained gemma-4 models on the picoCTF dataset; the E4B model achieves competitive rewards with superior token efficiency compared to brute-force retries. It solves 18.52% of tasks that standard agentic execution fails to complete. The work frames locally deployed open-weight SLMs as an escalating risk since they bypass proprietary API guardrails; code is released on GitHub.

arXiv cs.CR · 5d agoAI safety & security1

NovaFabric: Tamper-Evident, Replayable Evidence for Autonomous AI Agent Runs

NovaFabric seals autonomous AI agent runs into tamper-evident, replayable Run Capsules enabling third-party audit under EU AI Act and ISO 42001.

NovaFabric records autonomous agent runs without modifying agent logic into portable Run Capsules (fifteen-entity schema) sealed with DSSE signatures, RFC 3161 timestamps, a Merkle log, and redaction attestations, supporting four-mode replay and third-party Evidence Bundle verification. Evaluation shows tampering rejected across three tested classes, 14/14 credential types redacted while preserving 9/9 decoys, and 140/140 mutations localized; blast-radius queries reach 45.5ms p99 over 10M edges. Limits include only 2/10 tool-using workloads completing replay due to missing tool-response substitution and ingest capped at 61.6 req/s. The contribution integrates OpenTelemetry, DSSE/in-toto, and W3C PROV rather than new cryptography.

arXiv cs.CR · 6d agoAI safety & security1

Signing the Transaction but Not the Decision: Whisper Attacks and a Binding Defense for AP2

Research shows AP2 agent-payment signatures can be manipulated into valid but wrong carts; proposed A-VIP defense binds signed intent to purchases.

A study demonstrates Whisper attacks on the AP2 agent payment protocol, where ordinary product-description text steers shopping agents into carts that pass every cryptographic check but no longer match user intent. Using Gemini Flash-Lite models specified by AP2's default sample agents, three attacks succeeded at 90%, 56%, and 73.3%, with the vulnerability spanning seventeen Google models, three agent frameworks, cross-vendor anchors, and Google's consumer assistant. The proposed A-VIP defense treats signed intent as a capability grant, binding credential lookups to sessions and cart lines to seen listings, blocking the first two attacks with zero false positives while surfacing unauthorized spending. The authors release A-VIP code, machine-checked invariants, and AP2-WhisperBench with 1,544 evaluation scenarios.

arXiv cs.CRupdated · 6d agofirst · 6d agoAI safety & security 2 sources1· 1 read

From Intent to Execution Grant: An Execution-Boundary Conformance Profile for High-Risk AI Actions

Researchers specify EBL-Core, an execution-boundary conformance profile binding AI agent intents, policies, and evidence into verifiable execution grants, validated with bounded tests.

The paper defines EBL-Core, a conformance profile deciding whether one fully materialized AI-generated candidate action may receive action-scoped execution authority. It binds a structured intent object, Root and Operational Policies, typed evidence, and a verifiable Decision Derivation through an Execution Release Contract, with lifecycle rules for Redemption and Revocation. Evaluation included 34 static vectors, 15 lifecycle checks, and 100 trials of 32 concurrent Redemption attempts yielding exactly one winner per trial. The authors state these bounded results demonstrate executability of the specified subset, not production readiness or complete mediation.

arXiv cs.CR · 6d agoAI safety & security1

ToxicRAG: Compromising Retrieval-Augmented Generation Systems via Single-Shot Knowledge Poisoning Attacks

ToxicRAG shows a single narrative-form poisoned document can steer RAG answers, achieving 0.61-0.91 attack success rates across four LLMs.

The attack injects one document per target question written as a coherent knowledge-update narrative that acknowledges the previously accepted answer, introduces fabricated events that appear to invalidate it, and attributes the attacker-chosen answer to purported authorities. An optional answer-focused self-validation loop revises candidates when a surrogate LLM fails to reproduce the target answer. Across 100 target questions each from Natural Questions, HotpotQA, and MS-MARCO, with four victim LLMs and four dense retrievers, ToxicRAG achieves attack success rates of 0.61-0.91 and matches or exceeds the strongest baseline by 0 to 11 percentage points.

arXiv cs.CR · 7d agoAI safety & security1

BenchShield: Formal Model-Backed Instrumentation for Reward Integrity in LLM-Agent Evaluation Infrastructure

BenchShield uses lifecycle-model-backed instrumentation to detect reward hacking in LLM-agent benchmarks, lifting full-chain recall to 77-100% at up to 65% lower cost.

The framework grounds reward-hacking detection in a finite lifecycle model of an evaluation's reward-relevant events, combining a static phase-aware taint analysis with runtime infrastructure-side evidence attribution. Evaluation used a human-labeled corpus of 456 adjudicated trajectories drawn from more than 31,000 public agent runs across three benchmarks. BenchShield improves full-chain recall from 23-94% to 77-100% and same-vector coverage from 16-56% to 43-78%, cuts per-task cost by up to 65%, and achieves 96% accuracy detecting reward hacking at runtime.

arXiv cs.CR · 7d agoAI safety & security1

The Missing Boundary: How Autonomous Agents Lose Control

Tencent research finds agents lose control in 55-62% of trajectories when degraded control boundaries coincide with executable unsafe opportunities across five models and 16 domains.

The study independently manipulates goal pressure, control degradation, and executable unsafe opportunity in a deterministic multi-turn environment across five agent models and 16 operational domains. Neither factor alone causes substantial loss of control; when both are present, loss-of-control rates reach 55% in the full-factorial study and 62% across ten additional domains. Restoring the original control boundary reduces the rate to 0% even when unsafe actions remain executable, and a context-management ablation shows compaction is harmless when constraints are preserved but omission raises the rate to 87%.

arXiv cs.CR · 7d agoAI safety & security2

Active Adaptation, Not Static Defense: Temporal Dynamics of Preventative Steering in Adversarial Fine-Tuning

Researchers analyze why Preventative Steering protects LLMs against malicious fine-tuning, finding active adaptation drives protection, and propose Progressive Intensity Scheduling.

The paper studies Preventative Steering, a training-time defense that injects undesirable-trait persona vectors during adversarial fine-tuning and removes them at evaluation time. Temporal analysis shows protection emerges from an early compensatory adaptation phase followed by a steady-state phase, with attention output projections acting as the dominant residual-write route for defensive updates. Intervention Delta Preservation experiments show that preserving or reinjecting weight offsets fails to maintain protection, indicating reliance on active adaptation rather than a static defense. The proposed Progressive Intensity Scheduling improves safety robustness on Qwen2.5 and Gemma-3 while reducing harmful trait expression.

arXiv cs.CR · 7d agoAI safety & security1

Understanding the Security Boundary of Obfuscation-based On-Device LLM Protection

Researchers formalize obfuscation primitives for TEE-protected on-device LLMs and show a Collapse attack breaks ArrowCloak, TSQP, and LoRO, then extend the boundary.

The paper formalizes obfuscation primitives for TEE-Shielded LLM Partition (TSLP) schemes that offload computationally intensive layers from a Trusted Execution Environment to external GPUs. A novel primitive-guided attack, Collapse, demonstrates a shared vulnerability in prominent published methods including ArrowCloak (Security'25), TSQP (S&P'25), and LoRO (NeurIPS'25). The authors then introduce two new obfuscation primitives and integrate them with existing constructs to formulate an extended security boundary (O_ext).

arXiv cs.CR · 7d agoAI safety & security

Beyond Training: A Feasibility Taxonomy for Inference-Time AI Governance

A study maps twenty inference-time AI governance mechanisms, finding commercial readiness only against cooperative deployers and no adequate defense versus state-level adversaries.

The paper develops a feasibility taxonomy of twenty inference-time AI governance mechanisms across monitoring, verification, and enforcement, each rated on a four-point readiness scale against a four-vendor evidence base. Fifteen of the twenty mechanisms have commercial technical substrates in production today, though governance-grade assurance and adversarial robustness vary substantially. Stress testing shows readiness holds only against a cooperative deployer and low-to-medium-capability user: no mechanism rates adequate against a high-capability state-level deployer, and fine-tuning removes model-internal enforcement components. A second-rater reliability check on readiness ratings returned a quadratic-weighted Cohen's kappa of 0.74.

arXiv cs.CR · 7d agoAI policy

Distributed and Private Textual Data Synthesis from Embeddings

Researchers propose a distributed differentially private text synthesis method combining DP summaries and secure protocols, removing the need for a trusted curator.

The paper presents a differential privacy and cryptography co-design for synthesizing textual training data without a trusted curator or tightly synchronized user participation. It releases a one-time DP summary in embedding space, identifying frequent semantic regions and their DP centroids to enable training-free offline text synthesis, with semantic support protection to avoid exposing rare user texts. A custom secure protocol enforces end-to-end DP guarantees over distributed user data. Across four benchmarks the approach achieves utility comparable to the state-of-the-art centralized DP synthesis method.

arXiv cs.CR · 7d agoAI research

What Makes Adversarial Examples Transfer Across Deepfake Detectors?

A controlled study of 60 deepfake detectors shows adversarial example transfer depends heavily on source-target compatibility, with source averaging understating vulnerability.

The study evaluates adversarial example transferability across 60 deepfake detectors spanning six backbones, two pretraining regimes, and five training-data configurations, using AutoAttack (AA) and Carlini-Wagner with Expectation over Transformation (CW-EOT). Transfer rises sharply when source and target share an exact backbone, architecture family, pretraining regime, or training data, with the dominant factor depending on the attack. Mean attack success rate is 7.21% under AA and 19.52% under CW-EOT for single sources, while a multi-source oracle reaches 64.48% after excluding exact matches, showing source averaging can substantially understate target vulnerability. The authors release 240,000 adversarially perturbed images, pairwise transfer results, detector configurations, and evaluation code.

arXiv cs.CR · 8d agoAI safety & security

CS-Guard: Benchmarking LLM Guardrails for Code Generation Security

CS-Guard benchmark shows LLM code-generation guardrails fail widely, with ~50% jailbreak ASR text-to-code and up to 100% code-to-code.

Researchers introduce CS-Guard, the first systematic benchmark for evaluating LLM guardrails for code generation security, covering text-to-code (1,000 malware-generation prompts, 7 jailbreak attacks, and a novel fictional scenario attack) and code-to-code (331 prompts across infilling, completion, and translation). They evaluate 9 guardrails across seven LLMs, finding average jailbreak attack success rates around 50% for text-to-code and 14.4% to nearly 100% for code-to-code. The fictional scenario attack achieves ASR close to 100% across many guardrails, raising reliability concerns for real-world software development. The benchmark and data are released publicly.

arXiv cs.CR · 8d agoAI safety & security1