ZeroHour

Search: “llm agents”

28 stories

VEX-Bench: Benchmarking LLM Agents for Assessing Exploitability of Software Supply Chain Vulnerabilities

Introduces VEX-Bench, 75 expert-labeled real-world cases testing whether LLM agents can assess supply chain vulnerability exploitability; frontier models reach about 80% F1.

VEX-Bench is the first benchmark evaluating LLM agents on assessing whether upstream dependency vulnerabilities are exploitable in downstream projects, with 75 real-world expert-labeled cases across Python, Java, and Go mined from GitHub. Nine models across three agent harnesses were evaluated; GPT-5.5 and Claude Opus 4.6 reach approximately 80% F1 on binary vulnerability-status classification, but only GPT-5.5 surpasses 70% macro-F1 on fine-grained justification classification. The gap highlights the difficulty of moving beyond binary exploitability calls to explaining exploitability reasons, unlike prior benchmarks targeting zero-day settings.

arXiv cs.CR · 9d agoResearch1

Plug 'n' Pray: Agentic LLM-based Detection of Potential Log File Exposures in Third-Party Content Management System Plugins

Agentic LLM analysis validates 79 log file exposures across 62 of the 300 most-installed WordPress plugins, covering 250M+ active installations.

Researchers built an agentic LLM-based framework combining static and dynamic analysis to automatically detect insecure log files created by WordPress plugins. Scanning the 300 most-installed plugins, which account for roughly 75% of all active installations in the official ecosystem, it produced 81 findings with 79 manually reproduced across 62 plugins. Insufficiently secured log files can disclose credentials and personal data and have led to website compromises. The authors derive a taxonomy of log path and protection patterns and best practices, finding multi-layered protection often absent.

arXiv cs.CR · 1d agoResearch

PrivEscalate: Measuring and Augmenting the Threat of LLM-Automated Linux Privilege Escalation

Researchers release PrivEscalate, a 531-scenario benchmark showing LLM agents' Linux privilege-escalation success varies by vulnerability class, plus PrivEscAgent, a domain-specialized agent that boosts success.

The paper introduces PrivEscalate, an open-source benchmark of 531 Dockerized Linux privilege-escalation scenarios spanning 14 sub-categories, plus 329 parameterized variants measuring sensitivity to environmental distractors. Evaluating six LLMs across three agent architectures shows capability is heterogeneous across vulnerability classes, sensitive to perturbation, and architecture-dependent. The authors also present PrivEscAgent, a wrapper adding deterministic enumeration, category matching, and step planning that outperforms prior privesc-agent baselines without modifying the underlying LLM. The benchmark is released to support LLM agent evaluation, defensive tool validation, and red-team training.

arXiv cs.CR · 8d agoResearch

LLM-Based Penetration Testing in the Presence of Honeypots

Studies honeypot-aware budget allocation for LLM attack agents, showing detector-guided policies let agents skip deception and compromise real hosts efficiently.

The paper formalizes LLM attacker behavior against honeypots as a budgeted decision process, where agents choose to continue or skip targets when honeypot suspicion arises. A detector-guided policy lets LLM agents allocate execution budget effectively across a mixed host pool in a controlled testbed. Findings show LLM-driven attackers can reason about heterogeneous artifacts and use honeypot suspicion to guide target selection, challenging traditional deception defenses that rely on realism and obscurity against human or script-driven attackers.

arXiv cs.CR · 9d agoResearch

BlueSTAR: Tiered Agentic Architecture for Autonomous Cyber Defense

BlueSTAR is a tiered agentic LLM architecture for autonomous cyber defense, validated on live enterprise IT/OT cyber ranges against seven attack chains.

Researchers present BlueSTAR, a tiered agentic architecture for autonomous cyber defense in enterprise IT/OT networks that transforms high-volume security telemetry into compact indicators of compromise. It pairs deterministic containment for known threats with LLM reasoning for attacks requiring contextual and cross-cycle analysis, and introduces a resilience metric jointly weighing attacker reach, mission-critical impact, and defensive disruption. Evaluation on two live cyber ranges with seven attack chains based on real-world intrusion techniques covered credential theft, repeated compromise, concurrent attackers, and attacks on physical processes.

arXiv cs.CR · 6d agoResearch

The Model Proposes, the Code Disposes: A Pre-Registered Ablation of a Verifier-and-Acceptance Stage in an LLM-Orchestrated Offensive-Security Agent

Pre-registered ablation finds a model verifier stage in an LLM offensive-security agent suppresses findings; removing it eliminated suppression with precision tradeoff.

The paper evaluates a verifier-and-acceptance stage in an LLM-orchestrated offensive-security agent via a pre-registered 20-run confirmatory ablation and a 2x2 factorial study with 40 runs on vulnerable lab targets. Removing the stage eliminated pre-report suppression (median 2 vs 0 findings, p = 0.00003) but reduced model-blinded shipped precision (0.471 vs 0.353, p = 0.0087). Suppression was attributed to the model verifier rather than deterministic acceptance rules, and an instrumented canary recorded zero external contacts in all 60 runs. The full design retained 93.8% of model-adjudicated true candidates but failed its pre-registered non-inferiority floor of 0.90.

arXiv cs.CR · 2d agoResearch

PIA-Bench: Towards Automated Privacy Impact Assessment with Large Language Models

Researchers release PIA-Bench, the first open benchmark evaluating how accurately LLMs can automate privacy impact assessments using 73 curated federal PIAs.

PIA-Bench is the first open benchmark for evaluating large language models on real-world privacy impact assessments (PIAs). The authors audited 499 expert-authored PIAs published by US federal agencies and curated 73 structured PIAs comprising 451 privacy risk items and 831 mitigation items. Off-the-shelf LLMs were found to produce meaningful assessments while identifying clear avenues for improvement. The paper calls for domain-specific LLM agent workflows, accountable LLM infrastructure, and new quality standards for PIAs.

arXiv cs.CR · 5d agoResearch1

The AI Malware Maturity Gap

Recorded Future introduces AIM3, a five-level maturity model for AI malware, showing current attacker AI use is mostly AI-assisted rather than autonomous.

Recorded Future proposes AIM3, a five-level model defining AI malware from LLM-translated to LLM-embedded, spanning experimentation to fully autonomous agentic campaigns. Public examples remain early-stage: PROMPTFLUX uses Google Gemini to rewrite its VBScript dropper (Level 1), while Lamehug/PROMPTSTEAL, attributed to APT28, invokes the HuggingFace API to generate reconnaissance commands (Level 3). The authors argue most current AI malware augments existing tradecraft rather than enabling one-click autonomous attacks.

Recorded Future · 22d agoResearch

Automating Attack Graph Construction for Agentic Pentesting. Towards Neuro-Symbolic Vulnerability Hunting

Semi-automated pipeline converts Trivy, Semgrep, Nmap output into MulVAL attack graphs for agentic pentesting, 53.7% mean vulnerability coverage in CyBench.

The paper presents a semi-automated pipeline that parses Trivy, Semgrep, and Nmap findings into MulVAL predicates and uses an LLM-assisted process to build domain-specific Datalog rules linking scanner evidence to attack techniques. MulVAL/XSB then performs symbolic inference to generate structured, auditable attack paths for agentic pentesting. Evaluated on 54 web CTF tasks from CyBench, every task produced at least one goal-reaching graph with 53.7% mean ground-truth vulnerability coverage, 51.9% full coverage, and an 83.9% noise-path rate. Median end-to-end runtime was 24.9 seconds, making the pipeline runtime-practical for agentic workflows.

arXiv cs.CR · 2d agoResearch

Autonomy in Check: Governor-Mediated Adaptive Security at the Edge

Split-control architecture adds a deterministic governor to validate LLM and rule-based planner intents before eBPF enforcement at the edge.

Researchers propose a split-control edge security architecture in which an untrusted planner emits typed security intents that a deterministic governor checks against safety, resource, temporal-stability, and proportionality invariants. Admitted actions are bound to signed receipts and compiled into pre-installed eBPF map updates. A Raspberry Pi 5 prototype on a university 5G test network admitted, rejected, and bounded intents at microsecond cost.

arXiv cs.CR · 21h agoResearch

The History Is the Detector: Executing CVE Patch History, End-to-End

BUGSTONE-E2E converts CVE patch history into executable LLM-guided detection rules, yielding 1,033 rules and 644 runtime-verified findings across 14 programs.

The BUGSTONE-E2E framework mines reusable detection rules from verified fixing commits, organized by CWE and language, and applies them through a funnel pipeline that escalates from Tree-sitter anchors and lightweight heuristics to LLM-based agent inspection, runtime verification, and scope-checked patch generation. Built from 19,325 high-severity CVEs published 2022-2026, it produced 1,033 detection rules spanning 56 CWE families, packaged into 172 skills. Applied across 14 programs, it generated runtime evidence for 644 findings, demonstrating that vulnerability history can drive reproducible detection and repair.

arXiv cs.CR · 12d agoResearch

Building a Linux GPU Driver for the M4 Mac Mini in One Month

Two developers built a fully OpenGL ES 3.0 compliant Linux GPU driver for the M4 Mac Mini in one month via clean-room reverse engineering.

Niklas and the author reverse engineered Apple's AGX GPU firmware ABI and user-space components in about a month, a process that normally takes years, producing an OpenGL ES 3.0 conformant driver fast enough to run Minecraft at 200fps on an M4 Mac Mini. The work was done transparently using hypervisor traces without examining Apple binaries, following clean-room practices, and included a custom shader compiler, command stream builder, and a full Linux kernel driver for the firmware ABI. The A18 Pro firmware ABI proved significantly more complex than the M1's, with 1.5x as many structs and twice as many pointers. All experiments and provenance evidence were published in public agx-re repositories.

Risky Bulletin: Academics find source code overlaps between Geedge and China's Great Firewall

Academics linked Chinese vendor Geedge Networks' Tiangou Secure Gateway source code to one of the Great Firewall's three traffic filtering capabilities.

US researchers presenting at USENIX Security reconstructed Geedge Networks' Tiangou Secure Gateway firmware from over 100,000 leaked files, including Git repositories with commit history, and matched its filtering behavior to sections of China's Great Firewall. They found only 1 of 3 characterized DNS injectors matched Geedge code, noted the system relies on memory-unsafe C components and copied third-party code, and said its bugs could aid future circumvention tools. Geedge also exports censorship tools to Kazakhstan, Ethiopia, Pakistan, and Myanmar. The newsletter additionally rounds up multiple breaches.

Risky Business News · 27d agoResearch2

Cost-Aware Hierarchical Multi-Agent Ransomware Detection and Family Attribution

Researchers propose a hierarchical multi-agent system that cuts ransomware analysis cost by 44% while reaching 96.57% detection accuracy.

An arXiv paper (2609.04820) presents a Cost-Aware Hierarchical Multi-Agent System (HMAS) for adaptive ransomware detection and family attribution. Specialized agents run static analysis first, with dynamic and memory modalities invoked only when confidence is insufficient or specialists disagree; a Meta Orchestrator balances accuracy against computational cost via a cost model, and a locally deployed LLM verifies difficult cases. The system achieved 96.57% accuracy, 0.96 F1-score, and 0.99 ROC-AUC for binary detection, and 0.90 macro-F1 for multiclass family attribution. Average analysis cost dropped 43.97% versus exhaustive analysis, with 56.05% of cases resolved using static evidence alone.

arXiv cs.CR · 12d agoResearch