Search: “evaluation”

98 stories in the last 30d

Searchkeyword smart Sortlatest hot importance relevance

DomainAll Cyber AIWindow24h 3d 7d 30d AllIn the wild Important only Show duplicates

CategoryAll AI research 226 AI safety & security 160 Research 98 Industry 86 Model release 51 AI industry 50 Tools 23 Vulnerability 19 Exploit / PoC 17 AI policy 15 AI tools & infra 14 Threat actor 14 Data breach 9 Phishing & fraud 9

SeverityAll critical 9 high 51 medium 79 low 5 info 670

Why Johnny Can't Encrypt: A Usability Evaluation of PGP 5.0 (1999)

Seminal 1999 USENIX study finds most novice users cannot correctly sign and encrypt email with PGP 5.0 in 90 minutes.

Whitten and Tygar's USENIX Security Symposium paper evaluates whether cryptography novices can use PGP 5.0 effectively, using cognitive walkthrough analysis and a laboratory user test. The majority of test participants failed to successfully sign and encrypt a message within 90 minutes, despite PGP 5.0 having a well-regarded graphical interface. The authors argue that security requires usability standards beyond those of general consumer software and propose domain-specific UI design principles for security. The paper is a foundational reference in usable security research.

Lobsters · securitySep 9, 06:47 UTC · 8d agoResearch

Evaluating the NIST Bugs Framework Against CWE as a Successor for Automated Vulnerability Classification

NIST Bugs Framework evaluation shows it is more structured and automation-friendly than CWE for automated vulnerability classification, with gaps in attribute guidance.

The paper evaluates NIST SP 800-231's Bugs Framework (BF) against CWE as a target for automated CVE classification using a systematically screened corpus of CVE-to-CWE research. An inter-rater study with 2 subject-matter experts mapping 13 CVEs showed strong agreement on BF's cause and operation axes but only fair agreement on the attribute axis. Automated classification was tested across two LLM deployments under different budgets, and findings support BF as more structured and automation-friendly than CWE, though gaps include under-specified attribute guidance and missing fix commits for closed-source software.

arXiv cs.CRSep 14, 23:18 UTC · 2d agoResearch1

A First-Principles Evaluation of Graph-Based Network Intrusion Detection Systems

GIDS-Eval framework reveals evaluation gaps in graph-based network intrusion detection; two crafted edges fully evade three detector-dataset pairs.

Researchers introduce GIDS-Eval, a framework decomposing graph-based network intrusion detection systems into six interchangeable stages to enable controlled comparisons. Surveying nine GIDS and reimplementing five, they find two crafted edges achieve full evasion against three of eight detector-dataset pairs, snapshot windows alone cause a mean 38.3% relative swing in average precision, and none of 18 replayed detector-dataset pairs can alert as events arrive. Their encoder-free GIDS-Lite control ranks first by AP on two of four datasets at up to 575x lower runtime.

arXiv cs.CRSep 10, 22:38 UTC · 6d agoResearch1

RobResilience: Implementing and Evaluating a Resilience Framework for Cyber-Physical Embodied Systems

RobResilience implements a runtime resilience framework for robots in Webots/ROS2, evaluating tolerable disruption, degradation, and mitigation feasibility across eight attack scenarios.

The paper implements a formal resilience framework for embodied cyber-physical systems using a PR2 robot and ROS2 in a Webots simulation. At runtime it evaluates three predicates — tolerable disruption (δ), tolerable degradation (γ), and mitigation feasibility (μ) — over a compromised device set derived from IDS confidence scores, triggering mitigation strategies when resilience is lost. Eight attack scenarios systematically covering the full predicate state space confirm runtime behavior matches theoretical definitions. The work addresses 'graceful failure paralysis,' where autonomous systems cannot distinguish safe degraded states from catastrophic hazards during attacks.

arXiv cs.CRSep 15, 15:49 UTC · 1d agoResearch

MarkSec: Capability-Aware Evaluation of Adversarial Attacks Against LLM Watermarks

MarkSec unifies evaluation of stealing, scrubbing, and spoofing attacks against LLM watermarks with quality-constrained success metrics under shared reporting protocols.

MarkSec is a framework unifying analysis of stealing, scrubbing, and spoofing attacks against LLM watermarks under shared detector calibration, metric definitions, and reporting protocols. It introduces a quality-constrained attack success metric that jointly assesses attack effectiveness and text quality. Experiments across representative watermark families, attacks, LLMs, and datasets show that attacks strongest by watermark removal alone can fall behind general rewriting when success requires acceptable text quality, and stealing-based scrubbers often underperform the best general-scrubbing baselines.

arXiv cs.CRSep 15, 06:02 UTC · 2d agoResearch

You Get What You Sample: Evaluating Sampling Strategies for Web Security Measurements

Evaluation of 500k Tranco and 24.8M Common Crawl hosts shows Top-N domain sampling biases web security measurements; probability sampling yields unbiased estimates.

The study is the first comprehensive investigation of how sampling strategies affect web security measurement conclusions, comparing datasets and strategies across 500k Tranco domains and 24.8M Common Crawl hosts. It shows Top-N selection does not reflect the overall web distribution and may bias observed vulnerability rates, while probability-based strategies yield stable, unbiased prevalence and impact estimates. Hybrid sampling offers no advantage because its deterministic prefix consistently hurts accuracy, and the authors propose an adaptive probability-based strategy effective even when target prevalence is unknown.

arXiv cs.CRSep 10, 08:21 UTC · 7d agoResearch1

Towards Standardized Evaluation of GPU Memory Safety with GMSBench

GMSBench provides 149 CUDA tests covering spatial, temporal, and concurrency GPU memory errors, exposing detection gaps in Compute Sanitizer.

GMSBench is a GPU memory safety benchmark comprising 149 self-contained CUDA tests spanning spatial, temporal, and concurrency errors across different GPU memory spaces and execution scenarios. The authors evaluate NVIDIA's Compute Sanitizer across multiple GPU architectures using the suite, exposing gaps in its detection coverage. The benchmark offers a standardized foundation for comparative evaluation of GPU memory safety mechanisms.

arXiv cs.CRSep 8, 15:11 UTC · 8d agoResearch

ROSETTA: Efficient and Accurate Privacy-Preserving LLM Decoding via Hybrid CKKS/TFHE Evaluation

ROSETTA is a hybrid CKKS/TFHE homomorphic encryption framework for privacy-preserving LLM decoding, achieving up to 4.8x Softmax and 2.1x end-to-end speedups.

The paper proposes ROSETTA, a hybrid CKKS/TFHE fully homomorphic encryption framework for private inference on generative LLMs, targeting the nonlinear operations that dominate autoregressive decoding cost. It introduces an adaptive segmented lookup-table protocol based on TFHE and a scheme-aware operator-selection framework that assigns each nonlinear operator to CKKS or TFHE to minimize latency. Experiments show up to 4.8x Softmax speedup and 1.5-2.1x end-to-end decoding speedup over the state-of-the-art CacheMir framework.

arXiv cs.CRSep 15, 09:51 UTC · 1d agoResearch

A Cyber Range Evaluation of Autonomous Network Incident Response Agents

Cyber range evaluation shows reinforcement learning incident response agents defend emulated networks more efficiently than heuristic policies, depending heavily on adversary behavior.

The paper evaluates agents for automated network intrusion response in a cyber range designed for human operator training, featuring variable topology, red-team emulation, and simulated users. Alerts are generated by a SIEM platform and mapped to a data modeling language used by the agents, with reinforcement learning policies optimized to minimize combined defense and availability costs using a cyber attack simulator. Reinforcement learning agents defended the system more efficiently than heuristic policies, with performance highly dependent on the adversary policy and simulated user behavior.

arXiv cs.CRSep 15, 02:36 UTC · 2d agoResearch

Evaluating Practical Enumeration and Blocking Attacks on the Snowflake Circumvention System

Ethical measurements enumerated 21,000+ Snowflake proxy IPs across ~1,000 ASes; blocking top 1% of ASes disrupts 30% of Snowflakes.

Researchers tested Snowflake's assumptions that proxy IPs cannot be easily enumerated and that blocking them causes unacceptable collateral damage. Over 48 days of real-world measurements (May-June 2025), malicious-client-style enumeration collected over 21,000 unique proxy IPs across almost 1,000 autonomous systems. Blocking the top 1% of ASes blocks more than 30% of observed Snowflakes while affecting about 2.5% of Tranco Top 1M domains, and the broker's load-aware matching leaks stable high-capacity proxies to attackers. Some proposed mitigations have already been integrated into Snowflake.

arXiv cs.CRSep 10, 22:01 UTC · 6d agoResearch1

Engineered Persuasion: Evaluating Personalized Pretexts in LLM-Generated Spear Phishing

A study of 180 US workers found each LLM phishing personalization level raised click-intention odds by 28%, but credibility depends on context fit.

The arXiv paper evaluates how personalized pretexts in LLM-generated spear phishing affect perceived credibility, using 180 US working adults across 1,436 evaluations of emails with four cumulative personalization levels, from workplace context to shared-project details. Convincingness rose 2.40 points per level in sensitivity analysis and click-intention odds increased 28% per level, while non-clickers shifted toward deleting rather than reporting. Qualitative coding showed details matching the recipient's role and routines supported credibility, whereas incorrect, vague, or channel-inappropriate details raised suspicion. The authors argue personalization effectiveness depends on pretext fit, with implications for workplace security training.

arXiv cs.CRSep 3, 19:21 UTC · 13d agoResearch

Bridging the First-Hour Gap: Evaluating AI Reliability and Benchmarking Deficiencies in Cyber Incident Response for Law Enforcement

Survey of playbooks, LLMs, RAG, and agentic AI for law-enforcement cyber first responders finds RAG most viable but benchmarks inadequate for legal requirements.

The paper surveys decision-support architectures (playbooks, LLMs, RAG frameworks, agentic AI) for frontline law enforcement during the first hour of a cyber incident, where volatile digital artifacts risk procedural errors and evidence attrition. RAG-based systems are identified as a relatively viable intermediate solution, though prompt sensitivity and confident hallucinations in legal contexts pose major risks. The authors find current cybersecurity benchmarks insufficient for law enforcement safety and legal demands, and argue for a new benchmark focused on naive query robustness and evidence preservation.

arXiv cs.CRSep 11, 10:25 UTC · 5d agoResearch

What Breaks Local Watermarks? A Robustness Benchmark for Local Invisible Image Watermarking

First systematic robustness benchmark of five local invisible image watermarking methods across 55 transformations finds all are vulnerable, with inpainting and geometric misalignment completely breaking payload…

The paper presents the first systematic robustness benchmark for local invisible image watermarks, covering 55 image transformations across signal distortions, coordinate alignment changes, indirect local edits, and direct watermark edits. It evaluates five methods: MaskWM, WAM, OmniGuard, TrustMark, and PixelSeal, all supporting localization natively or with minimal adaptation. Results show every method is vulnerable to some transformation; MaskWM offers the strongest payload recovery and localization but the lowest clean-image quality, and synchronization further improves its recovery under geometric transformations. Geometric misalignment and generative local edits such as inpainting and outpainting can completely impair payload recovery, while signal distortions are often tolerated.

arXiv cs.CRSep 15, 08:30 UTC · 2d agoResearch

Fresh-Challenge VDF Attestations for Model-Relative Response Latency

Fresh-Challenge VDF Attestations bind verifiable delay functions to unpredictable public challenges, yielding succinct evidence of model-relative response latency.

The paper specifies FCLA, a protocol composition that binds a VDF to an unpredictable public challenge, a message, and independently auditable release/receipt records. Under explicit assumptions about VDF sequentiality and a calibrated bound on an adversary's sequential evaluation rate, an accepted transcript is inconsistent with post-challenge generation. A benchmark of the public reference implementation confirms the expected evaluation-versus-verification separation on one documented machine. The contribution is a protocol design analysis, not a new VDF construction.

arXiv cs.CRSep 11, 11:25 UTC · 5d agoResearch

PIA-Bench: Towards Automated Privacy Impact Assessment with Large Language Models

Researchers release PIA-Bench, the first open benchmark evaluating how accurately LLMs can automate privacy impact assessments using 73 curated federal PIAs.

PIA-Bench is the first open benchmark for evaluating large language models on real-world privacy impact assessments (PIAs). The authors audited 499 expert-authored PIAs published by US federal agencies and curated 73 structured PIAs comprising 451 privacy risk items and 831 mitigation items. Off-the-shelf LLMs were found to produce meaningful assessments while identifying clear avenues for improvement. The paper calls for domain-specific LLM agent workflows, accountable LLM infrastructure, and new quality standards for PIAs.

arXiv cs.CRSep 11, 08:16 UTC · 6d agoResearch1

Few-Shot Learning for Network Intrusion Detection: Methods, Datasets, and Performance

A PRISMA-style review of 21 few-shot learning studies for network intrusion detection finds meta-learning and CNNs dominant and evaluation inconsistently reported.

The systematic review screened 1,358 records from ACM Digital Library, IEEE Xplore, and Scopus covering 2022-2026 and retained 21 studies on few-shot learning for network intrusion detection. Meta-learning (8 studies) and convolutional neural networks (10) are the most common approaches, while CIC-IDS2017 and CSE-CIC-IDS2018 are the most frequently used datasets. Most evaluations use five or fewer samples per class, and missing parameters and source code limit reproducibility and direct comparison.

arXiv cs.CRSep 10, 09:07 UTC · 7d agoResearch1

1Password's AI patching benchmark is misleading

Trail of Bits reanalysis says 1Password's 26% AI clean-fix rate is misleading; 86% of eligible patches blocked exploits.

Trail of Bits critiques 1Password's FLAWED AI patching benchmark, arguing its 26% clean-fix headline mixes trials where agents were instructed to apply wrong fixes (22% of data) with trials that prohibited compiling or testing (36%). Restricting to reasonable conditions, 2,634 of 3,067 patches (86%) blocked the supplied exploit. Trail of Bits also reports 12.5% of 2,265 developer first fixes failed in its own 2024-2026 assessments, and released post-patch-validation and review-walkthrough agent skills.

Lobsters · securitySep 15, 20:03 UTC · 1d agoResearch1

Cross-Domain Inference for Human Localization: Applying Wi-Fi RSSI Data to CSI-Trained Models

Researchers show CSI-trained Wi-Fi models can localize people from RSSI data at ~80% confidence, enabling privacy attacks from ordinary IoT devices.

The paper investigates cross-domain inference, feeding RSSI data into an existing CSI-based Wi-Fi pose prediction model. RSSI is accessible on IoT devices without elevated OS permissions or specialized drivers, unlike CSI. Using an RSSI dataset synchronized with video ground truth, the model predicted human locations with approximately 80% confidence when movement was present. The results imply a wide range of commodity IoT devices could be used for privacy invasion in Wi-Fi-dense environments.

arXiv cs.CRSep 15, 13:56 UTC · 1d agoResearch

Unmasking Cloud Identities: From Behavioral Clustering to Automated Detection

Unit 42 clusters behavior of 40,000+ AWS identities from 125 cloud environments to map functional roles and enable lightweight SQL-based detection.

Palo Alto Unit 42 built an unsupervised behavioral clustering model using UMAP and HDBSCAN on AWS CloudTrail logs to map cloud identities to functional roles such as administrators, backup services, security tooling and DevOps. The study analyzed over 40,000 identities across 125 cloud environments over two months. The researchers show that heuristics extracted from the clustering map can be implemented in standard SQL, enabling role classification at scale without running a continuous ML pipeline. The methodology extends to audit logs from other cloud providers, SaaS and Kubernetes.

Palo Alto Unit 42Sep 14, 10:00 UTC · 2d agoResearch

Rare Not Random Using Token Efficiency for Secrets Scanning

Researcher proposes token efficiency (string length divided by BPE token count) as a better post-regex filter than entropy for secrets scanning, validated on CredData.

The post explores whether Byte-Pair Encoding tokenization can replace Shannon entropy as the primary filter for candidate secrets captured by regex in tools like Gitleaks. It defines 'token efficiency' as string length divided by token count under the cl100k_base tokenizer; secret-like strings such as GitHub tokens tokenize into many small tokens and score low, while natural text scores high. Evaluating labeled secrets from the CredData dataset shows a usable separation, with roughly 2.5 suggested as a minimum cutoff versus Gitleaks' 3.5 entropy threshold. The technique is positioned as a post-regex filtering step rather than a standalone detector.

Lobsters · securitySep 12, 06:42 UTC · 5d agoResearch

PrivEscalate: Measuring and Augmenting the Threat of LLM-Automated Linux Privilege Escalation

Researchers release PrivEscalate, a 531-scenario benchmark showing LLM agents' Linux privilege-escalation success varies by vulnerability class, plus PrivEscAgent, a domain-specialized agent that boosts success.

The paper introduces PrivEscalate, an open-source benchmark of 531 Dockerized Linux privilege-escalation scenarios spanning 14 sub-categories, plus 329 parameterized variants measuring sensitivity to environmental distractors. Evaluating six LLMs across three agent architectures shows capability is heterogeneous across vulnerability classes, sensitive to perturbation, and architecture-dependent. The authors also present PrivEscAgent, a wrapper adding deterministic enumeration, category matching, and step planning that outperforms prior privesc-agent baselines without modifying the underlying LLM. The benchmark is released to support LLM agent evaluation, defensive tool validation, and red-team training.

arXiv cs.CRSep 8, 17:34 UTC · 8d agoResearch1

VEX-Bench: Benchmarking LLM Agents for Assessing Exploitability of Software Supply Chain Vulnerabilities

Introduces VEX-Bench, 75 expert-labeled real-world cases testing whether LLM agents can assess supply chain vulnerability exploitability; frontier models reach about 80% F1.

VEX-Bench is the first benchmark evaluating LLM agents on assessing whether upstream dependency vulnerabilities are exploitable in downstream projects, with 75 real-world expert-labeled cases across Python, Java, and Go mined from GitHub. Nine models across three agent harnesses were evaluated; GPT-5.5 and Claude Opus 4.6 reach approximately 80% F1 on binary vulnerability-status classification, but only GPT-5.5 surpasses 70% macro-F1 on fine-grained justification classification. The gap highlights the difficulty of moving beyond binary exploitability calls to explaining exploitability reasons, unlike prior benchmarks targeting zero-day settings.

arXiv cs.CRSep 7, 22:52 UTC · 9d agoResearch1

Forgery of C2PA on a Pixel 10

Researcher forged a Google Pixel 10 C2PA content credential with genuine signatures, showing root-level attackers can fake photo provenance.

A Hacker Factor blog post demonstrates an AI-generated 'unicorn glitter milk' news photo carrying a valid, cryptographically signed C2PA manifest traceable to Google's Pixel camera certificate chain, passing validation in Adobe Inspect and the CAI Verify tool with a verified timestamp. The author, working with UMBC's PASAWG working group, reported to Google and C2PA in November 2025 that root access on a Pixel device could sign arbitrary images as camera captures; after 90 days without resolution, details were published. The finding undermines C2PA Assurance Level 2 claims made for Pixel 10 Content Credentials.

Lobsters · securitySep 16, 13:24 UTC · 20h agoResearch

Illusion of Depth: Revealing Hidden Stereo Vision Vulnerabilities in Depth Estimation

Simple repeating patterns let attackers shift stereo-camera depth estimates by up to 20 meters, triggering emergency braking in autonomous driving frameworks at 40 km/h.

The paper reveals an intrinsic vulnerability in stereo cameras stemming from pixel sampling and calibration processes, letting attackers finely control estimated depth of real obstacles using simple repeating patterns without adversarial ML techniques. The attack was evaluated against BM and SGBM stereo matching algorithms, deep learning models PSMNet, MoCha-Stereo, and UniMatch, the stereo-LiDAR fusion model SGM-DDC, and commercial cameras ZED2 and Intel RealSense D435; on ZED2, obstacles can be displaced up to 20 meters farther or 12 meters closer. A 0.5-second attack triggered emergency braking in a popular autonomous driving framework, with feasibility confirmed at driving speeds up to 40 km/h using CARLA. State-of-the-art defenses proved ineffective, and the authors propose a similarity-score strategy to dynamically detect and suppress depth discrepancies.

arXiv cs.CRSep 14, 20:51 UTC · 2d agoResearch

The Model Proposes, the Code Disposes: A Pre-Registered Ablation of a Verifier-and-Acceptance Stage in an LLM-Orchestrated Offensive-Security Agent

Pre-registered ablation finds a model verifier stage in an LLM offensive-security agent suppresses findings; removing it eliminated suppression with precision tradeoff.

The paper evaluates a verifier-and-acceptance stage in an LLM-orchestrated offensive-security agent via a pre-registered 20-run confirmatory ablation and a 2x2 factorial study with 40 runs on vulnerable lab targets. Removing the stage eliminated pre-report suppression (median 2 vs 0 findings, p = 0.00003) but reduced model-blinded shipped precision (0.471 vs 0.353, p = 0.0087). Suppression was attributed to the model verifier rather than deterministic acceptance rules, and an instrumented canary recorded zero external contacts in all 60 runs. The full design retained 93.8% of model-adjudicated true candidates but failed its pre-registered non-inferiority floor of 0.90.

arXiv cs.CRSep 14, 17:13 UTC · 2d agoResearch

Automating Attack Graph Construction for Agentic Pentesting. Towards Neuro-Symbolic Vulnerability Hunting

Semi-automated pipeline converts Trivy, Semgrep, Nmap output into MulVAL attack graphs for agentic pentesting, 53.7% mean vulnerability coverage in CyBench.

The paper presents a semi-automated pipeline that parses Trivy, Semgrep, and Nmap findings into MulVAL predicates and uses an LLM-assisted process to build domain-specific Datalog rules linking scanner evidence to attack techniques. MulVAL/XSB then performs symbolic inference to generate structured, auditable attack paths for agentic pentesting. Evaluated on 54 web CTF tasks from CyBench, every task produced at least one goal-reaching graph with 53.7% mean ground-truth vulnerability coverage, 51.9% full coverage, and an 83.9% noise-path rate. Median end-to-end runtime was 24.9 seconds, making the pipeline runtime-practical for agentic workflows.

arXiv cs.CRSep 14, 13:13 UTC · 2d agoResearch

An Open-Source End-to-End FHE Implementation for Privacy-Preserving Llama 3 8B Inference

Odin runs Llama-3-8B fully homomorphic encrypted inference on a single H100 in 366 seconds, a 4.51x speedup over THOR.

Odin is an open-source end-to-end GPU CKKS implementation for privacy-preserving Llama-3-8B inference that co-designs ciphertext packing with model execution. A feature-major cross-layer layout unifies residual connections and layer interfaces, while transient intra-operator layouts serve linear projections and attention, avoiding intermediate repacking of QK^T softmax outputs. Minimax polynomial approximation with input-range control reduces polynomial degree and multiplicative depth for nonlinear ops. With 128-token input, Odin evaluates all 32 Transformer layers on one NVIDIA H100 80 GB in 366.4 s using 58.9 GiB peak memory, versus 1651.9 s for the THOR baseline, a 4.51x speedup.

arXiv cs.CRSep 11, 02:51 UTC · 6d agoResearch

Predicting Privacy Leakage from Weight Spectral Density

Study shows WeightWatcher spectral metrics like stable rank correlate with membership inference vulnerability, enabling cheaper ML privacy auditing.

The paper tests whether spectral metrics from the heavy-tailed self-regularisation framework can proxy membership inference attack (MIA) vulnerability without training expensive shadow models. On image and tabular classification tasks, stable rank correlates positively with overall MIA success, while Log alpha-Norm correlates negatively at the low false-positive regime. These correlations are stronger than those obtained from the generalisation gap, suggesting weight spectra capture leakage information overfitting measures miss. The authors propose spectral analysis as a scalable direction for privacy auditing.

arXiv cs.CRSep 10, 16:34 UTC · 6d agoResearch

Certifying Adversarial Robustness of Quantum Classifiers under Known-Readout Query Access

Framework certifies adversarial robustness of quantum classifiers using only measurement statistics and finite-shot outcomes, demonstrated on IBM Quantum hardware.

The paper introduces a measurement-only certification framework for adversarial robustness of quantum classifiers under known-readout query access, requiring no tomography, parameters, or gradients. It returns a lower bound ruling out untargeted errors within a radius and an attack-independent upper bound witnessing an adversarial state, both estimable with finite-sample guarantees. Evaluations show the lower bound tracks exact optima on tractable instances while the upper bound stays informative when standard attacks fail. The method was validated on IBM Quantum hardware using 40 executions of two 8-qubit quantum neural networks.

arXiv cs.CRSep 10, 14:45 UTC · 6d agoResearch

Towards Scalable and Cost-Efficient Vulnerability Detection: A Study on Automatic Query Generation

A study finds LLM-synthesized CodeQL queries improve average F1-score by 82% over baseline queries, offering scalable vulnerability detection versus direct LLM scanning.

Researchers conducted an empirical study evaluating whether LLMs can synthesize executable CodeQL queries from National Vulnerability Database vulnerability data. LLM-generated queries significantly enhanced baseline CodeQL suites, yielding an 82% improvement in average F1-score across a diverse set of real-world vulnerabilities. A cost-benefit analysis shows direct LLM-based scanning of entire repositories is often computationally and financially prohibitive, while LLM query synthesis offers a scalable and cost-effective alternative for large-scale vulnerability detection.

arXiv cs.CRSep 9, 16:31 UTC · 7d agoResearch1

Maverick: Private and Verifiable LLM Inference Made Practical via Matrix-Vector Multiplication Delegation

Maverick protocol delivers private and verifiable LLM inference via matrix-vector multiplication delegation, achieving up to 45x throughput gains over local inference on Qwen3-4B.

Maverick introduces an information-theoretically sound protocol for delegating matrix-vector multiplication with transparent preprocessing, efficient batch verification, and virtually no server overhead, combined with LPN-based pseudorandom masking for input privacy. It addresses privacy and correctness concerns when users delegate open-weight LLM inference to third-party providers. An end-to-end prototype evaluated on Qwen3-4B achieved throughput gains over local inference of up to 45x with precomputed privacy masks and 44x for verification-only workloads, with a CPU server using up to 128 threads.

arXiv cs.CRSep 9, 14:53 UTC · 7d agoResearch1

NERVE Attacks: Breaking AI-Powered Brain-Computer Interfaces

Researchers characterize NERVE, five attack dimensions against AI-powered brain-computer interfaces, and release the EEGle framework uncovering 17 attack instances.

NERVE is a systematic attack class spanning the BCI stack across five orthogonal dimensions: Neuro-mimetic Forgery, Evasion via Desynchronization, Replay-based Hijacking, Vein Tapping, and Embedded Backdoors. The accompanying EEGle framework enables AI-assisted, extensible BCI security analysis and helped uncover 17 novel neuro-specific attack instances, revealing a stealth-effectiveness spectrum unique to BCI backdoor design. The authors show generative AI lowers the barrier to entry for non-expert attackers and release EEGle to the community.

arXiv cs.CRSep 8, 16:20 UTC · 8d agoResearch

Windows Malware Detector as a Compound AI System: Trade-Offs in Accuracy, Efficiency, and Adversarial Robustness

Researchers model industrial Windows malware detection as a Compound AI System, quantifying trade-offs between detection accuracy, efficiency, and adversarial robustness under tiered attacker knowledge.

Industrial Windows malware detectors combine rule-based mechanisms with ML-based static and dynamic analyses, but their architectures are rarely publicly disclosed. The authors propose a methodology balancing detection performance, computational cost, and robustness, plus system-level threat models capturing whole-pipeline evasion rather than isolated components. Experiments on real-world data show training time reductions with marginal detection loss, while more knowledgeable attackers craft increasingly effective adversarial examples. The paper derives deployment guidelines from the observed efficiency-robustness trade-off.

arXiv cs.CRSep 8, 08:04 UTC · 9d agoResearch1

"Shut Up and Let Me Enjoy My Otome": Understanding and Measuring the Toxicity in Otome Game Communities

First large-scale study finds 22.20% of Weibo otome game posts toxic versus 3.71% on Reddit, with LLM detectors reaching 0.82 F1.

Researchers present the first large-scale measurement of toxicity in otome game communities, introducing OtomeSCAN, which collected and analyzed 620,045 posts from Weibo and Reddit over 18 months. They manually annotated 4,308 posts, identified eight target groups, and evaluated seven toxicity detectors, with their best LLM-based model reaching F1-scores of 0.82 on Weibo and 0.78 on Reddit. The study found 22.20% of Weibo posts were toxic versus 3.71% on Reddit, and toxicity rose to 37.09% within 72 hours during an external attack on Weibo. The authors also flagged 191 potential-coordination clusters, 64.40% of which targeted game developers.

arXiv cs.CRSep 7, 21:44 UTC · 9d agoResearch

Why The Vulnerability Backlog Is About To Get Worse

Recorded Future analysis says AI-driven vulnerability discovery and faster weaponization will grow the triage backlog while shrinking defenders' response windows.

Disclosed vulnerabilities rose from roughly 21,000 in 2021 to nearly 50,000 in 2025, while Recorded Future assessed only 446 as actively exploited in 2025. VulnCheck found nearly 29% of 2025 KEV entries were exploited on or before CVE publication. The authors argue AI-assisted discovery and automated exploit development will multiply credible reports, cut disclosure-to-exploit time toward minutes, and force re-evaluation of medium-severity flaws as exploit-chain components.

Recorded FutureAug 26, 00:00 UTC · 22d agoResearch

Zombie Card Attack Can Revive Expired Visa Cards for Contactless Payments

UMass Amherst researchers demonstrate Zombie Card, an NFC relay attack that revives expired Visa contactless cards for in-store purchases without breaking cryptography.

Researchers at the University of Massachusetts Amherst presented the Zombie Card attack at USENIX Security 2026, showing that Visa's Kernel 3 does not cryptographically bind the Application Expiration Date (tag 5F24) the terminal reads with the Track 2 expiry seen by the issuer. By positioning an NFC man-in-the-middle relay, an attacker can rewrite the terminal-facing expiration date of an expired card and complete contactless purchases, provided the account remains open under the same PAN and the bank does not independently re-check expiry. Testing across five major US banks found three distinct policies; Visa Kernel 3 accepted the modified date, while Mastercard, American Express, and Discover kernels declined modified transactions. Findings were disclosed to Visa and affected banks in May 2025, no CVE has been assigned, and no exploitation has been reported.

The Hacker NewsAug 20, 18:30 UTC · 27d agoResearch

Analog Pin Directionality as an Exfiltration Attack Surface in Mixed-Signal ICs

Researchers demonstrate exfiltration through nominally input-only analog pins in mixed-signal ICs, recovering data at 10 kbps on a 55nm PPG front-end.

The paper identifies a directionality-based attack class in analog/mixed-signal (AMS) ICs where data-dependent circuit-offset modulation converts a nominally input-only pin into an outbound information channel. Three host conditions enable the attack: a closed-loop amplifier, an exposed amplifier input, and sufficiently high impedance at that pin. Silicon validation on a photoplethysmography analog front-end in 55nm CMOS showed exfiltration at up to 10 kbps with error-free PRBS recovery, under 0.001% area overhead, and only 0.03 dB SNR reduction.

arXiv cs.CRSep 16, 17:39 UTC · 15h agoResearch

When Agents Look Like Beacons: NIDS Evasion by Model Context Protocol Traffic

Research shows Model Context Protocol agent traffic structurally resembles C2 beaconing and evades Suricata signatures and RITA behavioral scoring in testbeds.

An arXiv study demonstrates that Model Context Protocol (MCP) JSON-RPC traffic over Streamable HTTP mimics the polling patterns of C2 frameworks like Cobalt Strike and is not flagged as anomalous by standard enterprise defenses. In a Docker testbed with eleven traffic profiles across three TLS conditions, Suricata with the Emerging Threats Open ruleset produced near-zero alerts and RITA assigned a consistent 0.0 beacon score, regardless of jitter or TLS inspection. The authors propose an agent-native network indication standard using Agent-Native ALPN and out-of-band headers.

arXiv cs.CRSep 16, 17:27 UTC · 16h agoResearch1

Hamming Ideals and Grobner Bases for ISD-like Syndrome Decoding

Researchers combine Grobner bases with Information Set Decoding for syndrome decoding, testing feasibility against Classic McEliece NIST Category 1 parameters.

The paper proposes GBDecode, an ISD-like decoding algorithm that fixes only a subset of an information set and solves the resulting multivariate nonlinear systems via MultiSolve, which replaces one Grobner basis computation with many computations on simpler systems. Hamming weight constraints are reformulated using elementary symmetric functions and Lucas' identity factorizations to bound equation degree. Experiments on random binary linear codes use parameters matching the NIST Security Category 1 set of the Classic McEliece cryptosystem, assessing practical feasibility rather than breaking the scheme.

arXiv cs.CRSep 16, 16:05 UTC · 17h agoResearch

Echo: Learning-based Matching Decompilation using Trusted Back Translation

Researchers introduce Echo, a matching decompiler using trusted back-translation that roughly doubles exact-match rates and outperforms GPT-5.6 and Codex on Mirai.

Echo performs matching decompilation by using compilation as trusted feedback for iterative search: a domain-specific model generates candidate code and compilation configurations, which are recompiled, compared at assembly level, and repaired via rule-based rewriting, neural refinement, and reasoning-based refinement. On function-level benchmarks, Echo produces 2.43x more exact matches than the strongest baseline and the highest structural similarity to ground truth. On the Mirai malware binary, it matches 2.75x and 7.4x as many functions as GPT-5.6 and Codex, respectively.

arXiv cs.CRSep 16, 14:14 UTC · 19h agoResearch