Search: “misuse-detection”

OpenAI previews privacy-focused system for detecting AI misuse

OpenAI previews Private Safety Processing, detecting cross-interaction AI misuse without exposing customer prompt content, rolling out in September.

OpenAI previewed Private Safety Processing for API and enterprise customers, a system that analyzes patterns across related interactions to flag potential misuse while restricting OpenAI personnel from accessing underlying content. It builds on Zero Data Retention (ZDR) deployments, keeping content on customer-controlled infrastructure or a planned option using customer-controlled encryption keys, with prompts and responses not retained after processing. A technical white paper and rollout are planned for September; images flagged as potential CSAM are the retention exception.

Help Net Security·Aug 20, 00:00 UTC · 27d agoAI safety & security

An Empirical Analysis of ReDoS Vulnerabilities and ReDoS Detection Tools

Study of NVD data finds ReDoS vulnerabilities growing more prevalent and more likely to be exploited, while five detection tools disagree substantially.

The study compares five publicly available ReDoS detection tools and one regex correction tool across three datasets. An empirical analysis of all ReDoS vulnerabilities reported to the NVD finds they are becoming more prevalent and are much more likely to be exploited than non-ReDoS weaknesses. The detection tools exhibited substantial disagreement on whether a given regex is vulnerable.

arXiv cs.CR·Sep 9, 15:09 UTC · 7d agoResearch

Why Is SHAP Not a Reliable Standalone Explanation Framework for Malware Detection?

arXiv paper shows SHAP gives unreliable standalone explanations for malware detection, with attribution dilution and sign reversal in dependent PE feature spaces.

The paper argues SHAP's formal guarantees are insufficient for reliable malware interpretation because the explained feature-coalition game is fixed only by analyst choices, not by malware behavior in the data. In static Portable Executable feature spaces, dependent feature groups cause conditional SHAP to dilute credit by a factor of 1/m across redundant features, attribute importance to features the model never uses, and even reverse attribution signs; interventional SHAP queries off-manifold coalitions no real executable exhibits. Experiments on EMBER-2018, EMBER-2024, and BODMAS with fixed LightGBM and XGBoost detectors confirm these effects. The authors position SHAP as a limited diagnostic requiring explicit data-distribution statements and domain validation, not a standalone explanation framework.

arXiv cs.CR·Sep 4, 02:00 UTC · 12d agoResearch1

Windows Malware Detector as a Compound AI System: Trade-Offs in Accuracy, Efficiency, and Adversarial Robustness

Researchers model industrial Windows malware detection as a Compound AI System, quantifying trade-offs between detection accuracy, efficiency, and adversarial robustness under tiered attacker knowledge.

Industrial Windows malware detectors combine rule-based mechanisms with ML-based static and dynamic analyses, but their architectures are rarely publicly disclosed. The authors propose a methodology balancing detection performance, computational cost, and robustness, plus system-level threat models capturing whole-pipeline evasion rather than isolated components. Experiments on real-world data show training time reductions with marginal detection loss, while more knowledgeable attackers craft increasingly effective adversarial examples. The paper derives deployment guidelines from the observed efficiency-robustness trade-off.

arXiv cs.CR·Sep 8, 08:04 UTC · 8d agoResearch

Not All Relations Are Equal: Relation-Balanced and Calibrated Graph Learning for Provenance-Based Intrusion Detection

RECAL improves provenance-based APT detection with relation-balanced masked graph learning and calibrated errors, reaching 99.99% F1 on DARPA E3 datasets.

RECAL is an unsupervised framework for provenance-based intrusion detection that uses relation-balanced masked graph learning to capture rare interaction patterns, addressing statistical heterogeneity where relation frequencies differ by roughly 140,000X in CADETS. It calibrates reconstruction errors against each relation's benign error distribution to produce comparable anomaly evidence and reduce false alarms. On three DARPA E3 datasets, RECAL achieves F1 scores of 99.99%, 99.93%, and 99.99%, outperforming the best baseline on each dataset, and reduces mean false positive rate by approximately 105X, 4X, and 41X versus the lowest-FPR baseline.

arXiv cs.CR·Sep 15, 00:31 UTC · 1d agoResearch

Few-Shot Learning for Network Intrusion Detection: Methods, Datasets, and Performance

A PRISMA-style review of 21 few-shot learning studies for network intrusion detection finds meta-learning and CNNs dominant and evaluation inconsistently reported.

The systematic review screened 1,358 records from ACM Digital Library, IEEE Xplore, and Scopus covering 2022-2026 and retained 21 studies on few-shot learning for network intrusion detection. Meta-learning (8 studies) and convolutional neural networks (10) are the most common approaches, while CIC-IDS2017 and CSE-CIC-IDS2018 are the most frequently used datasets. Most evaluations use five or fewer samples per class, and missing parameters and source code limit reproducibility and direct comparison.

arXiv cs.CR·Sep 10, 09:07 UTC · 6d agoResearch1

Client-Side Probing of Deleted Ridge Statistics in Federated Unlearning

Researchers show malicious federated learning clients can probe broadcast classifiers to recover deleted samples, exposing exact label leakage on MNIST and CIFAR-10.

The paper shows that federated unlearning systems broadcasting updated linear classifiers leak compact additive training summaries to clients. A malicious client can submit known changes, identify server states from returned classifiers, and compare states around an isolated deletion to expose the deleted sample, class, or client summary, potentially enabling reinsertion. On MNIST and CIFAR-10, high-precision broadcasts allowed exact label recovery for every tested deletion, while lower precision sharply reduced fine-grained recovery.

arXiv cs.CR·Sep 3, 20:59 UTC · 12d agoResearch

Mapping out your unknown: A threat hunter’s guide to GitHub

Datadog Security Labs publishes a threat-hunting guide with audit-log queries to detect GitHub token theft, device code phishing, and source code exfiltration.

Datadog's threat-hunting guide covers GitHub audit log queries for detecting compromised accounts, stolen personal access tokens, and malicious OAuth app authorizations. Attackers typically obtain credentials through phishing, credential stuffing, leaked secrets, or device code phishing, then map private repositories, exfiltrate source code, and pivot into connected cloud and CI/CD environments. The guide maps detections to MITRE techniques like T1078 and T1528 and documents GitHub logging quirks affecting attribution, token metadata, and visibility fields.

Datadog Security Labs·00:00 UTC · 19h agoResearch in the wild

Harnessing LLMs for Automating BOLA Detection

Unit 42's BOLABuster methodology uses LLMs to automate detection of broken object-level authorization vulnerabilities, uncovering flaws in Grafana, Harbor, and Easy!Appointments.

Palo Alto Unit 42 details BOLABuster, a methodology combining large language models with heuristics to automate detection of broken object-level authorization (BOLA) flaws, which traditional fuzzing and static analysis struggle to find. The approach uses LLM reasoning to understand application logic, map endpoint dependency relationships, and generate and interpret test cases. It found CVE-2024-1313 in Grafana, CVE-2024-22278 in Harbor, and 15 CVEs in Easy!Appointments. The team is continuing to hunt for BOLAs in open-source and internal projects.

Palo Alto Unit 42·Aug 17, 12:34 UTC · Aug 17, 2026ResearchCVE-2024-1313CVE-2024-22278CVE-2023-3285+14 CVEs

Self-Verifying Anomaly Detection using Explainable AI for Cybersecurity of DER Networks

ExCYDER framework self-verifies anomaly detection alerts for DER power grids using LightGBM and SHAP, reaching over 98% detection accuracy.

The paper presents ExCYDER, an explainable AI anomaly detection framework for Distributed Energy Resource networks that combines LightGBM with SHAP to validate whether each model decision aligns with its feature-attribution evidence. On a realistic DNP3 dataset it achieved over 98% detection accuracy, 14.5 ms SHAP latency per alert, and confidence deviation within 5%. The self-verifying mechanism distinguishes coherent from inconsistent alerts, improving interpretability and auditability for DER-focused security operations centers.

arXiv cs.CR·Sep 11, 00:19 UTC · 5d agoResearch

A First-Principles Evaluation of Graph-Based Network Intrusion Detection Systems

GIDS-Eval framework reveals evaluation gaps in graph-based network intrusion detection; two crafted edges fully evade three detector-dataset pairs.

Researchers introduce GIDS-Eval, a framework decomposing graph-based network intrusion detection systems into six interchangeable stages to enable controlled comparisons. Surveying nine GIDS and reimplementing five, they find two crafted edges achieve full evasion against three of eight detector-dataset pairs, snapshot windows alone cause a mean 38.3% relative swing in average precision, and none of 18 replayed detector-dataset pairs can alert as events arrive. Their encoder-free GIDS-Lite control ranks first by AP on two of four datasets at up to 575x lower runtime.

arXiv cs.CR·Sep 10, 22:38 UTC · 5d agoResearch1

Root-Cause Attribution Is a Search Problem: Continual Search for Long-Horizon Agent Failures

Continual Search framework iteratively prompts LLM judges to keep searching agent execution logs, boosting long-horizon failure root-cause attribution accuracy.

The paper frames automated root-cause attribution (RCA) for long-horizon AI agent failures as a search problem, since relevant evidence is sparse and distributed across massive execution traces. The authors propose Continual Search, an iterative framework that nudges an LLM judge across successive turns to keep hunting unresolved diagnostic evidence instead of settling on an early plausible diagnosis. They introduce MegaRCA-Mix, a benchmark of 50 human-annotated failure trials on long-horizon, execution-heavy tasks. On MegaRCA-Mix, Continual Search improves GPT-5.5's F1 from 0.349 to 0.498 (over 40% gain), and lower-tier models can surpass higher-tier counterparts when search is effective.

Hugging Face daily papers·Sep 10, 20:00 UTC · 5d agoAI research1

How Attackers Abuse VSS, and How Huntress Detects It

Huntress details how attackers abuse Windows Volume Shadow Copies for ransomware recovery sabotage and NTDS.dit credential theft, plus detection logic.

Huntress explains that attackers abuse VSS in three ways: deleting shadow copies to inhibit recovery before ransomware detonation, creating shadow copies to extract the NTDS.dit Active Directory database for offline credential theft, and manipulating shadow copy configuration. Because backup agents and RMM tools routinely create and delete shadow copies, raw events are too noisy to alert on alone. Huntress detections instead correlate VSS activity with lateral movement and credential harvesting over a time window, such as an observed sequence of PsExec spawning SYSTEM shells on a domain controller, vssadmin create shadow, a blocked deletion attempt, and DNS reconnaissance against another host.

Huntress·Sep 14, 13:00 UTC · 2d agoResearch

Det-LIME: Detector-Aware, Multi-Instance Local Interpretable Model-Agnostic Explanations for Automated Marine Mammal Detection

Det-LIME extends LIME to multi-instance object detection explanations, improving attribution for harbor seal aerial surveys.

Det-LIME adapts LIME to object detection by combining per-detection weighting, a proximity kernel emphasizing box-adjacent regions, and IoU-based matching to track instances across perturbations. It was evaluated on aerial drone imagery for harbor seal detection plus a seabird case study, and compared against vanilla LIME, Stabilized LIME, Deterministic LIME, and gradient-based attribution. Using Attribution Ratio and Max Saliency Hit Rate metrics, it consistently improved multi-instance attribution and produced box-aligned explanations useful for debugging and data augmentation.

arXiv cs.AI / cs.LG / cs.CL·Sep 15, 17:20 UTC · 1d agoAI research

FreqSpaNet: Frequency and Spatial Learning of SFPF for Physical Layer Hardware Integrity Detection

FreqSpaNet learns spatio-frequency polarization fingerprints to detect unauthorized wireless hardware replacement, reaching 96.31% mean AUROC across seven replacement scenarios.

FreqSpaNet is a representation learning network for open-set hardware anomaly detection using spatio-frequency polarization fingerprints (SFPFs), which capture device-dependent responses across frequencies and directions. A frequency branch models local variations among neighboring frequencies while a geometry-aware spatial branch models directional relationships via angular information, combined through adaptive fusion and complementary pretraining. It achieves a mean AUROC of 96.31%, 9.05 points above the baseline, and is verified under seven hardware replacement scenarios.

arXiv cs.AI / cs.LG / cs.CL·Sep 15, 17:32 UTC · 1d agoAI research1

Machine Unlearning as Private Retroactive Algorithms

A cs.CR paper defines private retroactive algorithms, showing machine unlearning is a data-maintenance problem and giving DP constructions for linear statistics, clustering, histograms.

The paper argues that machine unlearning's requirement to emulate retraining from scratch carries no meaningful privacy semantics against adversaries observing sequences of releases, recasting it as a data-maintenance question addressed by retroactive algorithms. It defines private retroactive algorithms, combining retroactivity with differential privacy under continual observation. Constructions achieve privacy and retroactivity at no asymptotic cost over privacy alone for linear statistics, clustering, and histograms, alongside impossibility results.

arXiv cs.CR·Sep 4, 16:32 UTC · 12d agoResearch

How we monitor internal coding agents for misalignment

OpenAI published its approach for monitoring internal coding agents for misalignment behaviors, detailing oversight methodology rather than a specific incident.

OpenAI describes how it monitors its internal coding agents for signs of misalignment. The post focuses on detection methods and infrastructure for catching agent behaviors that deviate from intended goals. No concrete misalignment incident is reported; the piece is primarily about methodology.

Hacker News · AI·Sep 6, 16:37 UTC · 10d agoAI safety & securityHN 47↑ · 45 comments2

Poster: Towards Selecting Threat Appropriate Industrial Intrusion Detection Systems

Poster proposes counter-threat-intelligence-based detector selection for industrial control systems, showing IDS performance varies strongly by attack scenario.

The poster proposes a counter-threat intelligence sharing mechanism to select appropriate intrusion detection systems for the current threat situation in industrial control system environments. Attack-level performance evaluations of various IDSs show detection performance varies depending on the attack scenario. The results emphasize the benefit of dynamically matching detectors to evolving ICS threats.

arXiv cs.CR·Sep 11, 09:46 UTC · 5d agoResearch

Trends in Web Threats: Attackers Were More Active During Holiday Season

Unit 42 tracked 533,000 malicious landing URL incidents from October-December 2021, showing web threats peaked during the holiday shopping season.

Unit 42 detected 533,452 malicious landing URL incidents (120,753 unique) and 2,906,875 malicious host URL incidents (165,255 unique) from October through December 2021. Threat activity peaked in November, likely tied to Black Friday in the United States, United Kingdom, and Germany. Most malicious domains appeared to originate in the United States, followed by Russia and Germany. Personal sites, blogs, business sites, and shopping sites were the most common apparently benign entry points for attacks.

Palo Alto Unit 42·Aug 17, 13:06 UTC · Aug 17, 2026Research

I accidentally turned LLM memory into program analysis

A pwning.systems write-up describes how LLM memory functionality was unexpectedly repurposed into a program analysis technique.

A security research post on pwning.systems describes the author's discovery that LLM memory behavior effectively functioned as program analysis. The write-up is hosted on a security-focused blog and surfaced via a security-tagged link aggregator. Detailed technical content is not included in this feed, limiting verifiable specifics.

Lobsters · security·Aug 29, 03:16 UTC · 18d agoResearch1

The History Is the Detector: Executing CVE Patch History, End-to-End

BUGSTONE-E2E converts CVE patch history into executable LLM-guided detection rules, yielding 1,033 rules and 644 runtime-verified findings across 14 programs.

The BUGSTONE-E2E framework mines reusable detection rules from verified fixing commits, organized by CWE and language, and applies them through a funnel pipeline that escalates from Tree-sitter anchors and lightweight heuristics to LLM-based agent inspection, runtime verification, and scope-checked patch generation. Built from 19,325 high-severity CVEs published 2022-2026, it produced 1,033 detection rules spanning 56 CWE families, packaged into 172 skills. Applied across 14 programs, it generated runtime evidence for 644 findings, demonstrating that vulnerability history can drive reproducible detection and repair.

arXiv cs.CR·Sep 4, 16:40 UTC · 12d agoResearch

Reference-Based Bias Detection in LLMs via Relative Representations of Hidden States

Researchers propose auditing LLM bias via relative hidden-state representations, detecting bias increases with 3-50x less compute than output-level benchmarks.

The paper introduces a reference-based bias auditing method that compares hidden-state representations across model variants, such as before and after fine-tuning, by encoding sentences relative to a fixed anchor set. The resulting Representational Bias Shift (Delta-B) correlates with output-level bias change in 15 of 18 tested settings, reaching |r| = 0.84 under full fine-tuning across WildGuardMix, DecodingTrust, and ToxiGen benchmarks. Thresholding Delta-B detects checkpoints whose bias increased with ROC AUC between 0.65 and 0.99 and beats a SEAT-based baseline, while auditing a model in about three minutes with 3-50x less compute.

Hugging Face daily papers·Sep 8, 20:00 UTC · 7d agoAI research

ReCAST: Restoration-aware Cascaded Stage-wise Training for Obfuscated SMS Risk Classification

ReCAST distills a large teacher model's de-obfuscation ability into smaller models for robust classification of obfuscated Chinese SMS fraud messages.

The paper proposes ReCAST, a restoration-aware cascaded stage-wise training framework for classifying obfuscated Chinese SMS messages. It distills a large teacher model's de-obfuscation capability into a smaller deployable student by supervising obfuscated span detection, obfuscation type prediction, and text restoration, then uses the student for risk classification. On an internally constructed real-world Chinese SMS benchmark, ReCAST substantially outperforms directly trained baselines under obfuscation, targeting production latency and throughput constraints.

arXiv cs.CR·Sep 4, 08:36 UTC · 12d agoResearch

CounterPersona: Append-Only Defense Against Unauthorized Persona Skill Distillation

CounterPersona appends targeted counter-persona evidence after data collection to block AI systems from distilling an individual's behavioral patterns into reusable skills.

CounterPersona defends against unauthorized persona skill distillation, where attackers extract recurring patterns from collected personal data to replicate an individual's behavior. Unlike perturbation-based defenses that require modifying data before collection, it works in an append-only setting where historical records cannot be altered or revoked. It constructs targeted counter-persona evidence, packs compatible behavioral states into compact realization units, and strengthens them via rationale-guided consistency rewriting. Experiments show strong effectiveness across lexical, semantic, and LLM-based measures, remaining robust across different distillers.

arXiv cs.CR·Sep 14, 06:19 UTC · 2d agoAI safety & security

Do Input-Level Defenses Transfer to Observation-Level Attacks on VideoLLMs?

A systematic study shows input-level adversarial defenses provide inconsistent, often near-zero protection against observation-level attacks on video LLMs.

Researchers introduce DefTEval, a controlled framework testing eleven input-level defenses against five attack types across five video LLMs. Harmful-content detection rates are frequently near zero, and defenses fail even when attacks embed harmful signals in every sampled frame. Token compression discards localized safety features and modality fusion down-weights weakened visual signals, with defense outcomes dominated by model architecture rather than the defense method.

arXiv cs.CR·Sep 8, 06:59 UTC · 8d agoAI safety & security

SEMA-GUARD: Semantic and Graph-Based Vulnerability Detection in Assembly Code

SEMA-GUARD uses semantic analysis and graph neural networks to detect vulnerabilities in assembly code, achieving 85.1% accuracy on a Juliet-derived benchmark.

SEMA-GUARD is a framework that detects vulnerabilities in compiled programs when source code is unavailable, targeting malware, firmware, and embedded systems analysis. It enriches control flow graphs with low-level execution semantics including stack manipulations, memory accesses, and data flow. Evaluated on a Juliet Test Suite set compiled to assembly and split into function-level chunks, it achieves 85.1% accuracy and an F1 score of 0.801, outperforming purely statistical or structural approaches.

arXiv cs.CR·Sep 15, 14:35 UTC · 1d agoResearch1

PIDS-Bench: Evaluating Prompt-Injection Detectors Under Over-Defense, Obfuscation, and Distribution Shift

PIDS-Bench shows prompt-injection detectors scoring F1 above 0.98 still misclassify about one-third of external benign security-adjacent prompts, revealing provenance-sensitive over-defense.

PIDS-Bench is a frozen multi-axis benchmark that jointly evaluates prompt-injection detectors on attack detection and benign false-positive behavior at fixed thresholds, spanning in-distribution inputs, hard-benign prompts, obfuscated attacks, and domain/structural distribution shifts. It evaluates seven detectors plus a rule-based lower-bound reference. A detector exceeding F1 = 0.98 on held-out data still misclassifies roughly one-third of an externally-sourced benign security-adjacent subset, and no internal detector reaches F1 >= 0.95 with hard-benign FPR <= 0.10 on the stress distribution. Hard-negative augmentation nearly eliminates over-defense on curated stress inputs but leaves it intact on externally-sourced prompts, a pattern termed provenance-sensitive over-defense.

arXiv cs.CR·Sep 14, 04:28 UTC · 2d agoAI safety & security

OWASP's subtractive security project measures the attack paths you erased

Help Net Security·Aug 4, 00:00 UTC · Aug 4, 2026Policy & legal

CrossLink: Breaking Location Privacy by Linking Device Identifiers Across Protocols

Researchers present CrossLink, a passive tracing algorithm linking temporary device identifiers across LTE, WiFi, and BLE, reconstructing full traces for 83% of simulated users.

Smartphones emit temporary identifiers simultaneously over LTE, WiFi, and BLE, and per-protocol randomization defenses implicitly assume their protections compose across protocols. CrossLink is an uncertainty-aware tracing algorithm that stitches device identifiers across time, space, and protocols even when the adversary is fully passive and rotations are unsynchronized. In large-scale mobility simulation it reconstructs full traces for 83% of users versus 22% for the best single-protocol baseline. It remains effective under partial sniffer coverage, including strategically placed sniffers near LTE handover regions, mobile sniffers, and limited high-coverage subregions.

arXiv cs.CR·Sep 9, 09:51 UTC · 7d agoResearch

ZK-Trace: Certified Collusion Tracing with Zero-Knowledge Credentials for Federated GNSS Interference Monitoring

ZK-Trace combines Tardos fingerprints and zero-knowledge credentials to trace leaked classifier copies in federated GNSS monitoring without leaker cooperation.

ZK-Trace addresses leakage of a proprietary classifier distributed to partly trusted stations in federated GNSS interference monitoring, combining public identity marks, recipient-specific Tardos fingerprints, and zero-knowledge credential verification to support offline tracing. The paper provides false-accusation and tracing-score bounds with an interval-arithmetic checker allocating a common budget across accusation and tamper decisions. In a simulated GNSS federation it isolates all 160 single-owner copies and traces 712 of 720 two-owner mixtures with a 0.001 false-naming budget, while feature marks survive feature matching in 20/20 runs at 4.8 percentage-point accuracy cost but are erased by function-only distillation.

arXiv cs.CR·Sep 8, 14:00 UTC · 8d agoResearch