Search: “safety-evaluation”

CTEM Technology Evaluation Scorecard

Horizon3.ai releases a scorecard for evaluating CTEM technologies on demonstrated exploitability and remediation evidence.

Horizon3.ai published a downloadable CTEM Technology Evaluation Scorecard for assessing security technologies across the six-stage Continuous Threat Exposure Management operating model, from discovering exposure through verifying risk removal. The scorecard uses a 0-3 scale based on repeatable evidence demonstrated in the evaluator's environment rather than stated feature claims, with emphasis on validating exploitability and verifying remediation. It is vendor marketing material aimed at security leaders and evaluation teams.

Horizon3.ai·16:56 UTC · 3h agoIndustry 2 sources

Safety for Whom? Boundary-Aware Self-Distillation for Controlled LLM Safety Refusal

A self-distillation safety framework tunes narrow-boundary refusals in Qwen3-8B, raising target-domain refusal to 84.75% while cutting over-refusal from 15.20% to 5.20%.

The paper formulates narrow-boundary safety, where deployments need refusals within specific topics rather than whole subjects, and proposes an offline self-generated framework with controlled topic generation, escalating retries, and harmful-benign boundary pairs. On political persuasion with Qwen3-8B, the method raised target-domain refusal from 9.47% to 84.75% and cut the mean unsafe-response rate across three broader benchmarks from 26.26% to 0.14%. Verified target-model responses reduced over-refusal from 15.20% to 5.20%, and boundary-pair data cut comply-side over-refusal on held-out pairs from 32.94% to 4.16%. Results show data composition controls the safety-usability trade-off and alignment should be evaluated on both sides of the refusal boundary.

Hugging Face daily papers·Sep 2, 20:00 UTC · 14d agoAI safety & security1

TIER: Threat Implicitness Benchmark for Evaluating LLM Safety Behaviors

TIER benchmark shows LLM safety behaviors shift gradually across threat implicitness levels, with jailbreaks exposing the largest robustness gaps.

The TIER benchmark evaluates LLM safety behaviors across four risk domains and four threat levels, from explicit harmful requests to sophisticated jailbreaks, using a six-label behavior scale and two independent LLM judges. Experiments on six open-weight LLMs show safety behaviors evolve gradually across threat levels rather than flipping from refusal to compliance. Models with similar Attack Success Rates can exhibit distinct response distributions, arguing for behavior-aware safety evaluation.

arXiv cs.CR·Sep 4, 13:13 UTC · 12d agoAI safety & security

Perturbation Probing: A New Diagnostic for the Fragility of LLM Safety

Unit 42 research shows LLM safety refusals concentrate in a thin neural layer, motivating external, multi-layered AI security controls.

Palo Alto Networks Unit 42 introduces Perturbation Probing, a diagnostic technique for measuring the fragility of LLM safety mechanisms. The research finds that safety refusal behavior is localized within a thin neural layer, implying small perturbations can undermine built-in refusals. The authors argue this motivates external, multi-layered security defenses on top of model-internal safety training.

Palo Alto Unit 42·Aug 28, 22:00 UTC · 18d agoAI safety & security

Emergency Department Revisit Quality Review Screening: Exploring Human Decision-Making and Artificial Intelligence Support

A retrospective study found GPT-4 over-flagged emergency department revisit cases while an LLM knowledge-graph screener achieved 83-100% positive predictive value.

In an exploratory retrospective study of 99 emergency department diagnosis pairs from a multihospital health system, clinicians and GPT-4 independently judged whether revisit pairs warranted further assessment. GPT-4 responses correlated poorly with clinicians, flagging 94% of pairs for follow-up, 4.4-13.3 times more than clinicians, though prompt engineering was minimal. An algorithm leveraging an LLM-populated knowledge graph (KGA) achieved 83-100% positive predictive value against at least one clinician rater, suggesting LLM-based screening could broaden revisit quality review without substantially increasing reviewer workload.

arXiv cs.AI / cs.LG / cs.CL·Sep 9, 16:37 UTC · 7d agoAI research1

Refuse without Refusal: A Structural Analysis of Safety-Tuning Responses for Reducing False Refusals in Language Models

Study shows training LLMs on refusal rationales instead of boilerplate refusal statements reduces false refusals while maintaining safety performance.

The paper decomposes safety-tuning responses into a boilerplate refusal statement and an explanatory rationale, finding that refusal statements push models to rely on superficial cues and misjudge benign queries as harmful. Training solely on rationales reduces false refusals while maintaining comparable safety performance, and the benefits carry over to in-context learning configurations and remain compatible with inference-time mitigations. The results argue for precisely curated, fine-grained safety supervision datasets when aligning LLMs.

Hugging Face daily papers·Sep 3, 20:00 UTC · 13d agoAI safety & security1

SAFIRE: Safety-Critical Benchmark for Fine-grained Fire and Smoke Understanding in Multimodal LLMs

SAFIRE, an 83K-image fire and smoke benchmark, shows open-source multimodal LLMs average only 61.9% accuracy on safety-critical fire reasoning.

SAFIRE is a large-scale benchmark for fire-smoke understanding in multimodal LLMs with 83K captioned images across 20 scenarios and 193K multiple-choice VQA questions spanning 10 evaluation dimensions from perception to higher-order reasoning. Annotations were built via a GPT-5.4-assisted multi-stage pipeline with MLLM majority voting. Ten open-source MLLMs (8B-38B) average 61.9% accuracy, exposing major gaps in safety-critical reasoning. Adapting vision encoders on 7% of the domain data raises fire-scene classification from 20.1% to 64.5%.

arXiv cs.AI / cs.LG / cs.CL·Sep 7, 17:59 UTC · 9d agoAI research

Evaluating the NIST Bugs Framework Against CWE as a Successor for Automated Vulnerability Classification

NIST Bugs Framework evaluation shows it is more structured and automation-friendly than CWE for automated vulnerability classification, with gaps in attribute guidance.

The paper evaluates NIST SP 800-231's Bugs Framework (BF) against CWE as a target for automated CVE classification using a systematically screened corpus of CVE-to-CWE research. An inter-rater study with 2 subject-matter experts mapping 13 CVEs showed strong agreement on BF's cause and operation axes but only fair agreement on the attribute axis. Automated classification was tested across two LLM deployments under different budgets, and findings support BF as more structured and automation-friendly than CWE, though gaps include under-specified attribute guidance and missing fix commits for closed-source software.

arXiv cs.CR·Sep 14, 23:18 UTC · 1d agoResearch1

When the prompt becomes the payload: A practical pen-testing guide for GenAI, LLM and RAG applications

CSO Online publishes a practical penetration-testing guide for GenAI, LLM, and RAG applications, covering prompt injection, retrieval poisoning, and tenant isolation testing.

The guide frames LLM applications as attack graphs spanning prompts, retrieval layers, vector stores, tools, identities, and downstream APIs, arguing that conventional web testing misses instruction-vs-data channel risks. It builds on OWASP prompt injection guidance (direct vs. indirect injection) and NIST's 2025 adversarial machine-learning taxonomy, noting that RAG and fine-tuning do not remove injection risk. Recommended practices include documenting trust transitions across components, using canaries and synthetic records to avoid test side effects, running multi-turn and obfuscated injection campaigns, and verifying chains from poisoned documents to observable state changes. It also details testing RAG pipelines via controlled document poisoning across metadata, OCR layers, and code comments, plus cross-tenant isolation checks on retrieved document IDs.

CSO Online·Sep 9, 10:00 UTC · 7d agoAI safety & security1

HHS Releases Updated Security Risk Assessment Tool

HHS OCR and ONC released version 3.7 of the Security Risk Assessment Tool for healthcare organizations.

The U.S. Department of Health and Human Services Office for Civil Rights (OCR) and the Office of the National Coordinator for Health IT (ONC) released version 3.7 of the Security Risk Assessment (SRA) Tool. The tool helps covered entities conduct HIPAA security risk assessments. ONC and OCR provided guidance on the updates.

DataBreaches.net·Sep 13, 14:49 UTC · 3d agoTools

Your Critical Vulnerabilities Might Not Be Your Biggest Risk

Autonomous penetration testing advocates prioritize exploitable attack paths over raw vulnerability severity for continuous security validation.

The article argues that scanner severity scores lack context: a critical flaw behind strong segmentation may be low priority, while a medium flaw on internet-facing systems can provide a foothold chained toward sensitive data. It positions autonomous penetration testing and attack path validation as the execution layer for continuous security validation, replacing point-in-time assessments. The piece is vendor-authored thought leadership rather than incident or vulnerability news.

The Hacker News·Sep 11, 11:30 UTC · 5d agoIndustry1

Quantifying IIoT Sensor Node Criticality by Fusing its Data Criticality and Security Vulnerability

Researchers propose a Dempster–Shafer framework fusing IIoT sensor data criticality with CVSS 4.0/3.1 vulnerability scores to rank node criticality.

The paper introduces a framework that evaluates Industrial IoT sensor node criticality by fusing data criticality and cybersecurity vulnerability scores using Dempster–Shafer (D-S) theory. It was validated on a dataset from red wine production and is claimed to generalize to other industrial settings with minimal modification. Results show criticality rankings derived from CVSS 4.0 scores differ significantly from those derived from CVSS 3.1, underscoring how vulnerability scoring methodology affects security prioritization.

arXiv cs.CR·Sep 9, 07:03 UTC · 7d agoResearch

Introducing Continuous Vulnerability Assessment: Real-Time Defense for the AI Threat Era

Wiz launched Continuous Vulnerability Assessment (CVA), alerting customers to exposure against newly published vulnerabilities in real time.

Wiz introduced Continuous Vulnerability Assessment, a capability that detects exposure to newly published vulnerabilities the moment they are disclosed. The company positions CVA as real-time defense for the AI threat era, reducing the window between disclosure and remediation decisions. The item is a vendor product announcement with no incident details.

Wiz Blog·Sep 1, 10:54 UTC · 15d agoTools

Bag of Tricks or Bag of Myths? Reducing Modeling Complexity with Task Knowledge in Explainable Suicide Risk Assessment

Researchers audit 31 NLP techniques for clinician-annotated suicide risk prediction, finding only 5 of 31 comparisons yield reliable gains.

A study of 1,635 clinician-annotated social media posts ran roughly 300 controlled experiments across 7 methodological families, auditing techniques such as model scaling, synthetic data, ensembling, and threshold tuning under severe class imbalance. The proposed system reformulates risk factor prediction as entailment between posts and codebook definitions, using architecturally diverse ensembles with class-balanced training and deployment-consistent calibration. It scores 0.8203 for risk, 0.7953 for evidence, and 0.7045 macro-F1 for factors, ranking third among 53 teams.

arXiv cs.AI / cs.LG / cs.CL·Sep 7, 17:14 UTC · 9d agoAI research

CTEM Is Not About the Stages. It’s About the Outcome.

Horizon3 argues CTEM programs should measure continuously reduced exposure rather than mapping technologies to Gartner's five stages.

Horizon3 contends that Continuous Threat Exposure Management should be judged by one outcome: continuously reducing attacker-reachable exposure, not by mapping a technology to each of Gartner's five stages. The post argues validation and verification, not visibility or closed tickets, provide evidence that attack paths are actually broken. It describes a Discover, Validate, Prioritize, Remediate, Verify, Repeat motion as its operationalization of CTEM.

Horizon3.ai·Sep 2, 13:11 UTC · 14d agoIndustry

Toward an Empirical Probabilistic Risk Manifestation Model of Organizational Cybersecurity in SMEs

Empirical study of 22 SME security assessments builds a probabilistic risk model and shows assessments can be cut 24-45% while retaining most critical findings.

Researchers analyzed 281 validated security findings from 22 real-world SME cybersecurity assessments conducted over two years via a pro bono university clinic. They derived an empirical Risk Manifestation Model linking eight organizational security functions to two exposure conditions, five attack mechanisms, and six outcome categories, using probability propagation to identify dominant risk pathways. The dominant pathway runs from asset exposure to credential compromise to unauthorized access, stable under leave-one-organization-out analysis. Retaining six functions reduces assessment burden by 24% while preserving 97% of critical findings; five functions cut burden 45% while preserving 89% of critical findings.

arXiv cs.CR·Sep 14, 01:15 UTC · 2d agoResearch

Bad Likert Judge: A Novel Multi-Turn Technique to Jailbreak LLMs by Misusing Their Evaluation Capability

Unit 42 details the Bad Likert Judge multi-turn jailbreak that abuses LLMs' evaluation capability, raising attack success rates over 60% across six frontier models.

Palo Alto Networks Unit 42 describes the Bad Likert Judge technique, a multi-turn jailbreak that asks a target LLM to act as a Likert-scale judge scoring the harmfulness of example responses. The highest-rated example in each scale can carry harmful content, bypassing the model's internal guardrails. Testing across six state-of-the-art text-generation LLMs showed an average attack success rate increase of more than 60% versus plain attack prompts, with tested models anonymized. The technique targets edge cases rather than typical use, and the article positions the work as guidance for defenders on potential jailbreak risks.

Palo Alto Unit 42·Aug 17, 12:34 UTC · Aug 17, 2026AI safety & security

An Empirical Analysis of CodeQL False Positives and Query Refinements for Java Vulnerabilities

Study of 167 Java CVE instances finds CodeQL false positives follow recurring patterns; query refinements remove 81.8% of reviewed ones.

Researchers ran CodeQL's Java security query suite on 167 CVE instances from 110 projects, manually reviewing 500 sampled false-positive paths and building a five-category taxonomy led by Missed Path Constraint or Sanitization (36.6%), Benign Execution Context (29.4%), and Missing Trust Boundary Modeling (27.6%). Guided by the taxonomy, query-level refinements removed 81.8% of reviewed false positives and 15.8% of reported paths across the selected queries while retaining 7 of 8 true positives. To address generalization, agentic coding tools given the refinement patterns as templates adapted them to new projects successfully in 56% and 62% of tasks, versus 28% without guidance.

arXiv cs.CR·Sep 3, 22:48 UTC · 12d agoResearch1

An Empirical Security Analysis of Open-Source Software Used in Onboard Satellite Systems

Study of 126 onboard satellite OSS repositories finds 2,827 security findings, 72% medium severity or higher, dominated by memory safety and code quality weaknesses.

Researchers performed an empirical security analysis of 126 public repositories of open-source software used in onboard satellite systems using SBOM generation, software composition analysis, static application security testing, infrastructure-as-code analysis, and secret scanning. After cleaning and deduplication the pipeline produced 2,827 findings, with medium-severity findings accounting for 49% and 72% classified medium or higher. A CWE-based taxonomy mapped all findings to eight weakness families, with Memory Safety and Code Quality dominating, followed by Input Validation and Injection. Project-developed code accounted for 81.4% of findings, though external dependency code remained relevant; findings do not establish mission-specific exploitability.

arXiv cs.CR·Sep 14, 11:54 UTC · 2d agoResearch1

Safe Meta-Reinforcement Learning via Information Space Reachability

Safe meta-RL framework reasons about safety in information space, learning a safety value function used for safety filtering and constrained policy optimization.

The paper proposes safe meta-RL that reasons about safety in information space, capturing both physical state and the agent's belief over the underlying task. A safety value function measures the probability of avoiding unsafe regions indefinitely and satisfies a self-consistency condition and Bellman equation, making it learnable via meta-RL. The resulting algorithm uses the learned function for safety filtering and constrained policy optimization, with effectiveness demonstrated on meta-RL benchmarks.

arXiv cs.AI / cs.LG / cs.CL·Sep 14, 17:27 UTC · 2d agoAI research

The State of Cloud Risk 2026: Most Security Findings Aren’t Real Attacker Opportunities

Wiz Research's 2026 cloud risk report finds most high-severity findings lack a viable path to compromise, urging exposure-based prioritization.

Wiz Research telemetry underpins a State of Cloud Risk 2026 report arguing that the majority of high-severity security findings are not real attacker opportunities because they lack a path to compromise. The report pushes defenders to prioritize findings by exposure and exploitability rather than raw severity scores. No specific CVEs, victims, or incident counts are provided in the available text.

Wiz Blog·Aug 26, 15:01 UTC · 21d agoResearch

Understanding Operator Attitudes Toward AI-Supported Decision Making in Maritime Operations

A survey of maritime professionals finds positive but scenario-sensitive attitudes toward AI collision-avoidance assistants, urging design for calibrated reliance.

The study surveyed maritime stakeholders on attitudes toward AI-supported collision-avoidance assistants for Maritime Autonomous Surface Ships, measuring technology anxiety, trust in automation, and explanation quality with established questionnaires plus thematic analysis of open responses. Results show generally positive disposition, no clear age-related openness differences, stable trust across scenarios, and multidimensional, scenario-sensitive explanation ratings. Participants valued decision support and situation awareness but worried about AI reliability, over-reliance, and skill loss; the authors recommend designing for calibrated reliance with domain experts in the loop rather than maximizing automation or trust.

arXiv cs.AI / cs.LG / cs.CL·Sep 10, 16:57 UTC · 6d agoAI research1

OpenAI locks down Astra over potential critical cyber capabilities

Help Net Security·Aug 10, 00:00 UTC · Aug 10, 2026Vulnerability

OpenAI previews privacy-focused system for detecting AI misuse

OpenAI previews Private Safety Processing, detecting cross-interaction AI misuse without exposing customer prompt content, rolling out in September.

OpenAI previewed Private Safety Processing for API and enterprise customers, a system that analyzes patterns across related interactions to flag potential misuse while restricting OpenAI personnel from accessing underlying content. It builds on Zero Data Retention (ZDR) deployments, keeping content on customer-controlled infrastructure or a planned option using customer-controlled encryption keys, with prompts and responses not retained after processing. A technical white paper and rollout are planned for September; images flagged as potential CSAM are the retention exception.

Help Net Security·Aug 20, 00:00 UTC · 27d agoAI safety & security

Engineered Persuasion: Evaluating Personalized Pretexts in LLM-Generated Spear Phishing

A study of 180 US workers found each LLM phishing personalization level raised click-intention odds by 28%, but credibility depends on context fit.

The arXiv paper evaluates how personalized pretexts in LLM-generated spear phishing affect perceived credibility, using 180 US working adults across 1,436 evaluations of emails with four cumulative personalization levels, from workplace context to shared-project details. Convincingness rose 2.40 points per level in sensitivity analysis and click-intention odds increased 28% per level, while non-clickers shifted toward deleting rather than reporting. Qualitative coding showed details matching the recipient's role and routines supported credibility, whereas incorrect, vague, or channel-inappropriate details raised suspicion. The authors argue personalization effectiveness depends on pretext fit, with implications for workplace security training.

arXiv cs.CR·Sep 3, 19:21 UTC · 13d agoResearch

Retrofitting Code Using LLMs to Support Exceptional Behavior

EXCODER combines static/dynamic analysis with LLMs to retrofit exception-handling code, achieving 85.92% pass@1 with Qwen 2.5 Coder 32B on Java benchmarks.

The paper introduces the task of retrofitting existing code with Exception Related Code (throw statements, guarding conditions, try/catch blocks) so that given Exceptional Behavior Tests pass. EXCODER performs context engineering by integrating static and dynamic program analysis output with LLMs; it was evaluated on a benchmark built from 304 methods across 75 GitHub Java projects. Combined with Qwen 2.5 Coder 32B, EXCODER achieves pass@1, 5, and 10 rates of 85.92%, 86.18%, and 86.51%, roughly 13 percentage points over baseline, and manual inspection reveals remaining limitations.

arXiv cs.AI / cs.LG / cs.CL·Sep 9, 16:18 UTC · 7d agoAI research1

Benchmark Scores Are Pipeline-Dependent: A Reliability Audit of Cybersecurity LLM Benchmarks

Audit of eight cybersecurity LLM benchmarks shows evaluation pipeline choices can swing scores by over 80 points and reshuffle most model rankings.

Researchers modeled eight cybersecurity benchmarks as configurable measurement pipelines and audited 10 proprietary, open-weight, and cybersecurity-specialized LLMs. They identified 15 systematic failure modes and showed a single pipeline choice can change a model's score by more than 80 percentage points and alter rankings; semantically similar task pairs rank the same models differently. Under a standardized harness, nine of 10 models shifted at least three ranks on at least one benchmark, motivating pipeline-aware auditing for reliable model evaluation.

arXiv cs.CR·Sep 8, 14:01 UTC · 8d agoAI research1

Enterprise Applications Carry 4.31x More Critical and High Vulnerabilities

Sonatype reports enterprise applications contain 4.31x more critical and high-severity vulnerabilities as accelerated software creation drives rising vulnerability levels.

Sonatype's analysis found enterprise applications carry 4.31 times more critical and high-severity vulnerabilities. The report links rising vulnerability levels to accelerated enterprise software creation. The findings are part of Sonatype's software supply chain research coverage reported by Infosecurity Magazine.

Infosecurity Magazine·Aug 18, 15:00 UTC · 29d agoIndustry

Batten the Hatches: Cybersecurity with Military Mariners

Interviews with 20 U.S. Navy and Coast Guard mariners reveal informal, safety-oriented shipboard cyber risk models that may delay attribution and containment.

The study conducts semi-structured interviews with 20 military mariners from U.S. Navy and Coast Guard vessels to understand how service members recognize and respond to cyber risk aboard ships. Unique consequences of compromising military systems identified include weapon takeover and purposeful geopolitical escalation. Cybersecurity is organizationally abstract on ships, so mariners build cyber risk models from informal experience rather than formal instruction. A safety-oriented incident-response model creates resilience but may delay cyber attribution and containment.

arXiv cs.CR·Sep 11, 13:04 UTC · 5d agoResearch