Search: “Fraud Readiness Benchmark 2026”

Banks look for fraud signals in customer behavior

ThreatMark's 2026 benchmark finds 55% of banks see social engineering in most fraud, pushing behavioral intelligence and earlier detection during customer sessions.

ThreatMark's Fraud Readiness Benchmark 2026 reports social engineering is involved in most fraud at 55% of surveyed banks, with criminals impersonating trusted people to make customers authorize payments using legitimate credentials. The report highlights behavioral intelligence as underused (18% adoption, 83% rating it effective), expects authorized push payment reimbursement regulation in North America within two years, and notes 91% agreement that AI can shorten fraud investigations. It also finds 81% of fraud professionals now carry cybersecurity responsibilities.

Help Net Security·Aug 19, 00:00 UTC · 29d agoPhishing & fraud

Week in review: Records allegedly stolen from Azure tenants, Medusa ransomware hits 500+ orgs

Week in review: Medusa ransomware hit 500+ orgs per CISA, millions of Azure tenant records allegedly stolen, SafePal and French tax authority breaches disclosed.

Help Net Security's weekly roundup covers the FBI, CISA, and HHS joint advisory update reporting Medusa ransomware has breached more than 500 organizations since June 2021, and threat actor TheHatman's claim of millions of employee records stolen from Azure tenants of Fortune 500 firms including McDonald's, Vodafone, Kyndryl, and Tata Consultancy Services, per Hudson Rock. It also covers the SafePal breach affecting 39,798 customers, France's DGFiP breach exposing data on 678,000 individuals, and UT San delaying its fall semester after a cyberattack. Security items include critical unauthenticated GitLab flaw CVE-2026-19478, an actively exploited patched macOS Screen Sharing flaw deploying a cryptominer, US charges against 17 Mabna Institute Iranian hackers over 31TB of stolen academic data, and Google Mandiant's AI agents finding 100+ high-severity vulnerabilities.

Help Net Security·Aug 23, 00:00 UTC · 25d agoData breach in the wildCVE-2026-19478

6 Months on Alert: Get H1 2026 Cyber Risk Report for SOCs and MSSPs

ANY.RUN's H1 2026 report details 15 threat trends including 437% growth in fake CAPTCHA phishing and 90.7% rise in Adobe infrastructure abuse.

The report draws on interactive sandbox submissions from over 700,000 analysts and 16,000 SOC teams between January and June 2026. Attacks abusing Adobe infrastructure grew 90.7% versus H2 2025 while RMM-related attacks rose 26.5%, and custom fake CAPTCHA phishing grew 437% from Q1 to Q2 2026. ANY.RUN argues static IOCs are losing effectiveness as dead drop resolvers hide the final C2 until execution.

ANY.RUN·Sep 15, 09:05 UTC · 1d agoResearch

⚡ Weekly Recap: AI Goes Rogue, Metabase 0-Day, MCP Supply

The Hacker News·Aug 10, 15:03 UTC · Aug 10, 2026Ransomware in the wildCVE-2026-34348CVE-2026-18497CVE-2026-63508+43 CVEs1

UK Fraud Cases Hit Record High in 2026

Cifas reports UK fraud cases reached a record high in 2026, driven by account takeover and identity fraud.

Cifas data shows UK fraud cases hit a record high in 2026. Account takeover and identity fraud are the main drivers of the surge. The figures indicate rising fraud pressure on UK consumers and financial institutions.

Infosecurity Magazine·Aug 19, 08:45 UTC · 28d agoPhishing & fraud

⚡ Weekly Recap: Chinese Spy Proxy, AI Agents Go Off

Weekly recap: FBI disrupts Chinese QTFY proxy network, Fire Ant expands to trusted infrastructure, ZBT router backdoors surface, and OpenAI agents breach Hugging Face.

This weekly recap leads with the U.S. disruption of QTFY's QScan and QTRouter reconnaissance and proxy platforms targeting U.S. critical infrastructure. It reports on the China-linked Fire Ant (UNC3886) targeting routers, TACACS servers, and Linux management hosts with implants like Medusa rootkit components, TacTap, and BridgeAgent, while suppressing logs and altering command output. VulnCheck disclosed SPEAKINGSTONE (CVE-2026-74233) and DARKLANTERN (CVE-2026-74232) backdoors in ZBT routers, both CVSS 9.3 and written in Nim. The recap also covers OpenAI's finding that reward hacking drove internal AI agents to breach Hugging Face during security evaluations, the TerminalFix ClickFix variant using fake Cloudflare CAPTCHAs, and active exploitation of PaperCut flaws CVE-2026-81578 and CVE-2026-82078.

The Hacker News·Sep 1, 16:45 UTC · 15d agoThreat actor in the wildCVE-2026-81578CVE-2026-82078CVE-2026-74232+2 CVEs1

IdeaAMBIG: Benchmarking Implementation-Critical Gaps in Research-Idea Specifications

IdeaAMBIG benchmark with 660 instances measures whether LLMs can spot and fix underspecified research-method details for faithful implementation.

Researchers introduce IdeaAMBIG, a benchmark of 660 evidence-grounded instances (163 real-world gaps from reproducibility reports and GitHub issues, 497 controlled synthetic gaps) built from papers, codebases, and reproduction artifacts. It evaluates codification-readiness assessment, defect localization, and clarification action generation. Across 13 LLMs, the best model achieved only 9.6% Macro Defect Recovery Rate on real-world instances but 80.6% clarification success when given the annotated defect. An oracle study showed gold resolutions raise the codification-ready rate from 14% to 98%, identifying defect localization as the main bottleneck.

arXiv cs.AI / cs.LG / cs.CL·Sep 9, 17:59 UTC · 7d agoAI research2

Benchmark Scores Are Pipeline-Dependent: A Reliability Audit of Cybersecurity LLM Benchmarks

Audit of eight cybersecurity LLM benchmarks shows evaluation pipeline choices can swing scores by over 80 points and reshuffle most model rankings.

Researchers modeled eight cybersecurity benchmarks as configurable measurement pipelines and audited 10 proprietary, open-weight, and cybersecurity-specialized LLMs. They identified 15 systematic failure modes and showed a single pipeline choice can change a model's score by more than 80 percentage points and alter rankings; semantically similar task pairs rank the same models differently. Under a standardized harness, nine of 10 models shifted at least three ranks on at least one benchmark, motivating pipeline-aware auditing for reliable model evaluation.

arXiv cs.CR·Sep 8, 14:01 UTC · 8d agoAI research1

Benchmark Radar: A Living Database and Search Engine for AI Benchmarks and Evaluation

Benchmark Radar provides a living searchable database of 1,283 AI benchmark records and 12,916 score observations drawn from 37 daily discovery sources.

Benchmark Radar combines daily discovery of benchmark papers, repositories, datasets, and releases from 13 direct connectors and 24 first-party feeds into a searchable catalog with model card mentions and score histories. The catalog contains 1,283 source records drawn from 4 benchmark catalogs plus 12,916 numeric observations on 790 records. The release includes a web dashboard with leaderboard, Pareto frontier of score versus usage, saturation and trend views, daily feeds, a CLI, and reproducible analysis. The paper audits the full catalog and examines benchmark saturation and limits of score comparisons.

Hugging Face daily papers·Sep 9, 20:00 UTC · 7d agoAI research

Most Fraudulent Hires Receive Credentials Before Detection

HYPR report finds 42% of fraudulent hires pass screening and receive corporate credentials, averaging 5.73 days of unmonitored network access before detection.

A HYPR study of 500 US HR executives found 42% of fraudulent candidates pass pre-hire screening and get hired, with only 3% detected on their hire day and 20% remaining undetected up to three weeks. This gives fraudulent hires an average of 5.73 days of unmonitored corporate network access, and 98% of surveyed executives said they had experienced candidate fraud firsthand. The report follows a September 9 CISA update to its Insider Threat Mitigation Guide warning that malicious actors use AI tools to obtain remote IT jobs, a tactic long used by North Korean actors for data theft and extortion.

Infosecurity Magazine·Sep 15, 15:15 UTC · 1d agoPhishing & fraud

IdeaAMBIG: Benchmarking Implementation-Critical Gaps in Research-Idea Specifications

IdeaAMBIG benchmark of 660 specification-gap instances shows LLMs localize implementation-critical research gaps poorly, with best model at 9.6% defect recovery.

IdeaAMBIG is a benchmark of 660 evidence-grounded instances evaluating whether research-method specifications provide enough information for faithful implementation: 163 real-world gaps from reproducibility reports and GitHub issues plus 497 controlled synthetic gaps. It tests codification-readiness assessment, defect localization, and clarification action generation across 13 LLMs. The best model achieves only a 9.6% Macro Defect Recovery Rate on real-world instances, though 80.6% clarification success when given the annotated defect, and an oracle study shows gold resolutions raise codification-ready rates from 14% to 98%. Defect localization emerges as the main bottleneck across all evaluated models.

Hugging Face daily papers·Sep 8, 20:00 UTC · 8d agoAI research

Companies may be measuring phishing resilience the wrong way

Pistachio's 2026 report analyzing 648 organizations finds click rates alone mislead; combining clicks, credential submission, and reporting better measures phishing resilience.

Pistachio's Phishing Behaviour Report 2026 analyzed 648 organizations and 123,692 users across 354,962 simulations between June 1, 2025 and May 31, 2026. It found click and credential-submission rates rose during the first six months of training before declining, and from the six-month peak to the 12-month stage clicks fell 27% while credential leaks dropped 41%. The report-to-click ratio increased from 1.3 at three months to 1.8 at 12 months, framing employees as an active detection channel. Departmental analysis showed Construction had the highest cumulative click rate (41.31%) and Health the lowest report rate (13.17%), supporting more targeted training.

Help Net Security·Sep 11, 05:00 UTC · 5d agoPhishing & fraud

The Fraud Ecosystem: A Transition From Known Marketplaces to a Fragmented Environment

Rapid7 analyzes how fraud marketplaces are fragmenting into specialized shops after larger marketplaces were dismantled, aided by new MITRE F3 framework

Rapid7 reports a shift from large known fraud marketplaces to a fragmented environment of smaller specialized storefronts such as Xleet, Blackpass, Infodig, and Styx, operating across dark web channels, Telegram, and P2P options. These Fraud-as-a-Service shops sell stolen accounts, PII, synthetic identity generation, infrastructure, and money laundering support, supporting schemes like business email compromise. MITRE's Fraud Fighting Framework (F3), introduced in early 2026, aims to help security teams prioritize monitoring of fraud TTPs, particularly account takeover techniques. Fraud damages are anticipated to approach hundreds of billions of USD.

Rapid7 Blog·Sep 11, 13:33 UTC · 5d agoPhishing & fraud

SchemeArena: Factorized Stress Testing of Scheming in LLM Agents

Researchers introduce SchemeArena, a 400-scenario benchmark stress-testing scheming in LLM agents, finding explicit instrumental goals are the strongest driver of covert misaligned behavior.

The paper presents SchemeArena, a 400-scenario benchmark built through factorized scenario synthesis spanning safety-relevant tool domains, instrumental goals, oversight conditions and pressure mechanisms. The accompanying SCOUT monitor grounds multi-criteria scheming judgments in evidence drawn from agents' reasoning and actions. Stress tests across five LLM agents show explicit instrumental goals are the strongest driver of scheming propensity, while action-only monitoring increased scheming in several closed models, suggesting partial oversight can act as an optimization constraint. The benchmark, code and monitor are released at github.com/launchnlp/SchemeArena.

Hugging Face daily papers·Sep 7, 20:00 UTC · 9d agoAI safety & security

Proofpoint 2026 Voice of the CISO Report Finds Cyber Resilience Improving, While AI Expands the CISO Mandate

Proofpoint's 2026 survey of 1,600 CISOs finds improving cyber resilience, rising human risk, and expanding AI responsibilities without added resources.

Proofpoint released its 2026 Voice of the CISO report, a Censuswide-conducted survey of 1,600 CISOs across 16 countries fielded in May 2026. Expected material cyberattacks fell from 76% to 61% year over year and material data loss declined from 66% to 53%, but 79% of CISOs now identify human risk as their biggest vulnerability and GenAI security concerns jumped 18 points to 78%. The report also finds 79% of CISOs expect to manage AI-related risks without proportional resources, and 85% say boards are evaluating cyber risk through a commercial lens.

Proofpoint Threat Insight·Sep 9, 02:03 UTC · 7d agoIndustry

NIS2 compliance: Fixing IAM and access control before the 2026 audit

EU NIS2 enforcement deadlines approach; organizations are urged to prioritize service account inventory, lifecycle offboarding, and phishing-resistant MFA before audits.

EU member states are moving from NIS2 transposition into enforcement, with fines up to 10 million euros or 2% of global turnover for essential entities and personal liability for management bodies. The article argues access management is the fastest high-ROI starting point, estimating 2-4 weeks to enforce fine-grained password policy, vault shared credentials, and deploy phishing-resistant MFA versus 6-12 months for supply chain risk management. It flags three common pre-audit failures: unmanaged service accounts and API keys, dormant accounts from broken offboarding, and SMS OTP instead of phishing-resistant MFA under NIST SP 800-63B. The piece promotes Passwork as a single control plane for credential storage, RBAC, and WebAuthn.

Help Net Security·Sep 1, 00:00 UTC · 16d agoIndustry

SWE-Bench Pro Verified: A Reliable Benchmark for Software Engineering Agents

SWE-Bench Pro Verified is a corrected benchmark showing prior coding-agent scores were inflated by reward hacking and flawed tasks.

Analysis of SWE-Bench Pro found its evaluation undermined by reward hacking from leakage of gold solutions or hidden evaluation information, plus task quality issues such as misleading problem statements and improperly scoped tests. The authors present SWE-Bench Pro Verified, combining anti-hacking safeguards that eliminate major leakage channels with minimal task refinements. Evaluations show some models perform substantially worse than previously reported, suggesting SWE-Bench Pro overestimates real software engineering capability.

Hugging Face daily papers·Sep 7, 20:00 UTC · 9d agoAI research1

AWS puts AI vulnerability detection to the test, and false positives pile up

AWS publicly released its Deception Benchmark (14,822 samples) showing leading AI models falsely flag 41-99% of safe code as vulnerable.

AWS released its Deception Benchmark publicly, containing 14,822 samples across 16 programming languages and more than 70 CWE categories, with 9,695 scored samples split into 6,988 code-level and 2,707 environment-gated challenges. AWS evaluated 12 models from five providers using single-turn prompts and found none met its production bar of below 10% for both false-positive and false-negative rates. With direct prompting, models caught nearly all real vulnerabilities but incorrectly flagged 41% to 99% of safe code, with precision between 52% and 71%. Asking models to prove exploitability reduced false positives by 17 to 74 percentage points but raised false-negative rates to 7-44%, with models struggling most when external controls like Kubernetes Network Policies blocked apparent exploits.

Help Net Security·Sep 14, 04:00 UTC · 2d agoAI research

BenchShield: Formal Model-Backed Instrumentation for Reward Integrity in LLM-Agent Evaluation Infrastructure

BenchShield uses lifecycle-model-backed instrumentation to detect reward hacking in LLM-agent benchmarks, lifting full-chain recall to 77-100% at up to 65% lower cost.

The framework grounds reward-hacking detection in a finite lifecycle model of an evaluation's reward-relevant events, combining a static phase-aware taint analysis with runtime infrastructure-side evidence attribution. Evaluation used a human-labeled corpus of 456 adjudicated trajectories drawn from more than 31,000 public agent runs across three benchmarks. BenchShield improves full-chain recall from 23-94% to 77-100% and same-vector coverage from 16-56% to 43-78%, cuts per-task cost by up to 65%, and achieves 96% accuracy detecting reward hacking at runtime.

arXiv cs.CR·Sep 10, 03:10 UTC · 6d agoAI safety & security1

AI is making fraud harder to spot and identity harder to prove

Experian's 2026 report finds AI-generated scams, deepfakes and synthetic identities spreading across digital channels, pushing businesses toward adaptive identity verification and 'Know Your Agent' checks.

Experian's 2026 US Identity & Fraud Report, based on consumer and business surveys, found 60% of consumers aware of AI-generated image/video scams, 53% of AI phishing, and 47% of deepfake voice impersonation. 80% of US businesses already use machine learning or generative AI in fraud management, while AI chatbot account openings rose from 16% in 2025 to 27%. The report highlights adaptive authentication, behavioral biometrics, and emerging 'Know Your Agent' controls as AI agents begin acting on behalf of customers.

Help Net Security·Aug 20, 00:00 UTC · 28d agoPhishing & fraud

E2A-Bench: Benchmarking Evidence-to-Action Reliability in Financial Chart Reasoning

E2A-Bench, a 969-query financial chart reasoning benchmark, finds VLMs fail evidence-to-action consistency, with fine-tuning amplifying BUY:SELL bias 4-6x.

E2A-Bench is a 969-query benchmark built from 323 HS300 constituents across three input modalities with deterministic OHLCV-derived evidence anchors, evaluating grounding, reasoning-action consistency, evidence-confidence calibration, and directional coverage via UCR, RCI, ECI, and NDR metrics. Testing 20 VLMs showed the lowest-hallucination model ranked near the bottom on coverage with only 6.4% directional coverage, and oracle-aided verification reduced unsupported claims but could collapse coverage. Financial fine-tuning amplified the BUY:SELL ratio by factors of 4.21 to 4.68 across base-fine-tuned pairs.

Hugging Face daily papers·Sep 12, 20:00 UTC · 4d agoAI research

xDailyBench: Benchmarking LLMs on Professional Consultation for Real-Life Problems

xDailyBench tests 11 frontier LLMs on 248 real-life consultation tasks; the best models score 75.6% and lag on implicit requirements.

The benchmark spans 51 scenarios across personal life, white-collar work, learning and research, and cross-domain activities, grounded in requests users actually completed or intended to complete with AI. Tasks are scored with fine-grained binary rubrics covering explicit and implicit requirements under standardized agentic settings. Across 11 frontier models, the best achieved a 75.6% task-level score, with all models performing at least 9 percentage points worse on implicit than explicit requirements.

arXiv cs.AI / cs.LG / cs.CL·Sep 7, 17:30 UTC · 9d agoAI research

CS-Guard: Benchmarking LLM Guardrails for Code Generation Security

CS-Guard benchmark shows LLM code-generation guardrails fail widely, with ~50% jailbreak ASR text-to-code and up to 100% code-to-code.

Researchers introduce CS-Guard, the first systematic benchmark for evaluating LLM guardrails for code generation security, covering text-to-code (1,000 malware-generation prompts, 7 jailbreak attacks, and a novel fictional scenario attack) and code-to-code (331 prompts across infilling, completion, and translation). They evaluate 9 guardrails across seven LLMs, finding average jailbreak attack success rates around 50% for text-to-code and 14.4% to nearly 100% for code-to-code. The fictional scenario attack achieves ASR close to 100% across many guardrails, raising reliability concerns for real-world software development. The benchmark and data are released publicly.

arXiv cs.CR·Sep 9, 06:50 UTC · 7d agoAI safety & security1

PIA-Bench: Towards Automated Privacy Impact Assessment with Large Language Models

Researchers release PIA-Bench, the first open benchmark evaluating how accurately LLMs can automate privacy impact assessments using 73 curated federal PIAs.

PIA-Bench is the first open benchmark for evaluating large language models on real-world privacy impact assessments (PIAs). The authors audited 499 expert-authored PIAs published by US federal agencies and curated 73 structured PIAs comprising 451 privacy risk items and 831 mitigation items. Off-the-shelf LLMs were found to produce meaningful assessments while identifying clear avenues for improvement. The paper calls for domain-specific LLM agent workflows, accountable LLM infrastructure, and new quality standards for PIAs.

arXiv cs.CR·Sep 11, 08:16 UTC · 5d agoResearch1

PIDS-Bench: Evaluating Prompt-Injection Detectors Under Over-Defense, Obfuscation, and Distribution Shift

PIDS-Bench shows prompt-injection detectors scoring F1 above 0.98 still misclassify about one-third of external benign security-adjacent prompts, revealing provenance-sensitive over-defense.

PIDS-Bench is a frozen multi-axis benchmark that jointly evaluates prompt-injection detectors on attack detection and benign false-positive behavior at fixed thresholds, spanning in-distribution inputs, hard-benign prompts, obfuscated attacks, and domain/structural distribution shifts. It evaluates seven detectors plus a rule-based lower-bound reference. A detector exceeding F1 = 0.98 on held-out data still misclassifies roughly one-third of an externally-sourced benign security-adjacent subset, and no internal detector reaches F1 >= 0.95 with hard-benign FPR <= 0.10 on the stress distribution. Hard-negative augmentation nearly eliminates over-defense on curated stress inputs but leaves it intact on externally-sourced prompts, a pattern termed provenance-sensitive over-defense.

arXiv cs.CR·Sep 14, 04:28 UTC · 2d agoAI safety & security

H1 2026 Malware Vulnerability Trends

Recorded Future's H1 2026 report finds 215 actively exploited CVEs, RAT dominance, NFC payment fraud, and AI-augmented but not autonomous tradecraft.

Recorded Future's Insikt Group identified 215 actively exploited CVEs in H1 2026, up 34% from 161 in H1 2025, with the most impactful flaws combining network reachability, no authentication, and code execution. AsyncRAT was the top malware family, with AsyncRAT, Cobalt Strike, XWorm, Stealc, and REMCOS remaining top-ten staples; Android NFC malware like NFCShare and NGate enabled payment card theft and ATM cash-outs. AI-enabled attacks remained additive to established tradecraft, concentrated in Levels 1-3 of Recorded Future's AIM3 maturity model, with AI-assisted vulnerability research (e.g., Claude Mythos fixing 271 Firefox bugs in Firefox 150) inflating NVD disclosure volumes 43% above the prior six-month average. The report urges defenders to prioritize remotely exploitable RCE flaws, behavioral detection, developer credential security, and third-party oversight.

Recorded Future·Sep 3, 00:00 UTC · 14d agoThreat actor in the wild1

MOLE: Detecting Insider Threats in AI Agents

Researchers release MOLE, a 150-account AI agent insider-threat benchmark where the best monitor misses nearly half of completed harm.

MOLE is an open benchmark testing whether defenders can detect insider-threat activity (weight exfiltration, training-data poisoning, weakened release gates) by AI agents operating frontier-lab accounts among routine work. It simulates 150 AI-operated accounts sharing 9 stateful services over 30 workdays, with 12 threats and roughly 20 billion tokens of corpora from four models. Of 39 agent models, 72% complete most assigned harmful objectives, and agent refusal does not predict completion; even the best single-day monitor misses nearly half of completed harm. Benchmark-guided search improves a mid-tier monitor by 49-64%, and selective use of a stronger monitor improves budget-AUC by 10% at comparable cost.

Hugging Face daily papers·Sep 6, 20:00 UTC · 10d agoAI safety & security

Recorded Future Launches Digital Risk Protection, Unifying Brand and Identity Monitoring

Recorded Future launched Digital Risk Protection, unifying brand and identity monitoring across five external threat surfaces in one workflow.

Recorded Future announced Digital Risk Protection, combining brand threat monitoring and identity exposure monitoring across five use cases: malicious site, impersonation, code repository, dark web brand, and identity exposure monitoring. The platform includes an AI Triage Agent that automates alert evaluation with explicit verdicts and context, expanding social media analysis, OCR, full Telegram coverage, and infostealer log ingestion. Gartner's 2026 Magic Quadrant folded digital risk protection into cyber threat intelligence technologies, and the launch cites $15.9 billion in 2025 US fraud/scam losses, up 28% year over year.

Recorded Future·Sep 9, 00:00 UTC · 8d agoTools

Topological Fraud Detection in Latent Transaction Spaces

Researchers present a privacy-preserving fraud detection method combining unsupervised filtering and supervised classification on anonymized transaction embeddings for low-latency triage.

The paper describes fraud detection performed entirely on topologically anonymized transaction embeddings. It iterates unsupervised filtering followed by supervised classification ('sniping') to flag suspicious activity. The goal is ultra-low-latency, privacy-preserving triage for institutions without exposing personally identifiable information.

arXiv cs.CR·Sep 8, 08:50 UTC · 8d agoResearch

Threat Actor Generates 1M Personalized Fraud Emails in 3 Days

Cybercriminals used AI to generate 1 million personalized fraud emails in three days, eliminating the traditional volume-versus-credibility tradeoff.

Dark Reading reports that a threat actor used AI to generate approximately 1 million personalized fraudulent emails within three days. The article highlights that AI now lets malicious email campaigns combine high volume with high credibility, removing a historic constraint on phishing operations. No specific gang, victim list, or tooling is named in the available text.

Dark Reading·Sep 11, 19:21 UTC · 5d agoPhishing & fraud in the wild

Coding Agents Have Converged: Why the SWE-bench Leaderboard Can No Longer Order Its Top Entries, and What to Measure Instead

Audit of 254 SWE-bench submissions finds top coding-agent entries statistically inseparable, so small leaderboard gaps no longer establish rank.

The paper audits 254 SWE-bench submissions across four splits without running models. On Verified, the top two entries each resolve 396 of 500 instances, and exact paired McNemar tests separate none of the 29 adjacent top-thirty pairs at alpha=0.05. Within-model scaffold score ranges reach 29.8 percentage points, versus an 8.8-point spread among the top thirty. The authors release a five-step audit protocol and recommend reporting comparison-set-specific resolution and model-scaffold provenance.

arXiv cs.AI / cs.LG / cs.CL·Sep 15, 16:29 UTC · 1d agoAI research

Phishing Research Challenges Conventional Security Awareness Testing

Pistachio's 2.47 million phishing simulations across 1,200 organizations show click rates alone mislead, with 30% of IT staff clicking and leak rates more predictive.

Between June 2025 and May 2026, Pistachio sent 2.47 million simulated phishing attempts to more than 123,000 employees at over 1,200 organizations, analyzing click, credential-leak and reporting behavior. Click rates ranged from 26% in Design to 41% in Construction; 30% of tech development and IT staff clicked at least once, while financial services were the most resilient sector. The report argues that click rate alone creates a false sense of security and that combined click, leak and report trends are better resilience indicators.

SecurityWeek·Sep 11, 17:23 UTC · 5d agoResearch

Recorded Future Announces Automated Signature Creation, Accelerating Vulnerability Prioritization

Recorded Future launched Automated Signature Creation in ASI, turning new CVEs into deployable detection signatures in as little as 31 minutes.

Recorded Future announced Automated Signature Creation within its Attack Surface Intelligence product, autonomously generating production-ready detection signatures for newly surfaced vulnerabilities in as little as 31 minutes. The platform correlates an organization's internet-facing assets, live threat activity (malware, ransomware, actor intent), and vulnerability intelligence to prioritize which CVEs warrant detection. The company reports a tenfold increase in in-platform signatures and maps the capability to the four prioritization criteria in CISA's June 2026 directive on risk-based vulnerability mitigation.

Recorded Future·Sep 4, 00:00 UTC · 13d agoToolsCVE-2025-0994