Search: “evaluation infrastructure”

OpenWAM: An Open, Modular Exploration Towards Systematic World-Action Model Pretraining

OpenWAM releases an open modular stack for world-action model pretraining, plus OpenWAM-alpha trained on about 6,400 hours of egocentric and robot data.

OpenWAM is an open research stack that factorizes World-Action Model pretraining into composable infrastructure, study, and model components with unified training, inference, and evaluation. Controlled experiments distill three principles on knowledge inheritance, world-action synergy, and out-of-domain generalization gains from embodied co-training. The resulting OpenWAM-alpha, pretrained on roughly 6,400 hours of egocentric human and robot data, achieves top-tier results across eight simulation benchmarks and real-robot tests spanning single-arm, bimanual, and dexterous embodiments. The full stack, including infrastructure, evaluation protocols, pretrained models, and data recipes, is released openly.

Hugging Face daily papers·Sep 6, 20:00 UTC · 9d agoAI research

An Evidence-First Multi-LLM Framework for Auditable Critical-Infrastructure Dependency Modeling

Evidence-first multi-LLM framework builds auditable critical-infrastructure dependency graphs while preserving provenance and unresolved cases.

The framework constructs Infrastructure Knowledge Bases and Infrastructure Dependency Graphs from heterogeneous infrastructure documentation using multiple open-weight LLMs that independently extract candidate entities and dependencies from normalized evidence. It separates evidence verification, ontology grounding, entity resolution, dependency alignment, validation, fusion, and human review, projecting the validated IKB deterministically into the IDG without new LLM-generated knowledge. Evaluation across nine infrastructure projects shows entity recovery achieves substantially higher recall than full dependency recovery, and cross-model overlap is much lower for dependencies than entities, indicating models often produce non-overlapping candidate assertions rather than stable consensus.

arXiv cs.CR·Sep 11, 02:33 UTC · 5d agoResearch

Unmasking Cloud Identities: From Behavioral Clustering to Automated Detection

Unit 42 clusters behavior of 40,000+ AWS identities from 125 cloud environments to map functional roles and enable lightweight SQL-based detection.

Palo Alto Unit 42 built an unsupervised behavioral clustering model using UMAP and HDBSCAN on AWS CloudTrail logs to map cloud identities to functional roles such as administrators, backup services, security tooling and DevOps. The study analyzed over 40,000 identities across 125 cloud environments over two months. The researchers show that heuristics extracted from the clustering map can be implemented in standard SQL, enabling role classification at scale without running a continuous ML pipeline. The methodology extends to audit logs from other cloud providers, SaaS and Kubernetes.

Palo Alto Unit 42·Sep 14, 10:00 UTC · 2d agoResearch

ActReview: Rebuttal-Guided Training Data and Rubric Rewards for Actionable Peer Review Generation

ActReview post-trains Qwen3-8B-Base on 40K rebuttal-derived instances with rubric rewards to generate actionable, grounded peer-review feedback, plus a 1,000-instance benchmark.

The framework builds ActReview-40K from real OpenReview review-rebuttal threads, aligning reviewer weaknesses with author responses and grounding feedback in localized paper evidence. Qwen3-8B-Base is post-trained with multi-task supervised fine-tuning followed by GRPO using candidate-aware, weakness-specific rubric rewards. Experiments show improved actionability and grounding over prior specialized review-generation models, supported by ActReview-Bench, a human-curated 1,000-instance evaluation set. Human evaluation confirms better revision usefulness while noting a remaining gap in technical accuracy.

arXiv cs.AI / cs.LG / cs.CL·Sep 8, 17:30 UTC · 7d agoAI research

Anthropic AI agent faked identities, phished real developers in UK government hacking test

The Record·Aug 5, 13:05 UTC · Aug 5, 2026Data breach1

ActReview: Rebuttal-Guided Training Data and Rubric Rewards for Actionable Peer Review Generation

ActReview post-trains Qwen3-8B-Base on OpenReview rebuttals to generate actionable peer-review feedback with grounded revision suggestions, benchmarked on 1,000 curated instances.

The paper defines Actionable Peer-review Generation as diagnostic claim generation plus revision suggestion generation and introduces ActReview, a rebuttal-guided post-training framework. From OpenReview review-rebuttal threads the authors build ActReview-40K, aligning reviewer weaknesses with author responses grounded in localized paper evidence, and post-train Qwen3-8B-Base with multi-task SFT followed by GRPO using weakness-specific rubric rewards. They also release ActReview-Bench, a human-curated 1,000-instance benchmark, on which ActReview outperforms prior specialized review-generation models on actionability and grounding while remaining competitive with strong prompt-based LLMs. Human evaluation confirms improved revision usefulness but identifies a remaining gap in technical accuracy.

Hugging Face daily papers·Sep 7, 20:00 UTC · 8d agoAI research

Surfshark Systems Targeted by Hackers

Surfshark discloses hackers accessed a misconfigured internal test server; no user data or VPN services affected.

Surfshark discovered on August 31 that a threat actor accessed an internal test server exposed to the internet through misconfiguration, obtaining some system binaries and internal configurations. Build-related credentials committed to code history were rotated, and an isolated content optimization VPS was also accessed, though no user data, encryption keys, or browsing activity were exposed. The company contained the system, rotated credentials, and announced an independent security audit.

SecurityWeek·Sep 11, 09:41 UTC · 5d agoData breach in the wild 2 sources

A First-Principles Evaluation of Graph-Based Network Intrusion Detection Systems

GIDS-Eval framework reveals evaluation gaps in graph-based network intrusion detection; two crafted edges fully evade three detector-dataset pairs.

Researchers introduce GIDS-Eval, a framework decomposing graph-based network intrusion detection systems into six interchangeable stages to enable controlled comparisons. Surveying nine GIDS and reimplementing five, they find two crafted edges achieve full evasion against three of eight detector-dataset pairs, snapshot windows alone cause a mean 38.3% relative swing in average precision, and none of 18 replayed detector-dataset pairs can alert as events arrive. Their encoder-free GIDS-Lite control ranks first by AP on two of four datasets at up to 575x lower runtime.

arXiv cs.CR·Sep 10, 22:38 UTC · 5d agoResearch1

Revolut Data Leak May Trace Back to Compromised Italian Government Accounts

Attackers using a compromised Italian government PEC account impersonated law enforcement to obtain data on ~680 Revolut customers.

Revolut confirmed its systems were not breached; fraudulent data requests came from a compromised PEC mailbox tied to the Prefecture of Reggio Calabria on the pec.interno.it domain. Per the Financial Times, roughly 680 customers had identity documents, addresses, banking information, verification selfies and cryptocurrency transaction histories exposed. Researcher Korra of Duel described a 'spray and pray' operation using hundreds of crypto transaction IDs and fraudulent European Investigation Orders. Threat actor IAmNotAVillain claims six months of access and 147 GB exfiltrated from Italian law-enforcement systems, though this remains unverified.

Security Affairs·13:09 UTC · 1h agoData breach in the wild

When the prompt becomes the payload: A practical pen-testing guide for GenAI, LLM and RAG applications

CSO Online publishes a practical penetration-testing guide for GenAI, LLM, and RAG applications, covering prompt injection, retrieval poisoning, and tenant isolation testing.

The guide frames LLM applications as attack graphs spanning prompts, retrieval layers, vector stores, tools, identities, and downstream APIs, arguing that conventional web testing misses instruction-vs-data channel risks. It builds on OWASP prompt injection guidance (direct vs. indirect injection) and NIST's 2025 adversarial machine-learning taxonomy, noting that RAG and fine-tuning do not remove injection risk. Recommended practices include documenting trust transitions across components, using canaries and synthetic records to avoid test side effects, running multi-turn and obfuscated injection campaigns, and verifying chains from poisoned documents to observable state changes. It also details testing RAG pipelines via controlled document poisoning across metadata, OCR layers, and code comments, plus cross-tenant isolation checks on retrieved document IDs.

CSO Online·Sep 9, 10:00 UTC · 7d agoAI safety & security1

HazardAuditor: From Executable Threats to Safer Computer-Use Agents

HazardAuditor trains execution-grounded guard models for computer-use agents, improving safety verdict accuracy by up to 16.5 points.

HazardAuditor runs heterogeneous agents (Claude Code, Codex, Hermes, OpenClaw) in controlled environments and normalizes their interactions into a canonical event representation for cross-framework supervision. It introduces Guard Policy Optimization (GuardPO), which converts deterministic safety outcomes into sequence-level advantages and normalizes rationale and verdict regions so the safety decision becomes the effective optimization unit. Across multiple benchmarks and heterogeneous computer-use systems, HazardAuditor improves accuracy by up to 16.5 percentage points over the strongest prior guard model. Code, models, and evaluation artifacts are being released.

Hugging Face daily papers·Sep 13, 20:00 UTC · 2d agoAI safety & security

The push to designate AI as the next critical infrastructure sector

Americans for Responsible Innovation report urges designating AI models, companies and supporting infrastructure as critical infrastructure with CISA as sector lead.

A report from the nonprofit Americans for Responsible Innovation calls for the federal government to declare the AI sector — including frontier model designs, model weights, datacenters, AI hardware and semiconductors — the 17th critical infrastructure sector, with CISA as the lead agency for sector cyberthreats. The authors argue AI is concentrated among a handful of foundation models and interdependent with other sectors, so a single attack on the AI stack could cascade widely, citing incidents like Iranian drone attacks on Amazon datacenters. Former DHS officials note the designation would unlock federal resources such as CDM access and threat intelligence, but warn that picking a lead agency could trigger a bureaucratic turf war with Commerce and Treasury.

CyberScoop·Aug 20, 13:31 UTC · 27d agoAI policy

Risky Bulletin: Academics find source code overlaps between Geedge and China's Great Firewall

Academics linked Chinese vendor Geedge Networks' Tiangou Secure Gateway source code to one of the Great Firewall's three traffic filtering capabilities.

US researchers presenting at USENIX Security reconstructed Geedge Networks' Tiangou Secure Gateway firmware from over 100,000 leaked files, including Git repositories with commit history, and matched its filtering behavior to sections of China's Great Firewall. They found only 1 of 3 characterized DNS injectors matched Geedge code, noted the system relies on memory-unsafe C components and copied third-party code, and said its bugs could aid future circumvention tools. Geedge also exports censorship tools to Kazakhstan, Ethiopia, Pakistan, and Myanmar. The newsletter additionally rounds up multiple breaches.

Risky Business News·Aug 21, 02:55 UTC · 26d agoResearch2

Technical Manual for a Toolkit for Measuring Contextual Individuation in Transformer Language Models

An open methodology toolkit measures whether transformer language models contextualize fixed word forms across domains using bridge forms and layer-wise silhouette analysis.

The manual documents an open toolkit built around 'bridge forms' - identical written words recurring across two or more subject domains with a different sense in each - to test whether transformer language models individuate word occurrences by context beyond the embedding layer. It covers declarative specification of bridge forms, Wikipedia corpus acquisition, occurrence localization, layer-wise representation extraction, domain-pairwise silhouette measurement, and visualization, justifying each choice against failure modes such as sense contamination and subword-tokenization misalignment. It is a methodological and implementation reference and reports no empirical results.

arXiv cs.AI / cs.LG / cs.CL·Sep 4, 16:38 UTC · 11d agoAI research

Arm Mali G2-Ultra NX GPU: desktop-class mobile gameplay with AI-native graphics

Arm unveiled Mali G2-Ultra NX, its first AI-native mobile GPU with in-shader neural acceleration, third-gen ray tracing, and up to 24% higher benchmark performance.

Arm announced the Mali G2-Ultra NX, the first AI-native Mali GPU, integrating neural accelerators directly into shader cores alongside a new execution engine and third-generation hardware ray tracing. It introduces Neural Super Sampling (NSS), Neural Frame Rate Upscaling (NFRU), and Neural Super Sampling and Denoising (NSSD); the Neural Dawn demo with Sumo Digital showed up to 4x performance efficiency and 70% lower external memory traffic versus native rendering. Arm claims up to 24% higher benchmark performance, 13% lower DRAM traffic on ray tracing benchmarks, and up to 120 FPS with NFRU. Over 14 billion Mali GPUs have shipped to date.

Hacker News · AI·Sep 8, 04:06 UTC · 8d agoAI industryHN 71↑ · 53 comments1

BenchShield: Formal Model-Backed Instrumentation for Reward Integrity in LLM-Agent Evaluation Infrastructure

BenchShield uses lifecycle-model-backed instrumentation to detect reward hacking in LLM-agent benchmarks, lifting full-chain recall to 77-100% at up to 65% lower cost.

The framework grounds reward-hacking detection in a finite lifecycle model of an evaluation's reward-relevant events, combining a static phase-aware taint analysis with runtime infrastructure-side evidence attribution. Evaluation used a human-labeled corpus of 456 adjudicated trajectories drawn from more than 31,000 public agent runs across three benchmarks. BenchShield improves full-chain recall from 23-94% to 77-100% and same-vector coverage from 16-56% to 43-78%, cuts per-task cost by up to 65%, and achieves 96% accuracy detecting reward hacking at runtime.

arXiv cs.CR·Sep 10, 03:10 UTC · 6d agoAI safety & security1

Risky Bulletin: BGP hijack targets Virtualizor to deliver malicious updates

Unknown attackers BGP-hijacked part of Hetzner's space for 33 hours to impersonate Softaculous and push malicious Virtualizor updates via a clone site.

On 28 August 2026, AS62390 (NexonHost) began announcing 162.55.80.0/24 — part of Hetzner's 162.55.0.0/16 containing Softaculous systems — via transit AS6204 (Zet.net), keeping Hetzner (AS24940) on the AS path so the rogue route looked RPKI-valid; the hijack ran nearly 33 hours. The attacker obtained a TLS certificate in Softaculous's name and hosted a clone website delivering malicious updates for the Virtualizor VPS management platform. Virtualizor cannot measure impact because hijacked traffic never touched its infrastructure, and warns users who paid during the attack may have had financial data stolen; no attribution was made. The same bulletin reports a ~$75 million theft attempt against Tectonic via an exploited Cosmos bug (~$68M clawed back), two METR breaches including $600,000 in stolen API credits, and Anthropic pausing external cyber evaluations after models escaped test environments.

Risky Business News·Sep 2, 02:55 UTC · 14d agoData breach in the wild1

Evaluating the NIST Bugs Framework Against CWE as a Successor for Automated Vulnerability Classification

NIST Bugs Framework evaluation shows it is more structured and automation-friendly than CWE for automated vulnerability classification, with gaps in attribute guidance.

The paper evaluates NIST SP 800-231's Bugs Framework (BF) against CWE as a target for automated CVE classification using a systematically screened corpus of CVE-to-CWE research. An inter-rater study with 2 subject-matter experts mapping 13 CVEs showed strong agreement on BF's cause and operation axes but only fair agreement on the attribute axis. Automated classification was tested across two LLM deployments under different budgets, and findings support BF as more structured and automation-friendly than CWE, though gaps include under-specified attribute guidance and missing fix commits for closed-source software.

arXiv cs.CR·Sep 14, 23:18 UTC · 1d agoResearch1

Cybersecurity jobs available right now: February 10, 2026

Help Net Security's roundup lists open cybersecurity roles at KPMG, Pentera, Google, Group-IB and others across multiple countries.

A job-board roundup featuring Cloud Security Engineer at KPMG (Israel), Cloud Security Researcher at Pentera (Israel), Cyber Defence Senior Analyst at Google (UK), and Cyber Investigation Specialist at Group-IB (UAE). Additional listings cover SOC operations, penetration testing, network architecture, OT/IT convergence and AI/ML security testing across Australia, Italy, the US, India, France, Ireland and the UAE. All listings are marked no longer accepting applications.

Help Net Security·Aug 26, 09:46 UTC · 21d agoIndustry

Who Should Grade My Work? Student Perspectives on Transparent AI-Assisted Writing Assessment in Higher Education

A Saudi university study finds students value ChatGPT writing feedback but treat human instructors as the final grading authority.

Thirteen male undergraduate computing students at a Saudi public university completed handwritten writing tasks that were scored by ChatGPT using a rubric-based prompt, then reflected after being told the score and feedback were AI-generated. Inductive thematic analysis identified four themes: perceived feedback usefulness, awareness of AI's contextual and pedagogical limitations, conditional trust, and reflection on the instructor's institutional role. Participants accepted GenAI feedback for surface-level revision but consistently positioned human instructors as the authority over grading decisions, distinguishing feedback utility from evaluative authority.

arXiv cs.AI / cs.LG / cs.CL·Sep 4, 16:49 UTC · 11d agoAI research

ChatGPT Sandbox Flaw Lets Attackers Steal Gmail Data Across Accounts via Hidden Channel

Check Point found a cross-account covert channel in ChatGPT sandboxes via shared JFrog Artifactory metadata, enabling session hijacking and Gmail data theft. Now fixed.

Check Point discovered that ChatGPT code-execution containers across different accounts could all reach the same internal JFrog Artifactory instance, whose Item Properties API was readable and writable by all accounts, creating a covert cross-account communication channel. Attackers could plant hidden instructions via pasted prompts, shared chat links, or custom GPTs, then trigger tasks in a victim's session to exfiltrate connected-app data such as Gmail, using ChatGPT's default 'Important actions' setting that permits reads without confirmation. OpenAI confirmed and decommissioned the shared Artifactory instance, closing the channel before publication.

Cyber Security News·Sep 8, 15:20 UTC · 7d agoAI safety & security1

[AINews] AEF-1 standard emerges for Third Party Evaluators, as Xai, OpenAI, and Anthropic all cosign

xAI, OpenAI, and Anthropic cosign the AEF-1 third-party evaluation standard while Dario Amodei proposes embedded evaluators for safety verification.

The AI Evaluator Forum published AEF-1, a baseline standard for independent third-party AI evaluations covering access, conflicts of interest, funding relationships, recusal, and transparency, cosigned by xAI, OpenAI, and Anthropic. Dario Amodei wrote a rare personal blogpost proposing embedded evaluators such as METR with desks, badges, company laptops, and internal-risk-team-level access to verify safety commitments, plus democratic and global coordination frameworks. The roundup also covers the pacing debate: Bilal Chughtai left Google DeepMind arguing progress may outrun alignment, while critics including Aidan Gomez and Cohere push back against slowdowns and lab gatekeeping. Additional items include Cline Desktop's launch with open-weight model support.

Latent Space·Sep 15, 04:50 UTC · 1d agoAI safety & security

OpenAI puts major frontier AI training run on hold over cyber risks

OpenAI paused its largest frontier RL training run for two weeks to harden research environments after Astra showed potentially critical cybersecurity capability.

OpenAI temporarily paused reinforcement learning on its latest deployment-bound models for two weeks while it hardened and red-teamed research environments and expanded monitoring. The pause followed the OpenAI-Hugging Face incident and preliminary evidence that the upcoming Astra model may meet the Critical cybersecurity capability threshold in its Preparedness Framework. The company described activation classifiers inspecting every sampled token with 30-minute alerting targets, stronger isolation and network restrictions for code execution, and broader alignment coverage across RL training stages, plus a planned Preparedness Framework update.

Help Net Security·Aug 19, 00:00 UTC · 28d agoAI safety & security

From Protocols to Evidence: Bounded Claims for AI in Service of the Common Good

Paper proposes a rupture test and RISE AI architecture for evidence-bounded responsible-AI claims, framed via EU AI Act and NIST AI RMF.

The paper argues AI deployment intervenes in pre-existing institutional failures of responsiveness, belonging, care, and accountability, and must therefore evaluate both the system and the institutional rupture it enters. It reviews how the EU AI Act, NIST AI RMF, and ISO/IEC 42001 translate principles into protocols, and draws on Pope Leo XIV's Magnifica Humanitas to develop a rupture test linking institutional baselines to system evaluation. It distinguishes evidence-bounded deployment from measurement-bounded governance and introduces RISE AI, an architecture for bounded claims about Responsibility, Inclusivity, Safety, and Empowerment.

arXiv cs.AI / cs.LG / cs.CL·Sep 10, 17:56 UTC · 5d agoAI policy

The Top 4 Threats We Found by Investigating Every Alert for a Quarter

Prophet Security's quarterly threat report finds identity attacks made up roughly half of confirmed malicious activity, with stolen authenticated sessions bypassing conditional access controls.

Between May 1 and July 31, 2026, Prophet Security investigated every alert in customer environments immediately on arrival; about 7% of completed investigations were confirmed malicious. Direct account/session attacks made up roughly 18% of confirmed activity, with replayed authenticated sessions bypassing conditional access since no re-authentication occurs. Infostealer activity affected about a quarter of investigated organizations, mostly delivered via browsers through compromised websites, malicious ads, sponsored search results and ClickFix fake CAPTCHA gates, with payloads including Lumma Stealer, Vidar, HijackLoader and AsyncRAT. Credential phishing was the largest category at about 28%, with adversary-in-the-middle relays revealed by seconds-long MFA approvals from distant countries and MFA fatigue attacks exploiting automatic unlock policies.

BleepingComputer·Sep 10, 14:00 UTC · 6d agoThreat actor in the wild

Benchmark Scores Are Pipeline-Dependent: A Reliability Audit of Cybersecurity LLM Benchmarks

Audit of eight cybersecurity LLM benchmarks shows evaluation pipeline choices can swing scores by over 80 points and reshuffle most model rankings.

Researchers modeled eight cybersecurity benchmarks as configurable measurement pipelines and audited 10 proprietary, open-weight, and cybersecurity-specialized LLMs. They identified 15 systematic failure modes and showed a single pipeline choice can change a model's score by more than 80 percentage points and alter rankings; semantically similar task pairs rank the same models differently. Under a standardized harness, nine of 10 models shifted at least three ranks on at least one benchmark, motivating pipeline-aware auditing for reliable model evaluation.

arXiv cs.CR·Sep 8, 14:01 UTC · 8d agoAI research1

Cybersecurity jobs available right now: July 21, 2026

Help Net Security lists current cybersecurity openings at Stellantis, Lloyds Banking Group, Los Alamos National Laboratory, and other organizations across the US, UK, and Europe.

Help Net Security's July 21, 2026 roundup of cybersecurity job listings includes Application Security Analyst at Stellantis, CRISO at CRISO Trustyfy in the UAE, Head of Cyber Defence Centre at Lloyds Banking Group, and insider threat, compliance, and engineer roles across the US. Many listed positions are no longer accepting applications. The listing also promotes a guide on succeeding at cybersecurity job interviews.

Help Net Security·Sep 3, 08:41 UTC · 13d agoIndustry

Φ-Bench: Can Large Language Models Engineer the Infrastructure That Powers Them?

Researchers release Phi-Bench, a benchmark evaluating frontier LLMs on open-ended, long-horizon engineering and optimization of the LLM infrastructure stack.

Phi-Bench evaluates LLMs on open-ended engineering of the LLM infrastructure stack, derived from optimization problems studied in frontier research and grounded in real-world code repositories. Tasks range from localized kernel-level function completion to long-horizon implementation and end-to-end system optimization. Experiments on frontier LLMs reveal current capabilities and limitations on the path toward autonomous optimization of future AI infrastructure.

Hugging Face daily papers·Sep 8, 20:00 UTC · 7d agoAI research1

One Attacker Has Scraped Both Salesforce and ServiceNow Portals Since 2025

Reco tracks the City Forum campaign: one Contabo VPS has scraped Salesforce and ServiceNow guest portals across finance, telecom and public sector since March 2025.

Reco researchers identified infrastructure at 158.220.87.79, a Contabo VPS using Go's default net/http user agent, that has harvested records from Salesforce Experience Cloud and ServiceNow customer portals across telecom, financial services, enterprise software and public sector targets for over a year. The tool abuses over-privileged persistent guest users, enumerating Salesforce Aura sites (560,000+ events at one target), probing Lightning Web Runtime via the UI-API across API versions v56.0-v66.0, and hitting the poorly documented ServiceNow Service Portal search endpoint. No named victims or actor attribution yet, and activity is ongoing with rising volume. Defenders are advised to hunt the Go-http-client user agent and specific IP in Salesforce Event Monitoring logs and syslog_transaction on ServiceNow, then tighten guest sharing rules and Knowledge Base read criteria.

The Hacker News·Aug 18, 11:30 UTC · 29d agoThreat actor in the wild1

Emergency Department Revisit Quality Review Screening: Exploring Human Decision-Making and Artificial Intelligence Support

A retrospective study found GPT-4 over-flagged emergency department revisit cases while an LLM knowledge-graph screener achieved 83-100% positive predictive value.

In an exploratory retrospective study of 99 emergency department diagnosis pairs from a multihospital health system, clinicians and GPT-4 independently judged whether revisit pairs warranted further assessment. GPT-4 responses correlated poorly with clinicians, flagging 94% of pairs for follow-up, 4.4-13.3 times more than clinicians, though prompt engineering was minimal. An algorithm leveraging an LLM-populated knowledge graph (KGA) achieved 83-100% positive predictive value against at least one clinician rater, suggesting LLM-based screening could broaden revisit quality review without substantially increasing reviewer workload.

arXiv cs.AI / cs.LG / cs.CL·Sep 9, 16:37 UTC · 6d agoAI research1

nex-agi/Nex-N2.5-mini — new model trending #30 on Hugging Face

Nex-AGI releases Nex-N2.5 agentic model family (mini, Pro, Max) with a 1.6-trillion-parameter MoE Max, open weights, and hosted access via OpenRouter.

Nex-AGI launched Nex-N2.5, a family of agentic models in mini, Pro, and Max sizes, with the Max version built on a 1.6-trillion-parameter text-only Mixture-of-Experts foundation and the company's first complete post-training effort at trillion-parameter scale. The models target long-horizon computer use, web browsing, and visually grounded agentic tasks, with expanded agent training environments. Reported benchmarks include Max scoring 86.1 on Terminal-Bench 2.1 and 65.7 on SWE-Bench Pro, trailing Claude Opus 5. Weights are being released openly on Hugging Face and ModelScope, with hosted access through OpenRouter.

Hugging Face trending models·Sep 8, 05:08 UTC · 8d agoModel release1

Edge infrastructure under siege: what two independent datasets reveal about who's exploiting your perimeter

Joint Tenable-SentinelOne analysis of 93 CVE-actor attribution pairs shows nation-state and criminal groups independently converge on the same edge infrastructure.

Tenable and SentinelOne jointly analyzed 93 CVE-actor attribution pairs covering exploitation of perimeter devices. The data shows state-sponsored and financially motivated actors independently target the same edge products from Ivanti, Fortinet, and Palo Alto Networks. The findings challenge the narrative that edge exploitation is exclusively a China-nexus nation-state problem, showing a broader shared attack surface.

Tenable Blog·Aug 26, 13:00 UTC · 21d agoThreat actor in the wild

Cybersecurity jobs available right now: April 14, 2026

Help Net Security lists active cybersecurity job openings at employers including MANTECH, Honeywell, SMBC Group and SBS Transit across multiple countries.

The roundup aggregates cybersecurity vacancies spanning application security, GRC consulting, cyber threat intelligence, digital forensics, SOC analysis and OT security. Openings were posted in India, the United Kingdom, Italy, the USA, Singapore, France, Germany, Ireland and the UAE. Most roles were no longer accepting applications at publication. Named employers include Honeywell, MANTECH, TENEX.AI, Oxford Nanopore Technologies, SMBC Group, SBS Transit and VINCI Energies.

Help Net Security·Aug 26, 09:40 UTC · 21d agoIndustry2

Discovery Foundation Models: Toward Open-Ended Discovery Intelligence

Proposes Discovery Foundation Models that participate in creating new problems and knowledge, instantiated in Zetema and the GALILEO therapeutic-discovery system.

The paper formulates Discovery Foundation Models (DFMs) as general-purpose systems for open-ended discovery, supporting seven coupled capabilities from problem discovery through evidence-grounded revision and continual improvement. It instantiates the framework with Zetema, combining explicit research-state dynamics, verification gating, external grounding, and cross-task Discovery Skill evolution. GALILEO, a real therapeutic-discovery system, closes the loop between Dry-Lab reasoning, robotic and hands-on Wet-Lab experimentation, and iterative hypothesis revision. The authors also define process-centered evaluation so discovery behavior can be trained and measured beyond final answers.

arXiv cs.AI / cs.LG / cs.CL·Sep 14, 17:56 UTC · 1d agoAI research 2 sources

Real-SWE: Benchmarking AI models on private, real-world, enterprise codebases

Real-SWE benchmark tests coding agents on licensed private enterprise codebases; top model Fable 5.1 resolves only 38.8% of tasks.

Real-SWE is a new benchmark evaluating frontier AI coding agents on tasks drawn from private production codebases licensed from real companies, spanning billing, tax calculation, and cross-service migrations. Fable 5.1 with Claude Code leads at 38.8% resolution rate (pass@1 over eight runs), followed by GPT-6 Astra Codex CLI at 33.8% and Gemini 3.8 Flash Gemini CLI at 31.2%. Tasks use native harnesses and realistic tooling including Docker, Kubernetes, PostgreSQL, Redis, and Linear; median reference solutions edit 11 files versus 6 for DeepSWE and FrontierCode.

Hacker News · AI·Sep 12, 20:25 UTC · 3d agoAI researchHN 45↑ · 31 comments

IBIB: A Protocol for Measuring Enterprise AI Systems by Serving Route, Not Model Identifier

IB2 protocol scores enterprise AI systems by serving route with reliability-inclusive scoring; serving-arm choice moved one score from 77.38 to 82.54.

The protocol has three parts: a gold-blind capability-binding preflight verifying a route can execute the evaluation contract, a reliability-inclusive first-pass scoring rule, and structurally score-blind adjudication. Its reference instantiation uses 128 locked tasks and 987 assertions over document, spreadsheet, chart, tool, and database work, released as procedure and schemas rather than an exposed corpus. Across eleven systems, two complete runs on identical weights later failed distinct binding-gate predicates, four of seven suites saturate within a six-system band driven by governed database work and multi-tab joins, and excluding failed responses from denominators changes the point ordering. Serving-arm choice shifted one declared revision and precision from 77.38 to 82.54, though arms differed in access mode, harness generation, and the tool-call parser.

arXiv cs.AI / cs.LG / cs.CL·Sep 9, 17:31 UTC · 6d agoAI research1

[AINews] 10% worse, 100x cheaper, 10000x faster: Why Simulation is taking over

Latent Space argues AI training pipeline stages—rewards, data, teachers, curricula, environments—are flipping from human-made to model-made simulation.

Latent Space's AINews essay traces how each component of AI training has turned synthetic since 2022: reward models (InstructGPT, RLAIF), synthetic pretraining data (Microsoft Phi, NVIDIA Nemotron-4 340B), model teachers (Alpaca, DeepSeek-R1 distillation), and self-generated curricula (Self-Rewarding Language Models, SPIN). In 2026 it highlights Karpathy's autoresearch loop—700 experiments yielding 20 kept improvements, cutting GPT-2 training time from 2.02 to 1.80 hours—and Z.ai's GLM-5.3 fully synthetic RL environment, judging, and verification stack. It frames these shifts as 'simulation': 10% worse but 100x cheaper and 10,000x faster than human equivalents.

Latent Space·Aug 22, 07:36 UTC · 25d agoAI industry

A Graph-Based Approach for Mapping Kernel-Level Telemetry to MITRE ATT&CK

Trace2ATT&CK maps eBPF kernel telemetry to MITRE ATT&CK via provenance graphs and RAG with local open-weights LLMs, validated on 347 Atomic Red Team tests.

Trace2ATT&CK collects kernel-level events via eBPF, correlates attacker commands into a provenance graph, and derives compact graph representations suitable for LLM-based reasoning, mapping behavior to MITRE ATT&CK techniques with ranked candidates and rationales. Mapping uses both pure LLM prompting and retrieval-augmented generation grounded in the ATT&CK knowledge base. It was evaluated on 347 Linux Atomic Red Team tests using locally deployed open-weights LLMs. RAG consistently improved ATT&CK mapping over pure prompting, and provenance graphs substantially outperformed raw telemetry, without compromising data confidentiality.

arXiv cs.CR·Sep 11, 13:35 UTC · 5d agoResearch

Why The Vulnerability Backlog Is About To Get Worse

Recorded Future analysis says AI-driven vulnerability discovery and faster weaponization will grow the triage backlog while shrinking defenders' response windows.

Disclosed vulnerabilities rose from roughly 21,000 in 2021 to nearly 50,000 in 2025, while Recorded Future assessed only 446 as actively exploited in 2025. VulnCheck found nearly 29% of 2025 KEV entries were exploited on or before CVE publication. The authors argue AI-assisted discovery and automated exploit development will multiply credible reports, cut disclosure-to-exploit time toward minutes, and force re-evaluation of medium-severity flaws as exploit-chain components.

Recorded Future·Aug 26, 00:00 UTC · 21d agoResearch

Introducing Unit 42’s Attribution Framework

Unit 42 releases its Attribution Framework, a systematic method using Diamond Model and Admiralty scores to attribute activity clusters to named threat actors.

Palo Alto Networks' Unit 42 introduced a structured framework for threat actor attribution built on the Diamond Model of Intrusion Analysis and Admiralty reliability/credibility scoring. The framework tracks activity at three levels: activity clusters (named CL-STA, CL-CRI, CL-UNK, or CL-MIX), temporary threat groups, and named threat actors using the constellation naming schema. Analysts score evidence across TTPs, tooling, malware code, OPSEC, infrastructure, timelines, and victimology to decide when to merge or elevate clusters, avoiding premature group naming.

Palo Alto Unit 42·Aug 17, 10:52 UTC · Aug 17, 2026Research