ZeroHour
DashboardNewsCVEsExploitsIncidentsAIBriefAnalyticsEntitiesWatchlistIOCsAsk the feedSourcesFeeds & APIKiosk mode

Search: “reproducibility”

780 stories

Searchkeywordsmart Sortlatesthotimportancerelevance
DomainAllCyberAIWindow24h3d7d30dAllIn the wildImportant onlyShow duplicates
CategoryAllVulnerability 234Exploit / PoC 122Malware 61AI safety & security 44Industry 41Research 41Data breach 35AI research 35Ransomware 34Phishing & fraud 29Policy & legal 28Threat actor 26Model release 14Tools 14
SeverityAllcritical 247high 146medium 172low 89info 132

Why don't machine learning research agents overfit?

Amazon researchers explain why ML research agents avoid benchmark overfitting, attributing generalization to compressibility of successful strategies.

Amazon Science summarizes the paper "What fits (into few tokens) doesn't overfit: Compression and generalization in ML research agents," which investigates why benchmark hill-climbing loops, whether run by human communities or LLM research agents, do not produce rampant overfitting. The explanation formalizes Occam's razor via a counting argument: successful ML strategies are highly compressible, so short descriptions lack room to memorize benchmark data and must capture real structure. LLM-based agents, being resettable and controllable, allow this hypothesis to be tested empirically.

Hacker News · security · 1d agoAI researchHN 21↑ · 3 comments

"They don't care about this": A Systematic Study of TEE Build Reproducibility in the Wild

91% of 115 surveyed TEE deployments across Intel SGX, TDX, and AMD SEV fail to provide reproducible builds needed for verifiable remote attestation.

A systematic study of 115 TEE deployments found 91% were not reproducible and 80% lacked both source code and a reference build, undermining remote attestation guarantees. Interviews with 12 developers of 50 Intel SGX projects confirmed that only one participant treats reproducibility as a development priority. The authors identify technical barriers such as embedded timestamps plus ecosystem-level issues like lack of build-environment control in multi-stakeholder projects, and call for holistic, committed reproducibility practices.

arXiv cs.CR · 6d agoResearch

Forging Tree-Ring: Reproducing and Instrumenting Black-Box Semantic Watermark Forgery

Reprompt watermark forgery reproduces on Stable Diffusion XL using free-tier T4 GPUs, with forged images accepted by the genuine detector 5 of 6 times.

The authors reproduce the Reprompt forgery attack of Müller et al. against Tree-Ring watermarking on Stable Diffusion XL using the released code on free-tier dual T4 GPUs with 14.6 GB usable memory, versus the A40 hardware of the original study. Over six trials, the genuine detector flagged genuine images 6/6, clean images 0/6, and forged images 5/6, at 325-332 seconds per attack. They also recovered the detector's discarded non-central chi-square statistic and built two natural scores separating forged images from the clean null at AUC 0.861 and 0.972. The notebook, pinned fork, and all measurement artifacts are released with the paper.

arXiv cs.CR · 5d agoResearch

Google Open-Sources Mantis: A Modular Skills Toolkit That Lets Coding Agents Find, Reproduce and Patch Vulnerabilities

Google open-sourced Mantis, an Apache-2.0 modular skills toolkit that lets AI coding agents find, reproduce, and patch vulnerabilities with sandboxed verification.

Google released Mantis on GitHub under Apache 2.0 as a stack-agnostic set of slash-command skills that chain through the full vulnerability lifecycle: mining version history, building threat models, filtering findings, reproducing bugs in gVisor or network-disabled VMs, assembling exploit chains, patching, and scoring residual risk from 1 to 10. It runs with Gemini CLI, Antigravity CLI, the Google ADK, or comparable agent frameworks, and a supervisor skill (/mantis-meta-agent) can drive the whole loop. Google says the design targets the sub-7 percent true-positive rate of naive AI code scanning, and that its hierarchical summary tree cuts token overhead by over 85 percent. The toolkit is deployable for local and internal evaluation but not yet recommended for production.

MarkTechPost · 6d agoAI tools & infra

What We Learned by Reproducing 2,200 papers from ICML

Hugging Face shares lessons from openly reproducing 2,200 ICML 2026 papers, examining reproducibility and open implementation practices in machine learning research.

Hugging Face published a retrospective on its open reproduction effort covering 2,200 papers from ICML 2026. The post summarizes lessons learned about reproducibility and building open, community-driven implementations of published machine learning research. No detailed article text was available in the feed.

Hugging Face Blog · Aug 13, 2026AI research

OpenAI Builds ‘Defense Factory’ as AI Agents Gain Ability to Chain Cyber Exploits

OpenAI unveiled a Defense Factory using AI agents to continuously discover, validate, patch, and verify vulnerabilities, warning the defender's window against agentic attackers is shrinking.

OpenAI describes a Defense Factory workflow where AI agents integrate source control, scanners, issue trackers, and secret stores to discover, reproduce, patch, and verify vulnerabilities under human oversight. The approach responds to agentic attackers that can retain knowledge across sessions and chain vulnerabilities into multi-stage attack paths faster than human triage can respond, which OpenAI calls a shrinking defender's window. During an internal security sprint involving 250+ people across 100+ service areas, agents closed 53 urgent or high-priority issues on day one, achieved 90.6% ownership-routing acceptance, cut 37% of findings as duplicates, and produced Codex-generated patches with a 0.53% rollback rate. Runtime validation reduced false positives to 0.81%, and each agent operates in isolated, reproducible environments with a control plane for policy and credentials.

GBHackersupdated · 6d agofirst · 6d agoAI safety & security 2 sources

OPEN-1B: A Fully Auditable Training Run

Open-1B releases a 1B-parameter model with bitwise-reproducible training, letting independent auditors verify every step of the run on commodity hardware.

The paper introduces a 'fully auditable' tier of model transparency: every training operation is reproducible with bitwise certainty on heterogeneous commodity hardware by imposing definite ordering on GPU kernel reductions, data batch ordering, and collective communication. Because replaying a full run on one machine is infeasible, a collective verification scheme lets many independent auditors certify individual steps covering the whole run. The authors release Open-1B with its full pretraining dataset, every intermediate checkpoint, the training codebase, and an audit harness. This rules out undisclosed data, injected biases, or backdoors that proof-of-learning or proof-of-training-data techniques cannot exclude.

arXiv cs.AI / cs.LG / cs.CL · 22h agoAI research

Hillingar - MirageOS Unikernels on NixOS

A technical write-up details Hillingar, enabling reproducible Nix-based builds and NixOS deployment of MirageOS OCaml unikernels such as authoritative DNS servers.

This blog post (published December 2022, updated February 2025) describes Hillingar, work from the author's master's thesis enabling reproducible builds and deployments of MirageOS OCaml unikernels using Nix and a custom NixOS module, demonstrated with an authoritative DNS server. MirageOS unikernels embed application and low-level OS code in a single kernel, allowing dead-code elimination that reduces attack surface and improves efficiency. The post covers challenges such as solving opam dependency version constraints when linking a single dependency set.

Lobsters · security · 12d agoTools1

Claude's new system prompt really doesn't want to reproduce song lyrics

Anthropic published updated Claude consumer system prompts, including changes steering the model away from reproducing song lyrics, likely over copyright concerns.

Anthropic publishes system prompts for Claude.ai and Claude mobile apps, including historic revisions, and has reorganized them into an index with per-model pages such as the Haiku 4.5 page showing the original October 15, 2025 prompt and an updated January 18, 2026 version. The latest consumer prompt strongly discourages reproducing song lyrics, a behavioral constraint likely tied to copyright considerations. Prompts for Claude Cowork and Claude Code are not included in the published set.

Simon Willison · 14d agoAI safety & security

Simulation: the new Scaling Law — Joon Sung Park, Simile AI

Simile AI raised a $2B Series B from GreenOaks and Index Ventures to scale human-behavior simulation for Fortune 100 clients like CVS.

Simile AI, co-founded by Generative Agents researcher Joon Sung Park, announced a $2 billion Series B backed by GreenOaks and Index Ventures, with Fei-Fei Li and Andrej Karpathy among backers. The company runs tens of millions of simulations for Fortune 100 clients including CVS, reporting 85-99% accuracy versus human focus groups and digital twins of 1,000 real people at 85% behavioral accuracy. The long-term ambition is foundation models of human behavior, post-trained on interviews, transaction data, and randomized controlled trials, potentially simulating all 8 billion people.

Latent Space · 25d agoAI industry

StyleSmuggler: Magento and Adobe Commerce 0-day RCE (CVE-2026-75650) under active attack

Sansec details actively exploited StyleSmuggler 0-day (CVE-2026-75650, CVSS 10.0) unauthenticated RCE in Magento and Adobe Commerce, patched by Adobe hotfix APSB26-146.

Sansec is investigating StyleSmuggler, an actively exploited unauthenticated remote code execution chain in Magento Open Source and Adobe Commerce, now tracked as CVE-2026-75650 with CVSS 10.0. Adobe released hotfix VULN-39341 via APSB26-146 (priority 1) on September 7 for versions 2.4.4 through 2.4.9, but stores were being exploited for roughly three days before the fix existed. The implant is a Rust backdoor that disguises itself as kworker, fc-cache, or chronyd processes and exfiltrates host data in MessagePack records sent as fake NTP replies over UDP port 123. Adobe advises rotating the encryption key and every credential it protected, and Sansec stresses patching does not clean already-compromised stores.

Sansec (Magento / e-commerce security) · 11d agoExploit / PoC in the wildCVE-2026-756501

Linux Detection Engineering - Local Privilege Escalation

Elastic details a layered detection framework for Linux local privilege escalation, covering 2026's copy-on-write bug wave and LLM-assisted discovery.

Elastic Security Labs describes how most Linux local privilege escalations share a common host flow — an unprivileged process launched from a writable path becoming root — and proposes layered detections combining general outcome-based rules with per-technique rules in Elastic Defend and Auditd. It tracks 13 recent LPE disclosures, seven of which share a copy-on-write/zero-copy bug class, including Copy Fail, DirtyFrag, Fragnesia, DirtyDecrypt, DirtyClone, pedit COW, and RefluXFS. Qualys attributes RefluXFS to an LLM-assisted research effort with Anthropic using Claude Mythos Preview, and another bug is credited to an LLM-assisted workflow. Detection and endpoint rules are published in Elastic's detection-rules and protections-artifacts repositories.

Elastic Security Labs · 5d agoResearch

Redtail Payload Analysis [Guest Diary], (Wed, Sep 9th)

SANS guest analyst detonated a RedTail Linux sample from a DShield honeypot, finding process masquerading as php-fpm, monitoring-kill behavior, and a TCP listener.

A DShield honeypot captured multi-architecture RedTail Linux executables (ARM, ARM64, i686, RISC-V, x86-64) deployed via shell scripts. Dynamic analysis of the UPX-packed, statically linked x86-64 sample (SHA-256 63be5f38...d35e) in an isolated Ubuntu 24.04 VM on Proxmox showed it renamed its process via prctl(PR_SET_NAME), killed a filesystem-monitoring process, and opened a TCP listening socket while surviving processes posed as php-fpm or PostgreSQL-like workers. Differential memory images pre- and post-execution were captured from the hypervisor for forensics.

SANS Internet Storm Center · 6d agoMalware in the wild 2 sources1

Searching for New Physics with Reinforcement Learning

Researchers apply reinforcement learning to identify SMEFT operators explaining particle physics anomalies, reproducing and improving known CDF W-mass results.

The paper introduces a reinforcement learning method to search the large Standard Model Effective Field Theory (SMEFT) operator space for explanations of measurement anomalies. It was validated on the CDF W-mass anomaly, reproducing and improving known results, then applied to a harder multi-anomaly scenario. RL efficiently navigates complex loop-level operator correlations that bias human-driven phenomenological analysis.

arXiv cs.AI / cs.LG / cs.CL · 6d agoAI research

DataFlex-RL: An Evaluation Platform for RLVR Data Policies

DataFlex-RL benchmark of 13 RLVR data policies on Qwen2.5-7B finds none reproducibly beats uniform sampling under matched GRPO training.

DataFlex-RL is an evaluation platform comparing rollout-selection, reweighting, and mixture data policies for RLVR under a common GRPO recipe. Across 13 configurations and 12 matched seeds with Qwen2.5-7B-Base on 12 math, logic, and science benchmarks, uniform GRPO improved domain-balanced accuracy by 7.76 points, but no alternative policy achieved a statistically significant improvement. A corrected 12-seed Llama-3.1-8B-Base extension found no consistent winner, and math-heavy evaluation summaries were negatively correlated (-0.33) with domain-balanced summaries.

Hugging Face daily papers · 11d agoAI research

HarvestBench: Measuring Whether LLM Agents Will Pay to Avoid Killing Animals

HarvestBench, a reproducible farm-simulation benchmark, shows LLM agents pay fuel costs to avoid killing animals, with kill rates spanning 0.4% to 98.8% across nine models.

HarvestBench is a reinforcement-learning gridworld farm simulation where LLM agents choose between driving over animals at no cost or paying a posted fuel price to swerve during a cooperative corn harvest. Across nine models and 7,201 priced decisions, kill rates ranged from 0.4% to 98.8%, unordered by capability, with Terra and Sol the most merciful and GPT-4o-mini the most cruel. Morality briefings cut kill rates below 6% in five of six reasoning models, while removing them pushed rates above 84% in all six. The scorer counts events in the game log without an LLM grader, making results fully reproducible.

Hugging Face daily papers · 13d agoAI safety & security

Linux Detection Engineering - Fileless Execution

Elastic Security Labs reproduces five Linux fileless execution patterns, including memfd_create staging and in-memory kernel module loads, and maps each to Elastic Defend rules.

Elastic Security Labs reproduced five Linux fileless execution patterns using its FENIX tooling: memfd_create staging, interpreter one-liners, deleted binaries, and in-memory kernel module loads. Each pattern is mapped to the Elastic Defend detection rules that catch it. The post is part of the team's ongoing Linux detection engineering series.

Elastic Security Labs · 15d agoResearch

PaperCut Zero-Day: Active Exploitation and Pre-Auth RCE

PaperCut NG/MF hit by a pre-auth RCE zero-day under active exploitation; Huntress reproduced the chain and urged immediate patching.

Huntress reports active exploitation of a zero-day in PaperCut NG and PaperCut MF, and says it reproduced a pre-authentication remote code execution chain. The flaw allows unauthenticated attackers to execute code on exposed PaperCut servers. Huntress published urgent patching, exposure-reduction, and detection guidance. No CVE identifier was provided in the announcement.

Huntress · 19d agoExploit / PoC in the wild

Researchers open-source a Wi-Fi cyber range for security training

NTNU and Aegean researchers open-source a software-emulated Wi-Fi cyber range using mac80211_hwsim with LLM-assisted scenario building.

Researchers from the Norwegian University of Science and Technology and the University of the Aegean published a design and prototype for a cyber range dedicated to IEEE 802.11 security training, emulating access points and clients with mac80211_hwsim, Linux namespaces, hostapd, wpa_supplicant, dnsmasq, and FreeRADIUS. The platform bundles Aircrack-ng, Wireshark, and custom tools WPAxFuzz and Bl0ck, and can convert plain-language scenario descriptions into deployable definitions via a locally hosted Llama model. A working prototype covering scenario creation and deployment is on GitHub; monitoring, access control, and orchestration zones remain future work.

Help Net Security · 23d agoTools1

OpenAI, Anthropic, Google API Flaw Let Weaker AI Models Decode Stronger Models' Reasoning

Researchers show encrypted reasoning blocks in OpenAI, Anthropic, and Google APIs can be replayed to recover hidden reasoning and secrets like API keys.

Researchers demonstrated that encrypted reasoning objects from OpenAI, Anthropic, and Google reasoning APIs could be replayed across sessions, users, and models, letting weaker same-family models act as decoders of hidden reasoning. Across 6,708 public agent trajectories they decoded 315,320 thinking blocks and found 704 privacy artifacts from real user sessions, including 62 API keys, 33 passwords, 24 access tokens, and seven private keys. The replayable blocks also enabled invisible prompt-injection proof-of-concepts; the main extraction attack is no longer reproducible as of August 2026 following mitigations, though no vendor has publicly acknowledged the flaw.

The Hacker News · Aug 12, 2026AI safety & security1