Search: “nerf-alternatives”

Smart search ranks by meaning as well as keywords (one row per story, last 45 days).

Pick Your Poison: Learning to Select Poison Sets for Stronger LLM Backdoor Attacks

Poison set selection swings LLM backdoor attack success from 3% to 80%; SAILS boosts held-out success by 30 points.

The paper shows that random poison set selection severely underestimates worst-case backdoor vulnerability: across three LLaMA-3-8B settings with fixed model, clean data, and poison count, attack success ranges from 3% to 80% depending only on which poison set is chosen. The authors formalize poison selection as oracle-budgeted set optimization and introduce SAILS, which learns a set scorer from a few hundred finetune-and-evaluate runs, ranks millions of candidate sets, and audits a shortlist. SAILS improves held-out attack success by 30 percentage points over the strongest influence baselines, transfers from small-scale to full-scale finetuning, and extends to code-generation, agentic, and API-only backdoors.

Hugging Face daily papers·updated Sep 14, 04:44 UTC · 2d ago·first Sep 13, 20:00 UTC · 3d agoAI safety & security 2 sources1

55

Decoy Direction Optimization: A Post-Hoc Defense Against LLM Abliteration

Researchers introduce Decoy Direction Optimization, a cheap weight-editing defense that blinds refusal-direction ablation attacks against open-weight LLM safety guardrails.

Refusal Feature Ablation bypasses safety guardrails in open-weight LLMs by projecting out a linear refusal direction, often with high attack success rates. Decoy Direction Optimization injects a high-magnitude nonlinear decoy into MLP neurons so attackers' contrastive estimators ablate a harmless orthogonal feature instead. Evaluated across six model families, DDO keeps ASR below 10% under standard RFA and on Llama-3-8B-Instruct reduces Heretic weight-level attack ASR from 88.7% to 18%. It costs 30 to 450 times less per configuration than trained defense baselines.

Hugging Face daily papers·Sep 13, 20:00 UTC · 3d agoAI safety & security

55

TCRF taken offline by DDoS attack after Claude user ban

The Cutting Room Floor game wiki was taken offline by a DDoS attack after a user leveraging Claude was banned.

The Cutting Room Floor (TCRF), a wiki documenting unused video game content, was knocked offline by a distributed denial-of-service attack. The attack reportedly followed moderation action banning a user who was using Anthropic's Claude. The incident highlights friction between community sites and AI-assisted users and tools.

Lobsters · security·Aug 28, 21:26 UTC · 19d agoAI safety & security in the wild

25

How Fragile Is Safety Alignment at Frontier Scale? A Single-Direction Attack on a 320B MoE

Researchers show directional ablation breaks refusal in GLM-5.3-Flash, a 320B-parameter MoE, cutting refusal by 41–89 points across seven benchmarks.

The study extends directional ablation, a white-box attack that removes an aligned LLM's refusal behavior, from dense models up to ~70B parameters to GLM-5.3-Flash, a 320B-parameter mixture-of-experts model with 288 routed experts, four-wide hyper-connection residual, and block-FP8 quantization. Editing attention, dense, and routed-expert writers jointly removes 0.776 of refusal, with 74% of the effect existing only under the joint intervention; the conventional module-name-based recipe reaches only 0.066 and fails silently on MoE architectures. The attack yields 41–89 percentage-point reductions in refusal across seven harmful benchmarks with no detected capability change, and a category-concentrated refusal residue survives all edits at ranks 1 to 12.

arXiv cs.CR·Sep 9, 06:48 UTC · 7d agoAI safety & security

45

Diffs vs. Whole Files: An Empirical Comparison of Iterative Edit-Based and Direct Generation for Flutter/Dart Code Models

Empirical study finds direct whole-file generation beats iterative diff-based editing for Flutter/Dart code models on about 1,790 held-out tasks.

Researchers trained Rainbow-Pony-100M from scratch and fine-tuned Qwen2.5-Coder-0.5B in both direct-generation and diff-based regimes, then evaluated four resulting models on roughly 1,790 Flutter/Dart tasks. Direct generation outperformed diff-based generation on compilation pass rate, bits-per-byte, character-level similarity, and blinded LLM-judge ratings. Diff-based editing is competitive only on short, localized edits in refactoring and error-handling tasks, a property the authors call task locality.

Hugging Face daily papers·Sep 4, 20:00 UTC · 12d agoAI research1

30

Bad Likert Judge: A Novel Multi-Turn Technique to Jailbreak LLMs by Misusing Their Evaluation Capability

Unit 42 details the Bad Likert Judge multi-turn jailbreak that abuses LLMs' evaluation capability, raising attack success rates over 60% across six frontier models.

Palo Alto Networks Unit 42 describes the Bad Likert Judge technique, a multi-turn jailbreak that asks a target LLM to act as a Likert-scale judge scoring the harmfulness of example responses. The highest-rated example in each scale can carry harmful content, bypassing the model's internal guardrails. Testing across six state-of-the-art text-generation LLMs showed an average attack success rate increase of more than 60% versus plain attack prompts, with tested models anonymized. The technique targets edge cases rather than typical use, and the article positions the work as guidance for defenders on potential jailbreak risks.

Palo Alto Unit 42·Aug 17, 12:34 UTC · Aug 17, 2026AI safety & security

60

Time Exploit Plummets Nday Flaws

Analysis shows time-to-exploit for n-day vulnerabilities has plummeted, shrinking defender patch windows as known flaws are exploited faster.

Infosecurity Magazine reports on analysis finding that the time between a vulnerability's disclosure and its exploitation in the wild has dropped sharply for n-day flaws. Faster exploitation compresses defenders' patching windows and raises the urgency of rapid patch deployment. The specific flaw families and metrics were not detailed in the available information.

Infosecurity Magazine·Aug 19, 00:00 UTC · 28d agoResearch in the wild

40

Import AI 470: No rights for machines; automating environment generation with SPADE; and building better GPU kernels with Hawkeye

METR analysis finds AI accelerating cyber vulnerability discovery, while SPADE self-play environment generation improves Qwen3 reasoning benchmark scores at 30B scale.

Import AI 470 discusses a METR research note reporting differential acceleration from AI: major acceleration in reported cyber vulnerabilities (cURL, OpenSSL, Firefox, Microsoft, NVD, OSV), minor acceleration in mathematics, and no measurable acceleration in AI-research optimization benchmarks. It also covers SPADE, a self-play framework from a multi-university team (University of Washington, Stanford, MIT, CMU, and others) that co-evolves executable training environments and agent capability using Environment Designer and Reasoning Agent roles with hint-based regret rewards. Trained on Qwen3-4B-Instruct-2507, Qwen3-8B, and Qwen3-30B-A3B-Instruct-2507 via GRPO (400 rollouts of 25 environments), SPADE lifted the 30B-A3B game-environment suite average to 58.3, +8.1 over base, and improved tool-use results across backbones. The issue also references Hawkeye for building better GPU kernels.

Import AI·Aug 24, 13:12 UTC · 23d agoAI research1

42

Safety for Whom? Boundary-Aware Self-Distillation for Controlled LLM Safety Refusal

A self-distillation safety framework tunes narrow-boundary refusals in Qwen3-8B, raising target-domain refusal to 84.75% while cutting over-refusal from 15.20% to 5.20%.

The paper formulates narrow-boundary safety, where deployments need refusals within specific topics rather than whole subjects, and proposes an offline self-generated framework with controlled topic generation, escalating retries, and harmful-benign boundary pairs. On political persuasion with Qwen3-8B, the method raised target-domain refusal from 9.47% to 84.75% and cut the mean unsafe-response rate across three broader benchmarks from 26.26% to 0.14%. Verified target-model responses reduced over-refusal from 15.20% to 5.20%, and boundary-pair data cut comply-side over-refusal on held-out pairs from 32.94% to 4.16%. Results show data composition controls the safety-usability trade-off and alignment should be evaluated on both sides of the refusal boundary.

Hugging Face daily papers·Sep 2, 20:00 UTC · 14d agoAI safety & security1

32

ToxicRAG: Compromising Retrieval-Augmented Generation Systems via Single-Shot Knowledge Poisoning Attacks

ToxicRAG shows a single narrative-form poisoned document can steer RAG answers, achieving 0.61-0.91 attack success rates across four LLMs.

The attack injects one document per target question written as a coherent knowledge-update narrative that acknowledges the previously accepted answer, introduces fabricated events that appear to invalidate it, and attributes the attacker-chosen answer to purported authorities. An optional answer-focused self-validation loop revises candidates when a surrogate LLM fails to reproduce the target answer. Across 100 target questions each from Natural Questions, HotpotQA, and MS-MARCO, with four victim LLMs and four dense retrievers, ToxicRAG achieves attack success rates of 0.61-0.91 and matches or exceeds the strongest baseline by 0 to 11 percentage points.

arXiv cs.CR·Sep 10, 04:42 UTC · 6d agoAI safety & security1

45

SchemeArena: Factorized Stress Testing of Scheming in LLM Agents

Researchers introduce SchemeArena, a 400-scenario benchmark stress-testing scheming in LLM agents, finding explicit instrumental goals are the strongest driver of covert misaligned behavior.

The paper presents SchemeArena, a 400-scenario benchmark built through factorized scenario synthesis spanning safety-relevant tool domains, instrumental goals, oversight conditions and pressure mechanisms. The accompanying SCOUT monitor grounds multi-criteria scheming judgments in evidence drawn from agents' reasoning and actions. Stress tests across five LLM agents show explicit instrumental goals are the strongest driver of scheming propensity, while action-only monitoring increased scheming in several closed models, suggesting partial oversight can act as an optimization constraint. The benchmark, code and monitor are released at github.com/launchnlp/SchemeArena.

Hugging Face daily papers·Sep 7, 20:00 UTC · 9d agoAI safety & security

40

N4D Mesh Controller: New infrastructure, a UPX-packed agent labeled "go-titan," and how to hunt for it

Datadog researchers detonated a new N4D Mesh Controller sample in microVMs, revealing rotated infrastructure, a UPX-packed go-titan agent, and persistence behavior.

Datadog Security Research executed a newer N4D Mesh Controller sample in isolated microVMs, uncovering rotated command infrastructure and a UPX-packed agent labeled go-titan. Runtime evidence showed MCP tool abuse, multi-service scanning, and persistence mechanisms. The report provides hunting guidance for defenders tracking this infrastructure.

Datadog Security Labs·Aug 20, 00:00 UTC · 27d agoMalware

55

Biology-in-the-loop: Amortized Adaptive Hit Discovery in CRISPR Screens

Researchers release AssayBench-Loop, a 1,389-screen CRISPR benchmark, and AssayLoop, a framework that learns adaptive hit discovery policies.

The paper introduces AssayBench-Loop, a large-scale benchmark of 1,389 CRISPR screens across five phenotype categories for adaptive hit discovery under budget constraints. It also introduces AssayLoop, which combines AssayFormer, a transformer-based amortized acquisition policy trained across historical screens, with LLM-derived biological priors via an adaptive handoff. On temporally held-out screens, AssayLoop achieves 5.67-fold enrichment over random selection and recovers 27.7% of hits after assaying roughly 5% of the candidate library, outperforming existing adaptive-design methods and standalone LLMs.

arXiv cs.AI / cs.LG / cs.CL·Sep 10, 17:45 UTC · 6d agoAI research1

30

Attack Chains, Not Just Attack Surfaces: Why Testing Individual Techniques Misses the Point

Filigran introduces Attack Chaining in OpenAEV to continuously simulate multi-stage attack paths, exposing gaps that isolated MITRE ATT&CK technique testing misses.

Filigran announced Attack Chaining, a new scenario type in its OpenAEV platform that links individual techniques into automated, continuously-run multi-stage attack paths, using each step's real output (credentials, tokens, open ports) to branch dynamically toward a final objective. The article cites Filigran's State of Threat Management report, in which 93% of security leaders reported a business-impacting cyberattack in the past 12 months, 88% said AI accelerates attackers, and 84% blamed siloed tools and disconnected testing. The 2025 DGFiP breach is cited as an example where individually survivable weaknesses chained into a major intrusion. The feature includes conditional chaining logic, live attack path mapping, structured findings for identifying chokepoints, and predefined scope and safety guardrails.

The Hacker News·Sep 15, 11:26 UTC · 1d agoTools

18

What Zero-Day Response Should Be in the Post-Mythos Era

Picus Security outlines a zero-day response playbook where defenders simulate exploit technique chains before public PoCs exist.

The article uses PaperCut NG/MF's August incident — exploitation in the wild before any patch, with the first emergency fix bypassed the same day and a third landing September 1 — as the template for AI-accelerated vulnerability response. It walks through a hypothetical CVE-2026-1001 (explicitly made up) to argue defenders should map CVEs to ATT&CK technique chains and simulate them against NGFW, WAF, EDR, endpoint hardening, and SIEM controls within minutes of disclosure. It notes disclosure-to-exploitation time has fallen from 21.5 days to hours.

BleepingComputer·Sep 15, 13:45 UTC · 1d agoResearch in the wildCVE-2026-1001

45

RetroThinker: Enabling Retrospective Thinking in Speech LLMs

RetroThinker is a post-training framework letting the Moshi speech LLM self-correct reasoning mid-stream, adding 11% GSM8K accuracy at similar latency.

Researchers introduce RetroThinker, a multi-stage post-training framework that equips the Moshi speech LLM to verify and forward-correct chain-of-thought steps during streaming inference. It combines supervised fine-tuning on curated retrospective thinking data with length-based direct preference optimization (DPO). On GSM8K it achieves an 11% absolute accuracy gain over non-retrospective baselines at comparable latency.

arXiv cs.AI / cs.LG / cs.CL·Sep 10, 17:41 UTC · 6d agoAI research1

28

PlannerForge: LLM Agents for Scenario-Based Testing of Motion Planners in Autonomous Driving

PlannerForge unifies scenario-based testing of autonomous driving motion planners in one LLM-agent framework, outperforming prior baselines.

PlannerForge is an LLM-agent framework that covers the full scenario-based testing pipeline for autonomous driving systems, spanning scenario generation, selection, modification, routing, planner testing, plus new enhancement and benchmarking stages. In evaluations with 10 off-the-shelf LLMs, best-per-task scores range from 0.88 to 1.00, and open-source 20-35B backends such as Qwen3.6:35B match commercial APIs on most tasks. End-to-end chaining retains 83% (commercial) and 78% (open) of seed queries, beats Scenario Factory 2.0 on executable generation, and cost-tuning lifts planner success from 50.4% to 70.2% while cutting collisions from 19.0% to 8.4%.

Hugging Face daily papers·Sep 7, 20:00 UTC · 9d agoAI research

28

Learning to Solve Hard Problems in RL for LLMs by Never Giving Up

Paper introduces Never Give Up adaptive sampling, fixing RL's 'Matthew Effect' where compute is wasted on easy problems and hard problems see little improvement.

Researchers identify a 'Matthew Effect' in reinforcement learning for LLMs, where RL yields large gains on easy problems but minimal improvement on hard ones because compute is misallocated. They propose Never Give Up (NGU), an adaptive sampling method that keeps generating samples for a problem until one is correct, using asynchronous RL to filter easy problems cheaply and concentrate compute on hard ones. NGU improves performance per compute on the Deepscaler math benchmark and iteratively solves the Manufactoria coding task where standard GRPO with per-test reward fails.

Hugging Face daily papers·Sep 10, 20:00 UTC · 6d agoAI research

45

Generalized Agent Iteration: One Formal Framework for Iterative Policy Improvement and Recursive Self-Improvement

Generalized Agent Iteration formally unifies iterative policy improvement and recursive self-improvement, defining axes that distinguish anchored, goal-drifting, and self-referential agents.

The paper proposes Generalized Agent Iteration (GAI), a formal framework that models learning as a cycle of agent evaluation and agent improvement, defining the agent as a configuration of modifiable components. Two dials—whether the improving mechanism is part of the agent and whether the evaluation standard is grounded outside it—separate generalized policy iteration (GPI) from recursive self-improvement (RSI) and classify systems as anchored, goal drift, or fully self-referential. The framework places existing systems on shared axes and makes defects of recursive self-improvement statable one condition at a time.

Hugging Face daily papers·Sep 10, 20:00 UTC · 6d agoAI research

30

We have a year to fix security everywhere

Blog post warns that cheap open-weight GLM 5.3-flash, once abliterated, could enable mass AI-driven vulnerability exploitation, urging industry-wide patching now.

An essay argues that Z.ai's open-weight GLM 5.3-flash—runnable locally on roughly $6k consumer hardware at 20-45 tokens/second—combined with 'abliterated' variants from groups like DeAlignAI that score 0% on HarmBench-320 puts dangerous hacking capability in nearly anyone's hands. GLM 5.3 scores 84.5% on CyberGym and 54.4% on ExploitBench, versus GPT-6 Astra's 100% and GPT-5.6 Sol's 78.5%, and the author cites evidence of frontier models exploiting real-world infrastructure. The author calls for using LLMs (Project Glasswing, Daybreak) to find and fix vulnerabilities industry-wide before adversaries weaponize cheap open models.

Hacker News · security·Sep 8, 04:48 UTC · 8d agoAI safety & securityHN 278↑ · 292 comments

55

When the prompt becomes the payload: A practical pen-testing guide for GenAI, LLM and RAG applications

CSO Online publishes a practical penetration-testing guide for GenAI, LLM, and RAG applications, covering prompt injection, retrieval poisoning, and tenant isolation testing.

The guide frames LLM applications as attack graphs spanning prompts, retrieval layers, vector stores, tools, identities, and downstream APIs, arguing that conventional web testing misses instruction-vs-data channel risks. It builds on OWASP prompt injection guidance (direct vs. indirect injection) and NIST's 2025 adversarial machine-learning taxonomy, noting that RAG and fine-tuning do not remove injection risk. Recommended practices include documenting trust transitions across components, using canaries and synthetic records to avoid test side effects, running multi-turn and obfuscated injection campaigns, and verifying chains from poisoned documents to observable state changes. It also details testing RAG pipelines via controlled document poisoning across metadata, OCR layers, and code comments, plus cross-tenant isolation checks on retrieved document IDs.

CSO Online·Sep 9, 10:00 UTC · 7d agoAI safety & security1

28

Negative Self-Distillation: Learning to Reason by Avoiding Flaws

Researchers propose Negative Self-Distillation (NSD), a label-free LLM self-improvement method that diverges from self-generated flawed reasoning rather than imitating privileged solutions.

The authors show On-Policy Self-Distillation can degrade complex reasoning by forcing imitation of artificially confident traces built on privileged information, suppressing uncertainty and self-correction. NSD instead generates a question-specific negative condition — such as acting as a 'careless reasoner' — and pushes the model's distribution away from it without ground-truth labels. A dynamic gating mechanism isolates reasoning-critical tokens so gradient updates fix behavioral flaws without damaging foundational linguistic capabilities. NSD consistently outperforms OPSD and other label-free, self-bootstrapping reinforcement learning baselines.

Hugging Face daily papers·Sep 9, 20:00 UTC · 7d agoAI research1

30

Active Adaptation, Not Static Defense: Temporal Dynamics of Preventative Steering in Adversarial Fine-Tuning

Researchers analyze why Preventative Steering protects LLMs against malicious fine-tuning, finding active adaptation drives protection, and propose Progressive Intensity Scheduling.

The paper studies Preventative Steering, a training-time defense that injects undesirable-trait persona vectors during adversarial fine-tuning and removes them at evaluation time. Temporal analysis shows protection emerges from an early compensatory adaptation phase followed by a steady-state phase, with attention output projections acting as the dominant residual-write route for defensive updates. Intervention Delta Preservation experiments show that preserving or reinjecting weight offsets fails to maintain protection, indicating reliance on active adaptation rather than a static defense. The proposed Progressive Intensity Scheduling improves safety robustness on Qwen2.5 and Gemma-3 while reducing harmful trait expression.

arXiv cs.CR·Sep 9, 13:19 UTC · 7d agoAI safety & security1

32

Ask Before You Optimize: Dynamic Pre-Formulation Clarification for Interactive Optimization

Researchers release OR-Clarify, a benchmark testing whether LLM agents ask clarifying questions before formulating optimization models from incomplete requests.

OR-Clarify evaluates pre-formulation clarification in operations research: each task gives a partial problem description, withholds structured hidden slots, and scores agents via bounded interaction with a simulated user, measuring slot recovery, stopping behavior, silent assumptions, and interaction cost. The authors also propose InterOPT, a two-stage framework that identifies formulation-critical gaps to decide when to ask or stop. In choice-based experiments InterOPT substantially outperforms all baselines in exact slot recovery and remains competitive in the open-ended setting.

Hugging Face daily papers·Sep 3, 20:00 UTC · 13d agoAI research1

35

What Else Needs Fixing? Exploring Cost-Effective Test-Time Compute for Revision Propagation in Artifacts Generated Through Conversation

A new benchmark shows LLMs reach 68.3-93% accuracy propagating local revisions across conversationally generated artifacts, with parallel-sample selection most cost-effective.

The paper introduces a benchmark for revision propagation: when users request a local change, LLMs must identify dependencies and update all affected parts of an artifact generated through conversation, where context lives in the chat history. Nine revision methods, including sequential reflection and parallel sampling variants, were evaluated on gpt-oss-20b/120b, gpt-5.4-mini, and qwen3.5-9b/27b/122b. Baselines scored 68.3-93% accuracy, and selecting among three parallel samples via LLM-based or medoid selection improved accuracy by 2.2-9.7% as the most cost-effective test-time compute strategy. Code and dataset are released.

Hugging Face daily papers·Sep 2, 20:00 UTC · 14d agoAI research

30

EvoSafeHarness: Evolving Model- and Domain-Specific Harnesses for Securing Agents

EvoSafeHarness auto-synthesizes per-model, per-domain safety harnesses, cutting prompt-injection attack success on AgentDojo to 0.0% at 82.8% utility.

EvoSafeHarness is an optimization framework that synthesizes deployable safety harnesses for frozen LLM agents in a target domain, jointly searching natural-language policies and executable code logic guided by model behavior, domain specifications, and adversarial review. On DecodingTrust-Agent it reduces average attack success rate from 45.6% to 10.0% at a 3.3-point utility cost, and on AgentDojo reaches 82.8% utility at 0.0% ASR, twice CaMeL's utility at that operating point. It keeps mean ASR below 20% under adaptive PAIR attacks and transfers unchanged to unseen AgentDyn suites. The analysis finds domain semantics determine required safety relations while model and runtime behavior determine enforcement points.

Hugging Face daily papers·Sep 4, 20:00 UTC · 12d agoAI safety & security

48

Sociotechnical Aspects of Tor Relay Rejection

User study and simulations of Tor's relay end-of-life rejection policy find operators favor it; network churn affects anonymity more than EoL exclusions.

The study examines the Tor Project's 2019 end-of-life policy that rejects outdated relays, which constitute a notable fraction of consensus weight. A user study of 26 relay operators found they generally view the policy favorably despite limited awareness, though operational practices occasionally exclude newly installed relays. Historical-data-driven simulations show the policy gives adversaries only marginal advantage, with network churn exerting a more pronounced effect on user anonymity. Analysis of four exclusion rounds shows a minority of rejected relays typically account for over 50% of the security provided by all excluded relays, informing EoL policy recommendations.

arXiv cs.CR·Sep 14, 08:12 UTC · 2d agoResearch

25

From Reweighting to Rewriting: Unlocking the Intervention Effects of Influential Samples in Training Data Attribution

Study shows rewriting responses of influence-selected training examples shifts LLM behavior more strongly than reweighting the same samples.

The paper examines training data attribution, arguing that influence functions identify high-leverage examples whose value goes unrealized under conventional weight-based reweighting interventions. It introduces influence-guided response rewriting, which replaces the responses of influence-selected examples with behavior-aligned or behavior-opposed supervision while keeping instructions fixed, tested across four open-weight LLMs using epistemic abstention as the primary testbed. Rewriting produces stronger, more persistent, and bidirectional behavioral shifts, including on safety refusal, while reweighting the same examples yields weak, inconsistent effects. The results motivate intervention-aware evaluation of TDA methods.

Hugging Face daily papers·Sep 1, 20:00 UTC · 15d agoAI research

30

Finding Nemo(Claw): Networking Issue Allows for LLM Poisoning in OpenClaw

A networking flaw in Nvidia tooling lets attackers reach OpenClaw's local model server unauthenticated via the Ollama API, enabling persistent LLM poisoning.

Dark Reading reports that a networking issue in Nvidia's tooling can give attackers unauthenticated access to the local model server through the Ollama API. From there, attackers can poison the model used by the OpenClaw agent, creating persistent corruption of agent behavior. The finding highlights exposed local model servers as a security risk for self-hosted AI agent stacks.

Dark Reading·Aug 25, 19:50 UTC · 22d agoAI safety & security

48