Search: “influence-functions”

30 stories

DomainAll Cyber AIWindow24h 3d 7d 30d AllIn the wild Important only Show duplicates

CategoryAll

SeverityAll critical 0 high 0 medium 0 low 0 info 0

Smart search ranks by meaning as well as keywords (one row per story, last 45 days).

From Reweighting to Rewriting: Unlocking the Intervention Effects of Influential Samples in Training Data Attribution

Study shows rewriting responses of influence-selected training examples shifts LLM behavior more strongly than reweighting the same samples.

The paper examines training data attribution, arguing that influence functions identify high-leverage examples whose value goes unrealized under conventional weight-based reweighting interventions. It introduces influence-guided response rewriting, which replaces the responses of influence-selected examples with behavior-aligned or behavior-opposed supervision while keeping instructions fixed, tested across four open-weight LLMs using epistemic abstention as the primary testbed. Rewriting produces stronger, more persistent, and bidirectional behavioral shifts, including on safety refusal, while reweighting the same examples yields weak, inconsistent effects. The results motivate intervention-aware evaluation of TDA methods.

Hugging Face daily papersSep 1, 20:00 UTC · 15d agoAI research

Necessary or Sufficient? Evaluating LLM Explanations With Behavioural Evidence

New framework tests whether LLM-cited explanation factors are necessary or sufficient, finding weak correlation across Claude, GPT, and Gemini models.

An arXiv paper introduces black-box intervention tests measuring whether factors LLMs cite in their explanations are necessary or sufficient for their outputs in agent oversight workflows. Across eight models from the Claude, GPT, and Gemini families, Spearman correlations between cited rankings and measured influence ranged from 0.349-0.354 (advisor recommendation) to 0.431-0.580 (prompt monitoring). Uncited factors scored above the lowest cited factor in up to 57.6% of advisor responses, showing cited top-three factors do not reliably identify the most influential inputs.

arXiv cs.AI / cs.LG / cs.CLSep 4, 17:37 UTC · 12d agoAI safety & security1

Toward an Empirical Probabilistic Risk Manifestation Model of Organizational Cybersecurity in SMEs

Empirical study of 22 SME security assessments builds a probabilistic risk model and shows assessments can be cut 24-45% while retaining most critical findings.

Researchers analyzed 281 validated security findings from 22 real-world SME cybersecurity assessments conducted over two years via a pro bono university clinic. They derived an empirical Risk Manifestation Model linking eight organizational security functions to two exposure conditions, five attack mechanisms, and six outcome categories, using probability propagation to identify dominant risk pathways. The dominant pathway runs from asset exposure to credential compromise to unauthorized access, stable under leave-one-organization-out analysis. Retaining six functions reduces assessment burden by 24% while preserving 97% of critical findings; five functions cut burden 45% while preserving 89% of critical findings.

arXiv cs.CRSep 14, 01:15 UTC · 2d agoResearch

Show HN: LLM Attention Visualization

A developer released a browser-based tool that visualizes which past tokens influence each LLM output token using aggregated, value-weighted attention scores.

A Show HN project presents a React application built on Transformers.js that renders per-token attention influence by aggregating attention weights scaled by value-vector magnitudes across all attention heads and layers. To expose internal tensors, the author instrumented the ONNX computation graph, hosted a modified model on Hugging Face, and pre-generated prompts to avoid long model downloads in the browser. Demos with a 600-million-parameter model show how verbatim copying draws heavily on source tokens and how single outputs blend information from multiple phrases.

Hacker News · AISep 8, 16:59 UTC · 8d agoAI tools & infra HN 41↑ · 9 comments

Engineered Persuasion: Evaluating Personalized Pretexts in LLM-Generated Spear Phishing

A study of 180 US workers found each LLM phishing personalization level raised click-intention odds by 28%, but credibility depends on context fit.

The arXiv paper evaluates how personalized pretexts in LLM-generated spear phishing affect perceived credibility, using 180 US working adults across 1,436 evaluations of emails with four cumulative personalization levels, from workplace context to shared-project details. Convincingness rose 2.40 points per level in sensitivity analysis and click-intention odds increased 28% per level, while non-clickers shifted toward deleting rather than reporting. Qualitative coding showed details matching the recipient's role and routines supported credibility, whereas incorrect, vague, or channel-inappropriate details raised suspicion. The authors argue personalization effectiveness depends on pretext fit, with implications for workplace security training.

arXiv cs.CRSep 3, 19:21 UTC · 13d agoResearch

Can Foundation Models Moderate Online Content? Evaluating Instruction- vs. Example-Driven Policy Operationalization

ModerationBench shows foundation models can nearly triple Bluesky's moderation F1 (0.60 vs 0.22), with instruction- and example-driven guidance performing comparably.

Researchers built ModerationBench, a new benchmark of 4,000 manually annotated in-the-wild posts from Bluesky, to test whether foundation models can reliably operationalize content moderation policies. They systematically compare instruction-driven guidance (reasoning from policy precepts) with example-driven guidance (generalizing from precedents) for Vision-Language Models. Both paradigms achieve comparable peak effectiveness, and foundation models nearly triple the F1 of Bluesky's deployed moderation system on Random Posts (0.60 vs 0.22).

arXiv cs.AI / cs.LG / cs.CLSep 9, 16:30 UTC · 7d agoAI research1

Causal Foundation Models

A paper introduces causal foundation models (CFMs): pretrained networks that estimate treatment effects on new datasets via in-context learning without fine-tuning.

Causal foundation models (CFMs) apply the foundation-model paradigm to causal inference, replacing bespoke per-problem estimator pipelines with networks pretrained once at scale. CFMs estimate causal quantities such as the average treatment effect on entirely new datasets through in-context learning, without model updates. The work serves as a practical introduction to the emerging area, covering background in causal inference and machine learning and including example code and Jupyter notebooks.

Hugging Face daily papersSep 1, 20:00 UTC · 15d agoAI research

Generative Marketing Mix Modeling: A Causal Inference Framework Linking GEO and GEM to Business Impact

Paper proposes Generative Marketing Mix Modeling to causally estimate Generative Engine Optimization and Marketing effects on business outcomes.

The authors develop GMMM, a causal inference framework for measuring how often users see and notice a firm's name in generated answers, which standard marketing data ignore. For GEO it combines repeated generated answers with question counts, shares of generative-system usage and notice probabilities; for GEM it uses sponsored placement records with notice probabilities. The framework compares expected business responses under alternative treatment sequences, establishes identification conditions, and is evaluated on simulated product-recommendation answers in English and Japanese.

arXiv cs.AI / cs.LG / cs.CLSep 10, 17:57 UTC · 6d agoAI research

When LLM judges agree, should we believe them?

Amazon ICML paper uses Ising models to correct correlated LLM-judge votes, beating accuracy-weighted panels by 9-14%.

Amazon Science describes an ICML paper, "Dependence-aware label aggregation for LLM-as-a-judge via Ising models," addressing how correlated judge outputs inflate majority-vote confidence. The unsupervised method models pairwise dependence between judges, learning both reliability and similarity without human reference labels. Tested on relevance, toxicity, and summarization tasks with 10 judge models at temperature zero, it outperformed accuracy-weighted voting by 9% to 14%.

Hacker News · AISep 14, 16:29 UTC · 2d agoAI research HN 41↑ · 23 comments

Why judgment is emerging as cybersecurity’s defining skill

CyberScoop op-ed argues CISOs should grant AI autonomy based on reversibility and blast radius rather than model confidence, and measure analyst overrides of AI recommendations.

A CyberScoop op-ed contends that as AI takes over analysis and recommendations in security operations, human judgment about context, reversibility and blast radius becomes the defining skill. The author argues autonomy decisions should rest on how reversible and impactful an action is rather than model confidence, citing examples such as patching vendor-certified medical devices and a service account whose 3 a.m. login spikes were normal quarterly-close activity. It also urges leaders to measure analyst approvals, edits and rejections of AI recommendations, and review latency, instead of automation rates or mean time to resolution.

CyberScoopSep 4, 10:00 UTC · 12d agoIndustry1

Evaluating Time-Series Foundation Models and Multimodal Dietary Context for CGM Forecasting

Study finds zero-shot time-series foundation models underperform on CGM forecasting; fine-tuned Chronos-Bolt cuts RMSE up to 18.4% and dietary context adds signal.

The paper evaluates time-series foundation models for continuous glucose monitoring forecasting across eight public datasets covering Type 1 diabetes, Type 2 diabetes, and non-diabetes populations. Under a unified protocol, zero-shot foundation models did not consistently outperform baselines like Elastic Net and PatchTST, but lightweight fine-tuning did, with fine-tuned Chronos-Bolt reducing RMSE by 6.5%-18.4% in the T1D cohort and 8.6%-18.2% in the non-diabetes/T2D cohort. A residual-based fusion framework adding dietary context from CGMacros reduced overall RMSE by about 3% and postprandial RMSE by about 15% versus CGM-only baselines.

arXiv cs.AI / cs.LG / cs.CLSep 10, 17:43 UTC · 6d agoAI research

5 useful things you'll learn in my new post-training textbook (shipping now!)

Nathan Lambert's new RLHF and post-training LLM textbook covers PPO, GRPO, GSPO, CISPO and related techniques, freely available online.

Nathan Lambert's book 'Reinforcement Learning from Human Feedback: Aligning and Post-training LLMs' is now shipping from Manning. It covers policy-gradient algorithms including PPO, GRPO, GSPO, CISPO, and RLOO, plus loss aggregation, truncated importance sampling, asynchronous RL systems, and post-training topics like rejection sampling, outcome reward models, and on-policy distillation. The book is freely available online with a 12-hour course, codebase, and exercises.

InterconnectsAug 10, 13:02 UTC · Aug 10, 2026AI research

Fine-tuning a 350M Model for Better Structured Outputs in 100 GRPO Steps

Hugging Face guide fine-tunes a 350M-parameter model with 100 GRPO steps to improve structured output reliability.

A Hugging Face blog post demonstrates fine-tuning a 350M-parameter model using GRPO (Group Relative Policy Optimization) with TRL over 100 training steps. The stated goal is more reliable structured outputs from small language models. No article body was available, so details beyond the title are limited.

Hugging Face BlogSep 3, 00:00 UTC · 13d agoAI tools & infra

GraphProfiler: Source-Linked Sensitive Attribute Inference via Personal Knowledge Graphs

GraphProfiler links LLM attribute inferences to source posts via personal knowledge graphs, enabling targeted redaction of privacy-leaking content.

GraphProfiler represents a user's post history as a source-linked personal knowledge graph where nodes and edges trace back to originating posts, making LLM-based attribute inference auditable. It reaches 86.7% attack success rate on the eight-attribute SynthPAI benchmark and 84.6% on PANDORA, within two points of strong text-only baselines, while citing supporting evidence for over 98% of predictions. Ablation experiments show removing cited posts reduces attack success substantially more than removing random posts, supporting targeted privacy mitigation.

arXiv cs.CRSep 11, 05:13 UTC · 5d agoResearch1

You Can't Prefer Emotions You Don't Sample: Intensity Undershoot in DPO-Tuned LLMs

Study quantifies DPO-tuned LLMs undershooting requested emotional intensity, tracing the gap to candidate-pool extremity rather than conditioning format.

Conditioning an instruction-tuned LLM on continuous valence-arousal targets yields gain of only 0.26 for valence and 0.13 for arousal on Llama-3.1-8B, far below faithful control of 1.0. The authors attribute undershoot to neutral-heavy preference corpora like EmoBank and candidate pools lacking extreme affect, leaving DPO without extreme exemplars. Uniform target coverage with a hotter candidate pool raises valence gain to 0.40 on Llama-3.1-8B and 0.44 on Qwen3-8B, with modest in-distribution cost; arousal gains remain unstable across seeds.

arXiv cs.AI / cs.LG / cs.CLSep 7, 17:47 UTC · 9d agoAI research

LimiX-2: A Contextual Mechanism Network Towards General Structured-Data Intelligence

LimiX-2 scales Contextual Mechanism Networks pretrained via context-conditional masked modeling, beating tabular foundation models on TabArena, TALENT, and BCCO benchmarks.

LimiX-2 is a new tabular model in the LimiX family, developed through model and data scaling guided by previously established scaling laws under the Contextual Mechanism Networks (CMNs) paradigm. It is pretrained with Context-Conditional Masked Modeling (CCMM) on synthetic datasets generated by structural causal models spanning diverse graph structures, functional mechanisms, and observation processes. It outperforms dataset-specific models and tabular foundation models on TabArena, TALENT, and BCCO, and its feature attention encodes direct causal relationships, enabling accurate causal skeleton recovery.

arXiv cs.AI / cs.LG / cs.CLSep 15, 17:30 UTC · 1d agoAI research1

Quantifying the Engagement Trap: Impact of Short-form Video Recommender Systems on Users with ADHD

A 302-participant study finds engagement-optimized short-form video recommenders cause disproportionate time blindness and distress for users with ADHD.

Researchers ran a stratified Prolific study with 302 participants comparing short-form video recommendation experiences with and without ADHD. Participants with ADHD reported significantly higher time blindness, post-usage regret, and emotional distress despite perceiving recommendations as similarly relevant. The paper proposes neurodiversity-aware, human-centered design interventions to mitigate these algorithmic harms.

arXiv cs.AI / cs.LG / cs.CLSep 7, 17:38 UTC · 9d agoAI research

Perturbation Probing: A New Diagnostic for the Fragility of LLM Safety

Unit 42 research shows LLM safety refusals concentrate in a thin neural layer, motivating external, multi-layered AI security controls.

Palo Alto Networks Unit 42 introduces Perturbation Probing, a diagnostic technique for measuring the fragility of LLM safety mechanisms. The research finds that safety refusal behavior is localized within a thin neural layer, implying small perturbations can undermine built-in refusals. The authors argue this motivates external, multi-layered security defenses on top of model-internal safety training.

Palo Alto Unit 42Aug 28, 22:00 UTC · 19d agoAI safety & security

Strategic Threat Intelligence for Better Decision-Making

Recorded Future outlines strategic threat intelligence: non-technical, OSINT-driven analysis for board-level decisions and the skill sets needed to produce it.

Recorded Future's blog post explains that threat intelligence spans strategic, tactical, operational, and technical categories, focusing on strategic intelligence. It defines strategic intelligence as non-technical, risk-based analysis produced mainly from OSINT sources on demand for C-suite and board audiences. The piece stresses the sociopolitical and business expertise required and urges boards to ask trend-focused questions rather than demanding specific attack predictions.

Recorded FutureAug 17, 00:00 UTC · Aug 17, 2026Industry

CausalArena: Benchmarking Causal Discovery in the Foundation Model Era

Researchers introduce CausalArena, a unified benchmark revealing that causal discovery rankings shift substantially across structural causal model families and protocols.

The paper presents CausalArena, a unified and evolvable benchmark for causal discovery combining synthetic structural causal models, semantically grounded operational SCMs, formula-grounded scientific SCMs, and public real-world datasets. Experiments across classical, neural, and pretrained causal discovery foundation models show large ranking shifts between benchmark regimes. The authors identify pretraining-evaluation overlap and benchmark diversity as central evaluation challenges.

arXiv cs.AI / cs.LG / cs.CLSep 10, 17:53 UTC · 6d agoAI research

One Symptom, Three Levers: A Critical Review of On-Policy Self-Distillation

A review paper frames on-policy self-distillation collapse as governed by three levers: token weighting, privileged information, and guidance decay.

The paper critically reviews On-Policy Self-Distillation (OPSD), where a language model trains on its own generations scored token-by-token by a teacher conditioned on privileged information such as reference solutions or environment feedback. It identifies collapse, the progressive narrowing of producible reasoning paths, as the dominant failure mode and analyzes it through three levers: signal weighting, the nature of privileged information, and teacher dynamics. The review is restricted to mathematical reasoning, reports no new experiments, and offers a shared vocabulary separating settled findings from disputed ones.

Hugging Face daily papersAug 25, 20:00 UTC · 22d agoAI research

ToolLoop: Closed-Loop Tool-Use Data Synthesis via Decomposed Generation and Dynamic Self-Feedback

ToolLoop introduces a closed-loop synthetic data framework whose 11K examples lift a 4B model to 86.40% on BFCL tool-use evaluation.

ToolLoop decomposes tool-use data synthesis into function-name sampling, backward derivation of user queries, and forward derivation of tool calls, with dynamic self-feedback at each stage. This shifts the paradigm from generate-then-filter to generate-verify-refine, reducing inefficient and imbalanced synthetic data. A 4B model trained on 11K synthetic examples reaches 86.40% accuracy on BFCL non-reasoning mode (86.07% in an Isolate variant excluding BFCL-overlapping functions) and 72.1% on ACEBench using only 18.3% of baseline training data.

arXiv cs.AI / cs.LG / cs.CLSep 8, 17:26 UTC · 8d agoAI research1

The Frontier AEO Tracker: What Astra Chooses (and every other frontier model, and what you can do about it)

Latent Space launches an AEO tracker scoring 7 frontier models' product recommendations across 161 categories, revealing generational bias flips.

Latent Space built a tracker measuring Answer Engine Optimization by running 6 prompt variations across 7 frontier models with search enabled over 161 product categories, scoring first choices, alternatives, mentions, and anti-recommendations. It found 28 categories with a universally dominant primary choice and observed soft biases, such as models favoring their own lab's coding agents. Analysis of Anthropic's Sol→Astra and Opus→Fable generations showed newer models consulting fewer sources and being less likely to change answers when questions are paraphrased.

Latent SpaceSep 7, 21:32 UTC · 9d agoAI research

Before You Poll with LLMs: A Deliberative Diagnostic Framework

Deliberative diagnostic shows all five tested frontier LLMs misrepresent human belief shifts after arguments, with GPT-5.1 reversing on outgroup questions.

The Deliberative Polling Diagnostic Framework compares human and LLM persona belief shifts after identical informational interventions, using data from America in One Room (526 personas, 72 questions). All five frontier models tested failed uniquely: GPT-5.1 exhibited partisan reversal (80% on outgroup vs 26% on policy questions), Gemini 2.0 Flash, Claude Sonnet 4.5 and Llama 3.3 70B overshot at 5-7x human magnitude, and DeepSeek V3 showed near-zero change (rigidity). The authors term the underlying signature 'self-sycophancy', conformity to the model's internal persona stereotype rather than reasoning from provided information.

arXiv cs.AI / cs.LG / cs.CLSep 14, 16:41 UTC · 2d agoAI research

Distill Globally, Adapt Locally: Reasoning Distillation and Product-Type Test-Time Training for Scalable Trade-Up Recommendation

A distillation framework compresses LLM reasoning into a 15.5M-parameter trade-up recommendation model reaching AUC 0.941 with product-type test-time training.

The paper targets trade-up recommendation, which identifies higher-quality alternatives that preserve customer purchase intent. A retrieval-augmented few-shot LLM teacher generates labels and rationales that supervise a compact embedding-pair classifier; at inference the 15.5M-parameter student uses only two precomputed 768-dimensional embeddings with no LLM calls. On 8,352 annotated pairs, label-only training scored AUC 0.912, reasoning distillation reached 0.924, and product-type test-time training lifted it to 0.941 with average precision 0.940. The distilled student is roughly 5,000x faster and 10,000x cheaper than direct LLM inference on a 100K-pair proxy catalog.

arXiv cs.AI / cs.LG / cs.CLSep 4, 17:08 UTC · 12d agoAI research

Why AI raises the stakes for exposure validation

Fal.Con 2026 commentary argues AI accelerates vulnerability discovery and exploitation, making evidence-based exposure validation essential for defender prioritization.

CSO Online reports on the exposure-validation theme at CrowdStrike's Fal.Con 2026 conference, where CEO George Kurtz described AI as the new cyber battlefield and emphasized AI red teaming and continuous security. The piece argues that as AI speeds up vulnerability discovery and exploitability analysis on both sides, teams must determine which exposures are actually exploitable in their environments—chained weaknesses, credential abuse, lateral movement, privilege escalation—rather than chasing theoretical risk. It points readers to Horizon3's conference perspective.

CSO OnlineSep 11, 21:49 UTC · 5d agoIndustry

Discovery Foundation Models: Toward Open-Ended Discovery Intelligence

Paper defines Discovery Foundation Models with seven coupled capabilities for open-ended discovery, demonstrated via Zetema and GALILEO systems.

The paper formulates Discovery Foundation Models (DFMs) as general-purpose model systems for open-ended discovery, supporting seven coupled capabilities spanning problem discovery, formulation, representation construction, hypothesis formation, intervention, evidence-grounded revision, and continual improvement. It instantiates the framework with Zetema, which couples explicit research-state dynamics, verification and experimental gating, and cross-task Discovery Skill evolution. The framework is grounded with GALILEO, a real therapeutic-discovery system combining dry-lab reasoning with robotic wet-lab experimentation in a closed physical discovery loop; code is released on GitHub.

Hugging Face daily papersupdated Sep 14, 17:56 UTC · 2d agofirst Sep 13, 20:00 UTC · 3d agoAI research 2 sources

TIER: Threat Implicitness Benchmark for Evaluating LLM Safety Behaviors

TIER benchmark shows LLM safety behaviors shift gradually across threat implicitness levels, with jailbreaks exposing the largest robustness gaps.

The TIER benchmark evaluates LLM safety behaviors across four risk domains and four threat levels, from explicit harmful requests to sophisticated jailbreaks, using a six-label behavior scale and two independent LLM judges. Experiments on six open-weight LLMs show safety behaviors evolve gradually across threat levels rather than flipping from refusal to compliance. Models with similar Attack Success Rates can exhibit distinct response distributions, arguing for behavior-aware safety evaluation.

arXiv cs.CRSep 4, 13:13 UTC · 12d agoAI safety & security

Introducing Unit 42’s Attribution Framework

Unit 42 releases its Attribution Framework, a systematic method using Diamond Model and Admiralty scores to attribute activity clusters to named threat actors.

Palo Alto Networks' Unit 42 introduced a structured framework for threat actor attribution built on the Diamond Model of Intrusion Analysis and Admiralty reliability/credibility scoring. The framework tracks activity at three levels: activity clusters (named CL-STA, CL-CRI, CL-UNK, or CL-MIX), temporary threat groups, and named threat actors using the constellation naming schema. Analysts score evidence across TTPs, tooling, malware code, OPSEC, infrastructure, timelines, and victimology to decide when to merge or elevate clusters, avoiding premature group naming.

Palo Alto Unit 42Aug 17, 10:52 UTC · Aug 17, 2026Research

Searching for New Physics with Reinforcement Learning

Researchers apply reinforcement learning to identify SMEFT operators explaining particle physics anomalies, reproducing and improving known CDF W-mass results.

The paper introduces a reinforcement learning method to search the large Standard Model Effective Field Theory (SMEFT) operator space for explanations of measurement anomalies. It was validated on the CDF W-mass anomaly, reproducing and improving known results, then applied to a harder multi-anomaly scenario. RL efficiently navigates complex loop-level operator correlations that bias human-driven phenomenological analysis.

arXiv cs.AI / cs.LG / cs.CLSep 9, 16:07 UTC · 7d agoAI research