Search: “refusal”

Knowing What Not to Answer: Selective Non-Compliance in Vision-Language Models

Researchers introduce KoNA, a benchmark exposing vision-language models' failures at selective non-compliance, plus fine-tuning that improves refusal and abstention accuracy.

KoNA is a benchmark for evaluating selective non-compliance in vision-language models across five categories: False Premise, Visual Inaccessibility, Universal Unknown, Task Feasibility and Safety. It tests both query-level and component-level non-compliance using paired single and compound queries, and evaluations across diverse VLMs show models often fail to refuse, correct or abstain appropriately, with failures worsening on compound queries. Fine-tuning VLMs on KoNA examples substantially improves non-compliance accuracy while largely maintaining performance on fully answerable tasks.

Hugging Face daily papers·Sep 3, 20:00 UTC · 13d agoAI research1

GPT-6 Astra pilots a surveillance drone and runs a business on its own

GPT-6 Astra outperforms Claude Fable 5.1 on Vending-Bench and becomes the first model to beat the human-AI baseline on all five Drone-Bench subtasks.

Andon Labs tested OpenAI's GPT-6 Astra on two agent benchmarks: Vending-Bench 2, where Astra averaged $15,515 running a simulated vending-machine business versus Claude Fable 5.1's $5,422, and Drone-Bench, where models write code for a DJI Tello EDU drone to navigate an office and follow a specific person. Astra is the first model whose best submissions beat the human-AI baseline on all five Drone-Bench subtasks, using a COLMAP and DA3 pipeline with depth filtering for 3D reconstruction. Reliability remains limited, as an average Astra run has only a 2.8 percent chance of passing all five drone steps sequentially. In Vending-Bench Arena, Astra refused a price-fixing proposal from GLM-5.3, while Claude Fable 5.1 participated in an arrangement Andon Labs classified as illegal price-fixing.

The Decoder·Sep 13, 10:52 UTC · 3d agoAI research

1Password open sources a benchmark to stop AI agents from leaking credentials

Help Net Security·Feb 12, 00:00 UTC · Feb 12, 2026AI research

Can Skills Learned in Games Transfer to Real-World Work?

Good Start Labs trains models in strategy games like 1830 and Diplomacy, showing terminal-agent training transfers to financial research benchmarks.

Good Start Labs, spun out of Every with $3.6M from General Catalyst and Inovia, trains AI models in verifiable strategy games. A 30B model trained as a multi-turn terminal agent in 1830: The Game of Railroads and Robber Barons improved Finance-Agent benchmark performance, while single-turn QA training did not transfer. The founders also co-authored COS-PLAY, a paper on co-evolving LLM decision and skill-bank agents for long-horizon tasks.

Latent Space·Sep 15, 20:11 UTC · 1d agoAI research

What Breaks Under Pruning in Smart Homes, and When? Evaluating LLM Degradation Across Architectures and Task Complexity

Pruning study across four LLM architectures finds dense models degrade sharply on smart-home tool calling while MoE models tolerate far more.

Researchers systematically study pruning-induced degradation in smart-home tool calling across four LLMs spanning dense Transformer, dense hybrid, and mixture-of-experts architectures, combining depth, width, hybrid, and expert pruning methods, and evaluate over 19,500 instances from three datasets after post-pruning supervised fine-tuning. Dense models show narrow safe pruning regions followed by sharp degradation, while MoE models tolerate substantially more pruning. Pruning degrades grounded specificity (operation, device, argument, value) before schema-level intent, and aggressive dense pruning can induce systematic over-refusal.

arXiv cs.AI / cs.LG / cs.CL·Sep 15, 17:51 UTC · 1d agoAI research

Why don't machine learning research agents overfit?

Amazon researchers explain why ML research agents avoid benchmark overfitting, attributing generalization to compressibility of successful strategies.

Amazon Science summarizes the paper "What fits (into few tokens) doesn't overfit: Compression and generalization in ML research agents," which investigates why benchmark hill-climbing loops, whether run by human communities or LLM research agents, do not produce rampant overfitting. The explanation formalizes Occam's razor via a counting argument: successful ML strategies are highly compressible, so short descriptions lack room to memorize benchmark data and must capture real structure. LLM-based agents, being resettable and controllable, allow this hypothesis to be tested empirically.

Hacker News · security·Sep 14, 16:32 UTC · 2d agoAI researchHN 21↑ · 3 comments

Real-SWE: Benchmarking AI models on private, real-world, enterprise codebases

Real-SWE benchmark tests coding agents on licensed private enterprise codebases; top model Fable 5.1 resolves only 38.8% of tasks.

Real-SWE is a new benchmark evaluating frontier AI coding agents on tasks drawn from private production codebases licensed from real companies, spanning billing, tax calculation, and cross-service migrations. Fable 5.1 with Claude Code leads at 38.8% resolution rate (pass@1 over eight runs), followed by GPT-6 Astra Codex CLI at 33.8% and Gemini 3.8 Flash Gemini CLI at 31.2%. Tasks use native harnesses and realistic tooling including Docker, Kubernetes, PostgreSQL, Redis, and Linear; median reference solutions edit 11 files versus 6 for DeepSWE and FrontierCode.

Hacker News · AI·Sep 12, 20:25 UTC · 4d agoAI researchHN 45↑ · 31 comments1

MAxBench: A Multinomial Concept Recovery Benchmark

MAxBench evaluates multinomial concept recovery methods, finding affine subspaces steer most reliably but none consistently beats prompting.

MAxBench is a geometry-agnostic evaluation framework for multinomial concept representations in language models, based on sampling from recovered concept representations. It compares 10 localization methods covering 5 geometry types across 6 concepts and 4 models. Findings show affine subspaces steer more reliably than rank-one or linear subspaces due to better non-zero offsets, manifold steering is competitive where applicable, and no method consistently outperforms prompting.

arXiv cs.AI / cs.LG / cs.CL·Sep 11, 17:08 UTC · 5d agoAI research

From Reweighting to Rewriting: Unlocking the Intervention Effects of Influential Samples in Training Data Attribution

Study shows rewriting responses of influence-selected training examples shifts LLM behavior more strongly than reweighting the same samples.

The paper examines training data attribution, arguing that influence functions identify high-leverage examples whose value goes unrealized under conventional weight-based reweighting interventions. It introduces influence-guided response rewriting, which replaces the responses of influence-selected examples with behavior-aligned or behavior-opposed supervision while keeping instructions fixed, tested across four open-weight LLMs using epistemic abstention as the primary testbed. Rewriting produces stronger, more persistent, and bidirectional behavioral shifts, including on safety refusal, while reweighting the same examples yields weak, inconsistent effects. The results motivate intervention-aware evaluation of TDA methods.

Hugging Face daily papers·Sep 1, 20:00 UTC · 15d agoAI research

Import AI 468: 23 RSI ideas; PostTrainBench+; and how trust and transparency interplay with AI racing

Import AI covers 23 IFP policy ideas for automated AI R&D risks and MIT/Columbia's game theory of AI racing slowdowns.

Think tank IFP published 23 policy recommendations across seven categories to help policymakers address risks from increasingly automated AI R&D. MIT and Columbia researchers released 'Racing to Ruin,' a game theory model showing that coordinated slowdowns between rival AI firms hinge on trust and transparency. The newsletter also links a short story on interacting with powerful AI systems.

Import AI·Aug 10, 12:32 UTC · Aug 10, 2026AI research