50
AWS puts AI vulnerability detection to the test, and false positives pile up
AWS publicly released its Deception Benchmark (14,822 samples) showing leading AI models falsely flag 41-99% of safe code as vulnerable.
AWS released its Deception Benchmark publicly, containing 14,822 samples across 16 programming languages and more than 70 CWE categories, with 9,695 scored samples split into 6,988 code-level and 2,707 environment-gated challenges. AWS evaluated 12 models from five providers using single-turn prompts and found none met its production bar of below 10% for both false-positive and false-negative rates. With direct prompting, models caught nearly all real vulnerabilities but incorrectly flagged 41% to 99% of safe code, with precision between 52% and 71%. Asking models to prove exploitability reduced false positives by 17 to 74 percentage points but raised false-negative rates to 7-44%, with models struggling most when external controls like Kubernetes Network Policies blocked apparent exploits.
55
OpenAI Builds ‘Defense Factory’ as AI Agents Gain Ability to Chain Cyber Exploits
OpenAI unveiled a Defense Factory using AI agents to continuously discover, validate, patch, and verify vulnerabilities, warning the defender's window against agentic attackers is shrinking.
OpenAI describes a Defense Factory workflow where AI agents integrate source control, scanners, issue trackers, and secret stores to discover, reproduce, patch, and verify vulnerabilities under human oversight. The approach responds to agentic attackers that can retain knowledge across sessions and chain vulnerabilities into multi-stage attack paths faster than human triage can respond, which OpenAI calls a shrinking defender's window. During an internal security sprint involving 250+ people across 100+ service areas, agents closed 53 urgent or high-priority issues on day one, achieved 90.6% ownership-routing acceptance, cut 37% of findings as duplicates, and produced Codex-generated patches with a 0.53% rollback rate. Runtime validation reduced false positives to 0.81%, and each agent operates in isolated, reproducible environments with a control plane for policy and credentials.
58
30
30
SlipSense: Multimodal Tactile Learning for Low-Latency and Generalized Slip Detection
SlipSense fuses a 32x32 piezoresistive array and MEMS accelerometer to detect robotic grip slips within 23.1 ms, generalizing zero-shot across platforms.
SlipSense is a multimodal tactile slip-detection framework built on TacV5, a sensor combining a 32x32 piezoresistive array at 240 Hz and a 3-axis MEMS accelerometer at 8 kHz. It performs modality-specific encoding, intra-sensor fusion, and cross-modal attention with causal temporal prediction at 240 Hz. On a 1.4-million-frame dataset spanning 37 objects it achieves 96.7% Macro F1 with a false-positive rate below 1.6%, detecting 76% of slip events within 23.1 ms. Trained solely on UMI data, it transfers zero-shot to a Tesollo dexterous hand across unseen objects, sensor units, and platforms.
15
PIDS-Bench: Evaluating Prompt-Injection Detectors Under Over-Defense, Obfuscation, and Distribution Shift
PIDS-Bench shows prompt-injection detectors scoring F1 above 0.98 still misclassify about one-third of external benign security-adjacent prompts, revealing provenance-sensitive over-defense.
PIDS-Bench is a frozen multi-axis benchmark that jointly evaluates prompt-injection detectors on attack detection and benign false-positive behavior at fixed thresholds, spanning in-distribution inputs, hard-benign prompts, obfuscated attacks, and domain/structural distribution shifts. It evaluates seven detectors plus a rule-based lower-bound reference. A detector exceeding F1 = 0.98 on held-out data still misclassifies roughly one-third of an externally-sourced benign security-adjacent subset, and no internal detector reaches F1 >= 0.95 with hard-benign FPR <= 0.10 on the stress distribution. Hard-negative augmentation nearly eliminates over-defense on curated stress inputs but leaves it intact on externally-sourced prompts, a pattern termed provenance-sensitive over-defense.
38
FlowBalance: Verifier-Grounded Self-Improvement from On-Policy Reasoning Experience
FlowBalance is a verifier-grounded self-improvement method that beats FlowRL on Qwen3-4B and Qwen3-8B math reasoning while improving training stability.
FlowBalance calibrates dense self-guidance scores with verifier-derived group advantages: guidance is retained on positive-advantage trajectories, reversed on negative-advantage trajectories, and disabled when rollout groups show no outcome preference. The method exponentially reweights a reference policy via trajectory balance, with guarantees including within-group contrast preservation and a minimum-change reverse-KL characterization. On mathematical reasoning it outperforms FlowRL on Qwen3-4B and Qwen3-8B, trains faster and more stably, avoids direct OPSD's response-length collapse, and shows higher correct-strategy diversity on AIME24.
32
[AINews] Claude Fable/Mythos 5.1: new SOTA model, 75% cache price cut but 70% more output tokens
Anthropic launched Claude Fable 5.1 and Mythos 5.1, claiming new SOTA benchmarks, with 75% cache-read price cut and 1M-token context.
Anthropic released Claude Fable 5.1 and Mythos 5.1 as flagship models for coding and knowledge work, with a 1M-token context window and pricing of $10/$50 per million input/output tokens and cache reads cut 75% to $0.25. Artificial Analysis Intelligence Index scored Fable 5.1 at 66 versus 63 for Claude Opus 5, with HLE at 59.1% and Terminal-Bench v2.1 at 91.4%, though per-task cost rose ~20% due to 1.7x output token usage. Community analysis suggested Fable and Mythos may share underlying weights with different safety/routing behavior, and release notes highlighted Enterprise Frontier Safeguards and zero-data-retention support.
92
30
30
30
30
30