ZeroHour

Search: “false positives”

23 items

AWS puts AI vulnerability detection to the test, and false positives pile up

AWS publicly released its Deception Benchmark (14,822 samples) showing leading AI models falsely flag 41-99% of safe code as vulnerable.

AWS released its Deception Benchmark publicly, containing 14,822 samples across 16 programming languages and more than 70 CWE categories, with 9,695 scored samples split into 6,988 code-level and 2,707 environment-gated challenges. AWS evaluated 12 models from five providers using single-turn prompts and found none met its production bar of below 10% for both false-positive and false-negative rates. With direct prompting, models caught nearly all real vulnerabilities but incorrectly flagged 41% to 99% of safe code, with precision between 52% and 71%. Asking models to prove exploitability reduced false positives by 17 to 74 percentage points but raised false-negative rates to 7-44%, with models struggling most when external controls like Kubernetes Network Policies blocked apparent exploits.

Help Net Security · 3d agoAI research

FlowBalance: Verifier-Grounded Self-Improvement from On-Policy Reasoning Experience

FlowBalance is a verifier-grounded self-improvement method that beats FlowRL on Qwen3-4B and Qwen3-8B math reasoning while improving training stability.

FlowBalance calibrates dense self-guidance scores with verifier-derived group advantages: guidance is retained on positive-advantage trajectories, reversed on negative-advantage trajectories, and disabled when rollout groups show no outcome preference. The method exponentially reweights a reference policy via trajectory balance, with guarantees including within-group contrast preservation and a minimum-change reverse-KL characterization. On mathematical reasoning it outperforms FlowRL on Qwen3-4B and Qwen3-8B, trains faster and more stably, avoids direct OPSD's response-length collapse, and shows higher correct-strategy diversity on AIME24.

Hugging Face daily papers · 14d agoAI research

SlipSense: Multimodal Tactile Learning for Low-Latency and Generalized Slip Detection

SlipSense fuses a 32x32 piezoresistive array and MEMS accelerometer to detect robotic grip slips within 23.1 ms, generalizing zero-shot across platforms.

SlipSense is a multimodal tactile slip-detection framework built on TacV5, a sensor combining a 32x32 piezoresistive array at 240 Hz and a 3-axis MEMS accelerometer at 8 kHz. It performs modality-specific encoding, intra-sensor fusion, and cross-modal attention with causal temporal prediction at 240 Hz. On a 1.4-million-frame dataset spanning 37 objects it achieves 96.7% Macro F1 with a false-positive rate below 1.6%, detecting 76% of slip events within 23.1 ms. Trained solely on UMI data, it transfers zero-shot to a Tesollo dexterous hand across unseen objects, sensor units, and platforms.

arXiv cs.AI / cs.LG / cs.CL · 2d agoAI research

Training a 3.8B LLM to 0.384 CORE for $998 – Hugo Vergnes

Independent developer Hugo Vergnes trained a 3.8B-parameter Llama-style model to 0.384 CORE on 65B tokens for $998 in 43 hours on rented B200s.

Hugo Vergnes trained little-lm, a 3.848B-parameter decoder-only LLM, on 65.3B tokens in 43 hours for $998 using rented NVIDIA B200s, scoring 0.384 on the CORE benchmark and beating nanochat d32 (0.310) at similar cost. The Llama-style architecture uses RMSNorm, RoPE, GQA with 24 query and 8 KV heads, relu-squared MLPs, QK-norm, and ResFormer-style value embeddings that account for 19% of parameters. Key wins included the Muon optimizer for matrix parameters, a trapezoidal learning-rate schedule with linear cooldown, FP8 training plus vocabulary padding for roughly 33% throughput gains, and the ClimMix dataset over FineWeb-Edu. The project, inspired by Karpathy's nanochat, was built as a config-driven YAML framework for small LLM training.

What researchers learned about building an LLM security workflow

Oslo and FFI researchers show structured agentic workflows lift LLM alert-triage accuracy from 0% to about 93% on malicious cases.

Researchers at the University of Oslo and the Norwegian Defence Research Establishment tested GPT-5-mini, Claude 3 Haiku, Qwen3:30B, and Gemma 3:27B on alerts from the AIT Log Data Set V1.1; given only alert descriptions and log summaries, all four models correctly flagged zero percent of true-positive cases involving reconnaissance, brute-force logins, and initial access. Wrapping the same models in a workflow with constrained SQL queries over Suricata logs, an evidence summarizer, and a verdict stage with revision loops raised malicious-case accuracy to an average of 93 percent, with GPT-5-mini identifying every malicious case across 100 runs. The authors flag it as a proof-of-concept on one synthetic scenario and note models skewed conservative on benign alerts, with GPT-5-mini marking every benign case uncertain.

Help Net Security · 24d agoAI research1

Hierarchical NeRF with JAX3D for Volumetric Rendering, Novel-View Synthesis, and 3D Reconstruction

MarkTechPost tutorial implements a hierarchical NeRF in JAX using jax3d volume-rendering primitives for novel-view synthesis and 3D reconstruction.

The tutorial builds an end-to-end hierarchical Neural Radiance Field using JAX, Flax, Optax, and jax3d's volume-rendering functions (sample_along_rays, volume_rendering, sample_piecewise_constant_pdf). It implements positional encoding, skip connections, separate coarse and fine networks, and view-direction conditioning with hierarchical importance sampling. Training uses JAX JIT compilation, Adam optimization, exponential learning-rate decay, and gradient clipping. Evaluation covers PSNR, depth and opacity visualization, 360-degree rendering, and marching-cubes geometry extraction.

MarkTechPost · 3d agoAI research

VDiff-Bench: A Challenging Benchmark for Fine-Grained Image Difference Identification

VDiff-Bench, a 1,756-question benchmark, shows multimodal LLMs struggle with fine-grained image-difference identification, scoring as low as 8.7% on low-level changes.

VDiff-Bench is a multiple-choice benchmark of 1,756 four-way questions over image pairs covering 10 change categories including position, motion, color, texture, OCR/text and illumination, with curated hard negatives. Evaluation of 11 state-of-the-art open- and closed-source MLLMs shows fine-grained visual comparison remains brittle: 7-8B-scale open-source models score 52.5-70.6% on semantic changes but only 8.7-33.3% on low-level changes like noise and texture. Notably, Grok 4.3 shows a sharp performance drop on noise and texture differences, falling behind large open-source models like Kimi K2.5 and K3.

Hugging Face daily papers · 12d agoAI research