30
IdeaAMBIG: Benchmarking Implementation-Critical Gaps in Research-Idea Specifications
IdeaAMBIG benchmark with 660 instances measures whether LLMs can spot and fix underspecified research-method details for faithful implementation.
Researchers introduce IdeaAMBIG, a benchmark of 660 evidence-grounded instances (163 real-world gaps from reproducibility reports and GitHub issues, 497 controlled synthetic gaps) built from papers, codebases, and reproduction artifacts. It evaluates codification-readiness assessment, defect localization, and clarification action generation. Across 13 LLMs, the best model achieved only 9.6% Macro Defect Recovery Rate on real-world instances but 80.6% clarification success when given the annotated defect. An oracle study showed gold resolutions raise the codification-ready rate from 14% to 98%, identifying defect localization as the main bottleneck.
28
IdeaAMBIG: Benchmarking Implementation-Critical Gaps in Research-Idea Specifications
IdeaAMBIG benchmark of 660 specification-gap instances shows LLMs localize implementation-critical research gaps poorly, with best model at 9.6% defect recovery.
IdeaAMBIG is a benchmark of 660 evidence-grounded instances evaluating whether research-method specifications provide enough information for faithful implementation: 163 real-world gaps from reproducibility reports and GitHub issues plus 497 controlled synthetic gaps. It tests codification-readiness assessment, defect localization, and clarification action generation across 13 LLMs. The best model achieves only a 9.6% Macro Defect Recovery Rate on real-world instances, though 80.6% clarification success when given the annotated defect, and an oracle study shows gold resolutions raise codification-ready rates from 14% to 98%. Defect localization emerges as the main bottleneck across all evaluated models.
30
47
30
55
30
47
30
30
30
60
60
35
CDN Tsunami Attack Abuses HTTP/3 Translation for Up to 350x DoS Amplification
Researchers disclosed two DoS attacks exploiting CDNs' HTTP/3-to-HTTP/1.1 translation to amplify low-bandwidth requests up to 350x against origin servers.
Researchers named the two techniques HTTP/3 Bandwidth Amplification (HBA) and HTTP/3 Connection Amplification (HCA), exploiting the gap where CDNs speak HTTP/3 to browsers but only HTTP/1.1 to origins. All six tested CDNs (Alibaba, Baidu, Cloudflare, CloudFront, Fastly, Tencent) were susceptible to HBA, and five to HCA, with Cloudflare immune to HCA because it buffers full requests before opening backend connections. Attackers using under 5 Mbps forced over 100 Mbps at origins, and the 350x factor applies only to Alibaba, Baidu, and Tencent, which support the QPACK dynamic table. Testing identified 42,330 potentially vulnerable subdomains out of 151,685 hosted on the six providers across the Tranco Top 1M.
65
30
42
42
30
55
42
30
42
45
60
60
60
30
[AINews] Jev: a “System One Model” that only decides/classifies/routes/scores — >100x faster, >200x cheaper than small frontier LLMs
TypeSafe launches Jev, an RLCD-trained decision model claiming 20-200x faster, 40-400x cheaper classification than frontier LLMs, alongside Gemini 3.8 Live and Neon.
TypeSafe's Jev is a 'System One' decision model trained with RLCD, claiming 20-200x faster and 40-400x cheaper classification and routing than frontier LLMs with free output tokens and no hallucinated text. Google launched Gemini 3.8 Live and 3.8 Live Extended Thinking, supporting 97 languages and async tool calls, debuting #1 on Artificial Analysis' speech-to-speech index at 82.6. Periodic Labs' Neon is a ~1T-parameter XRD analysis model trained with RL on proprietary lab data using 1,300 H200s, lifting FrontierXRD success from 2.7% to 55.3% and beating GPT-6 Astra at lower inference cost.
75
45
60
30
30
55
35
60
47
55
⚡ Weekly Recap: BadCam Attack, WinRAR 0-Day, EDR Killer, NVIDIA Flaws, Ransomware Attacks & More
60
Week in review: Cisco patches SD-WAN 0-day, unpatched Microsoft Exchange Server flaw exploited
60