Search: “Grok 4”

6 stories

Searchkeyword smart Sortlatest hot importance relevance

DomainAll Cyber AIWindow24h 3d 7d 30d AllIn the wild Important only Show duplicates

CategoryAll Exploit / PoC 161 Ransomware 87 Policy & legal 52 Data breach 51 Malware 34 Vulnerability 24 Threat actor 17 Phishing & fraud 14 AI safety & security 8 Model release 6 AI research 6 Advisory 4 AI policy 2 Industry 2

SeverityAll critical 410 high 13 medium 20 low 9 info 17

Real-SWE: Benchmarking AI models on private, real-world, enterprise codebases

Real-SWE benchmark tests coding agents on licensed private enterprise codebases; top model Fable 5.1 resolves only 38.8% of tasks.

Real-SWE is a new benchmark evaluating frontier AI coding agents on tasks drawn from private production codebases licensed from real companies, spanning billing, tax calculation, and cross-service migrations. Fable 5.1 with Claude Code leads at 38.8% resolution rate (pass@1 over eight runs), followed by GPT-6 Astra Codex CLI at 33.8% and Gemini 3.8 Flash Gemini CLI at 31.2%. Tasks use native harnesses and realistic tooling including Docker, Kubernetes, PostgreSQL, Redis, and Linear; median reference solutions edit 11 files versus 6 for DeepSWE and FrontierCode.

Hacker News · AISep 12, 20:25 UTC · 4d agoAI research HN 45↑ · 31 comments1

Can Skills Learned in Games Transfer to Real-World Work?

Good Start Labs trains models in strategy games like 1830 and Diplomacy, showing terminal-agent training transfers to financial research benchmarks.

Good Start Labs, spun out of Every with $3.6M from General Catalyst and Inovia, trains AI models in verifiable strategy games. A 30B model trained as a multi-turn terminal agent in 1830: The Game of Railroads and Robber Barons improved Finance-Agent benchmark performance, while single-turn QA training did not transfer. The founders also co-authored COS-PLAY, a paper on co-evolving LLM decision and skill-bank agents for long-horizon tasks.

Latent SpaceSep 15, 20:11 UTC · 1d agoAI research

Show HN: Pelican-bicycle alternatives (updated for 2026)

Hobbyist benchmark re-runs the pelican-bicycle SVG test on six 2026 frontier models, comparing generation time and API cost per image.

A Show HN post re-runs the classic pelican-bicycle and similar SVG generation tests across six 2026 models: GPT-6 Astra, Claude Fable 5.1, Gemini 3.8 Flash, DeepSeek V4 Pro, Qwen3.8 Max, and Fugu Ultra v2, recording wall-clock time and cost. It also lists 2025 baseline runs with ten models including Claude Sonnet 4.5, GPT-5.2 Pro, and Qwen3-VL-235B-A22B-Thinking. DeepSeek V4 Pro is consistently cheapest ($0.04-$0.10) while Qwen3.8 Max is slowest, taking up to roughly 17 minutes per generation.

Hacker News · securitySep 14, 13:20 UTC · 3d agoAI research HN 24↑ · 4 comments

An AI CAPTCHA solver talked itself out of the right answer

Bern researchers solved rotation CAPTCHAs in 0.006 seconds with classical computer vision, while Gemini 3.1 Pro needed 67 seconds and overruled correct tool answers.

Researchers at Bern University of Applied Sciences built a script using 1970s circle-detection math and signal matching that solved rotation CAPTCHAs in 0.006 seconds, scoring 10/10 on real-world puzzles. Frontier models fared poorly: Gemini 3.1 Pro scored 7/10 taking 67 seconds, while GPT-4o and Grok scored 1/10. When given the script's correct answer as a tool, Gemini overruled it and lost a fifth of its score; models could verbally describe targets, such as identifying a cyan ring, but could not produce accurate click coordinates. The paper also notes these no-JavaScript CAPTCHAs reduce tracking, leaving only shape-matching tasks classical vision solves easily.

Help Net SecuritySep 2, 00:00 UTC · 15d agoAI research1

Playing log(N)-Questions over Wikipedia Abstracts: Communication Efficiency Between Paired Frontier Models

Six frontier models play a two-agent log(N)-Questions game; Claude Opus 5 lags with 28/68 wins while the top five are near-tied.

The study evaluates six frontier models on a two-agent game where a questioner must identify one of N Wikipedia lead paragraphs in exactly log2 N yes/no questions, run over 408 games at $363 total API cost. Claude Opus 5 wins 28 of 68 games versus 45-56 for GLM-5.3, GPT-5.6 Sol, Grok 4.6, Gemini 3.8 Flash, and Kimi K3. Pooled top-five win rates decline with set size (r=-0.973) and fit win = p^(log2 N) with per-round reliability p=0.928, and information per question correlates with win rate at r=+0.88.

arXiv cs.AI / cs.LG / cs.CLSep 16, 17:41 UTC · 19h agoAI research 20 sources

VDiff-Bench: A Challenging Benchmark for Fine-Grained Image Difference Identification

VDiff-Bench, a 1,756-question benchmark, shows multimodal LLMs struggle with fine-grained image-difference identification, scoring as low as 8.7% on low-level changes.

VDiff-Bench is a multiple-choice benchmark of 1,756 four-way questions over image pairs covering 10 change categories including position, motion, color, texture, OCR/text and illumination, with curated hard negatives. Evaluation of 11 state-of-the-art open- and closed-source MLLMs shows fine-grained visual comparison remains brittle: 7-8B-scale open-source models score 52.5-70.6% on semantic changes but only 8.7-33.3% on low-level changes like noise and texture. Notably, Grok 4.3 shows a sharp performance drop on noise and texture differences, falling behind large open-source models like Kimi K2.5 and K3.

Hugging Face daily papersSep 4, 20:00 UTC · 12d agoAI research