ZeroHour
DashboardNewsCVEsExploitsIncidentsAIBriefAnalyticsEntitiesWatchlistIOCsAsk the feedSourcesFeeds & APIKiosk mode

Search: “agent benchmarks”

11 stories in the last 7d

Searchkeywordsmart Sortlatesthotimportancerelevance
DomainAllCyberAIWindow24h3d7d30dAllIn the wildImportant onlyShow duplicates
CategoryAllAI research 41AI safety & security 24Model release 14AI industry 13Industry 10Tools 6AI tools & infra 3Research 1
SeverityAllcritical 0high 3medium 3low 0info 106

Iris-mini and Iris-pro are the strongest open-weight search agents in their class

Chinese lab AllSpark releases Iris-mini (35B) and Iris-pro (397B) open-weight search agents claiming best-in-class results on BrowseComp and other research benchmarks.

AllSpark's paper introduces Iris-mini (35B parameters, built on Qwen3.6-35B-A3B) and Iris-pro (397B parameters, built on Qwen3.5-397B-A17B), both with 256,000-token context windows. Iris-pro scores 88.6 on BrowseComp, 85.1 on BrowseComp-ZH, 92.9 on DeepSearchQA, and 56.4 on Humanity's Last Exam; Iris-mini reaches 82.2, 84.8, 86.9, and 52.3 respectively. Training tasks are reverse-engineered from web link structure, filtered by a judge model, and refined via alternating SFT and reinforcement learning ('SFT-RL climbing') against live web search. Weights are available on Hugging Face, and the Iris Harness with agent loop, tools, and all four benchmarks is on GitHub.

The Decoder · 5d agoModel release1

[AINews] Jev: a “System One Model” that only decides/classifies/routes/scores — >100x faster, >200x cheaper than small frontier LLMs

TypeSafe launches Jev, an RLCD-trained decision model claiming 20-200x faster, 40-400x cheaper classification than frontier LLMs, alongside Gemini 3.8 Live and Neon.

TypeSafe's Jev is a 'System One' decision model trained with RLCD, claiming 20-200x faster and 40-400x cheaper classification and routing than frontier LLMs with free output tokens and no hallucinated text. Google launched Gemini 3.8 Live and 3.8 Live Extended Thinking, supporting 97 languages and async tool calls, debuting #1 on Artificial Analysis' speech-to-speech index at 82.6. Periodic Labs' Neon is a ~1T-parameter XRD analysis model trained with RL on proprietary lab data using 1,300 H200s, lifting FrontierXRD success from 2.7% to 55.3% and beating GPT-6 Astra at lower inference cost.

Latent Spaceupdated · 7h agofirst · 2d agoModel release 2 sources2· 1 read

GPT-6 Astra: Pokemon champion in 18 hours, potato farmer after one Creeper mishap

OpenAI's GPT-6 Astra beats prior models on agentic game benchmarks, finishing Pokemon FireRed in 18 hours and scoring 62.7% on ARC-AGI-3.

GPT-6 Astra completed Pokemon FireRed in 18h 12m versus 96h 35m for GPT-5.6 Sol, and scored 62.7% on ARC-AGI-3 via the standard interface versus 7.78% for GPT-5.6 Sol and about 30% for Claude Opus 5. In a Vals AI Minecraft run driven through general computer use (screen, mouse, keyboard), the agent built a Nether portal within three hours and the 141-hour run ended after a Creeper explosion triggered risk-averse potato farming. ARC Prize attributes the leap to the model converting observations into compact symbolic rules it develops itself, and it also completed Portal, Fallout 2, Fallout 3, RimWorld, and Factorio: Space Age runs.

The Decoder · 23h agoModel release 2 sources1

Atria Dawn: The Dawn of Agentic Superintelligence

Atria Dawn Preview, an agentic foundation model trained on verifiable experiences, tops five of 16 research and engineering benchmarks.

Atria Dawn Preview is a foundation agentic language model for scientific research and engineering workflows, trained via a Verifiable Experience Pipeline connecting tool-mediated interactions to executable environments and externally verified outcomes. Across 16 benchmarks spanning research, engineering, and digital work it is competitive with frontier agents and achieves the highest reported score on five of them. The release includes a human-AI collaboration case study analyzing 769 task records from 56 participants, where about one-third of completed AI-assisted tasks were rated infeasible without AI and agents frequently proposed methods and implemented revisions while humans retained final decisions.

Hugging Face daily papers · 4d agoModel release1

Google Releases Gemini 3.8 Live and 3.8 Live Extended Thinking for Production Grade Voice Agents

Google launches Gemini 3.8 Live and Extended Thinking speech-to-speech models for production voice agents, topping speech-to-speech benchmarks.

Google released Gemini 3.8 Live and Gemini 3.8 Live Extended Thinking, native speech-to-speech models for real-time voice agents, available hosted via the Gemini Live API and AI Studio. Extended Thinking ranks #1 on Artificial Analysis' Speech-to-Speech Quality Index with 82.6, scores 68.6% on τ-Voice, 35.1% on Sierra's τ-Voice-banking, and 97.7% on Big Bench Audio. The models support asynchronous function calling, near-real-time visual context, alphanumeric precision, and 97 languages, priced at $0.005/min audio input and $0.018/min audio output. All generated audio carries Google DeepMind's imperceptible SynthID watermark.

MarkTechPost · 2d agoModel release 3 sources

[AINews] DeepSeek v4.1-Flash: 763B-P8B-D16B novel causal Encoder–Decoder architecture with vision marks the Return of the Whale

DeepSeek released V4.1-Flash, an open-weight 763B-parameter model with a novel causal encoder-decoder architecture, 1M context, vision input, and MIT license.

DeepSeek launched V4.1-Flash, an open-weight MIT-licensed model using a novel causal encoder-decoder architecture with 763B total parameters and asymmetric active parameters: 8B for prefill and 16B for decode. It supports 1M-token context and text+image input, priced at $0.30 per 1M input and $1.20 per 1M output tokens with a 50% off-peak discount. Artificial Analysis scored it 40 on its Intelligence Index, above DeepSeek V4 Pro 0813, and Vals ranked it the #1 open-weight model ahead of Kimi K3. Baseten shipped day-0 support and Ollama began rolling it out to paid subscribers.

Latent Space · 6d agoModel release1

prism-ml/Ternary-Bonsai-2-27B-gguf — new model trending #29 on Hugging Face

Prism ML released Ternary-Bonsai-2-27B, a 27B ternary-weight model derived from Qwen3.8-27B that runs full reasoning in ~5.95 GB GGUF.

Prism ML published Ternary-Bonsai-2-27B, an Apache 2.0 open-weights model derived from Qwen3.8-27B (27.36B parameters, hybrid attention ~75% linear, 262K-token context). Weights are end-to-end ternary {−1, 0, +1} with g128 FP16 scales at 1.72 bits/weight, shrinking the model from ~54 GB FP16 to 5.95 GB (PTQ1_0) or 7.21 GB (PQ2_0) while reportedly retaining 98.2% of FP16 quality (84.78 average across 14 thinking-mode benchmarks). It ships as GGUF for llama.cpp (CUDA, Metal, CPU) with custom ternary kernels, plus an MLX 2-bit build for Apple Silicon and an optional Q8_0 vision tower pack.

Hugging Face trending models · 1d agoModel release 2 sources1

Alibaba Qwen Releases Qwen3.8-Omni-Flash: A 1M-Context Omni-Modal Model Built Around Agentic Audio-Video Understanding and Tool Use

Alibaba's Qwen team launched Qwen3.8-Omni-Flash, an API-only omni-modal model with 1M-token context and agentic audio-video understanding and tool use.

Qwen3.8-Omni-Flash accepts text, images, audio, and video and returns text, built on the Qwen3.8-Flash-Next architecture with a 1M-token context window and thinking enabled by default. Qwen reports a 25%+ average improvement over Qwen3.5-Omni-Plus across 29 evaluations, with OmniVideoBench rising from 63.4 to 67.8 while using about 45.7% fewer tokens via coarse-to-fine agentic perception. It is hosted on QwenCloud, Model Studio, and Qwen Studio at $0.15/$0.47 per 1M input/output tokens; no open weights were released, but Qwen open-sourced Qwen-MM-Plugins under Apache-2.0.

MarkTechPost · 5h agoModel release 3 sources

XingChen-AGI/Xing4.0-29B-A4B — new model trending #30 on Hugging Face

China Telecom's XingChen-AGI released Xing4.0-29B-A4B, an open-weights 29B-parameter MoE model (4B active) with 256K context, trained entirely on Ascend NPUs.

XingChen-AGI, the AI unit of China Telecom and successor to the TeleChat series, published Xing4.0-29B-A4B weights in Transformers format. The MoE model has 29B total parameters with 4B activated per token, 40 layers, MLA attention with 64 routed experts, and supports 256K context extensible to 512K. It is the first model at this scale trained entirely on Ascend NPU hardware with MindSpore, reporting roughly 96% training throughput gains from co-optimization. Benchmarks include 75.0 on SWE-bench Verified, 57.5 on Terminal-Bench 2.1, and 90.0 on AIME2026, and it deploys via vLLM, SGLang, and KTransformers.

Hugging Face trending models · 2d agoModel release

Cognition Releases SWE-2: A Kimi K3 Post-Trained Coding Model That Matches Fable 5.1 on FrontierCode at 64% Lower Cost

Cognition released SWE-2, an RL post-trained coding model from Kimi K3, scoring 50.0% on FrontierCode 1.1 Main and available only inside Devin.

Cognition released SWE-2, its most capable coding model, post-trained with reinforcement learning from Moonshot AI's 2.8T-parameter Kimi K3 base. It scores 50.0% on FrontierCode 1.1 Main, within 1 point of Fable 5.1 at 64% lower cost, and RL reportedly adds 5-6 points over the K3 base on many benchmarks. It is the first Cognition model with selectable reasoning-effort levels all trained in a single RL run using Pareto-slope-matched cost penalties. There are no open weights and no standalone API; it runs only inside Devin (Desktop, CLI, with Web and Fusion rolling out), free for paid tiers through October 10, 2026.

MarkTechPost · 5d agoModel release1

Jev: New frontier model 40-400x cheaper and 20-200x faster

TypeSafe AI launches Jev, an early-access 'System One' model delivering calibrated structured outputs claimed 40-400x faster and cheaper than LLMs.

TypeSafe AI, founded by former OpenAI researcher Diogo Almeida, released its first 'System One Model' called Jev in early access. Jev forgoes string generation and is trained with Reinforcement Learning for Calibrated Decisions (RLCD) to produce type-safe structured values with calibrated probabilities. The company claims 70-500ms response times (40-200x faster), input pricing of $0.042 per million tokens, and free output tokens via a parallel sampling architecture. Target use cases include AI-powered workflows, real-time applications, and verification/guardrail tasks.

Hacker News · security · 2d agoModel releaseHN 59↑ · 11 comments