ZeroHour
DashboardNewsCVEsExploitsIncidentsAIBriefAnalyticsEntitiesWatchlistIOCsAsk the feedSourcesFeeds & APIKiosk mode

Search: “model serving”

3,090 items

Searchkeywordsmart Sortlatesthotimportancerelevance
DomainAllCyberAIWindow24h3d7d30dAllIn the wildImportant onlyShow duplicates
CategoryAllExploit / PoC 526Ransomware 380Vulnerability 363Malware 322Policy & legal 318Industry 220Data breach 187Threat actor 167Phishing & fraud 144AI tools & infra 115AI safety & security 72AI industry 58Model release 49Advisory 47
SeverityAllcritical 1606high 389medium 362low 382info 351

FlashVector: Agent for Hierarchical Model Serving Stack Optimization

FlashVector agent optimizes all layers of Unity's ad-serving stack, delivering up to 2x model-server throughput and 1.98x latency speedup in production.

FlashVector is an agentic system that optimizes performance across GPU kernels, ML framework computation graphs, model servers, and on-demand feature processing. Deployed in Unity's Vector advertising platform, it achieved up to 2x model-server throughput increase, 1.98x latency speedup, and 1.6x feature-store throughput gain. Optimizations spanned NVIDIA Triton's C++ codebase and the Python feature transformation service, demonstrating extensibility beyond single-kernel tuning.

arXiv cs.AI / cs.LG / cs.CL · 1d agoAI research

The Coding-Agent Trap: When a "Free" LLM Endpoint Is the Adversary, (Mon, Aug 31st)

A SANS honeypot caught a real coding-agent session routed to a rogue "free" LLM endpoint, exposing a Windows user's transcript and tool outputs.

A SANS analyst describes how an internet-exposed inference honeypot was discovered, relabeled with sought-after model names like DeepSeek, and enrolled in infrastructure serving "free" LLM backends. On 2026-08-30 an opencode terminal coding agent sent an 88-message, 224 KB transcript 210 times in 91 seconds via a China Unicom relay, exposing directory listings, tool outputs and read file portions. The analyst frames tool-enabled agents treating model endpoints as trusted control planes as a novel risk — a "rogue model endpoint" that could request tool executions on the user's machine.

SANS Internet Storm Center · 15d agoAI safety & security1

IBIB: A Protocol for Measuring Enterprise AI Systems by Serving Route, Not Model Identifier

IB2 protocol scores enterprise AI systems by serving route with reliability-inclusive scoring; serving-arm choice moved one score from 77.38 to 82.54.

The protocol has three parts: a gold-blind capability-binding preflight verifying a route can execute the evaluation contract, a reliability-inclusive first-pass scoring rule, and structurally score-blind adjudication. Its reference instantiation uses 128 locked tasks and 987 assertions over document, spreadsheet, chart, tool, and database work, released as procedure and schemas rather than an exposed corpus. Across eleven systems, two complete runs on identical weights later failed distinct binding-gate predicates, four of seven suites saturate within a six-system band driven by governed database work and multi-tab joins, and excluding failed responses from denominators changes the point ordering. Serving-arm choice shifted one declared revision and precision from 77.38 to 82.54, though arms differed in access mode, harness generation, and the tool-call parser.

arXiv cs.AI / cs.LG / cs.CL · 7d agoAI research1

Agnes-AI/Agnes-3.0-Flash — new model trending #30 on Hugging Face

Agnes AI releases open-weight Agnes-3.0-Flash Preview, a 33B multimodal model with 262k-token context under Apache 2.0.

Agnes AI released Agnes-3.0-Flash Preview, an open-weights multimodal checkpoint with 33B parameters and a 262,144-token context window under Apache 2.0. The model supports text, image, and video understanding, tool calling, and adjustable reasoning effort. The repo clarifies this preview checkpoint is distinct from the production/API Agnes 3.0 Flash model, which uses a different configuration with a 1M-token context window. Reported reference results include IFBench 74.20 and SciCode 38.08 against peers such as Qwen3.6-35B-A3B, Kimi K2.5, and MiniMax M3.

Hugging Face trending models · 5d agoModel release

HoneyRoute: Honeypot-Model Routing for Adversarial LLM Serving

HoneyRoute detects malicious LLM serving requests and diverts them to a honeypot model, reaching F1 0.911 with 38 ms median added latency.

HoneyRoute is an inference-serving layer pairing a streaming router (a frozen 0.8B embedding backbone with per-domain MLP heads) with a dual-implementation honeypot and an analysis loop that converts trapped interactions into attacker fingerprints for router retraining. On a production trace plus a seven-domain attack corpus it matches 96% of a two-tier guard-LLM cascade's F1 at 1/385th of its latency with 0% evasion under 13 adversarial transformations. Diverting malicious traffic cuts production token consumption under GCG-suffix flooding by 97.8%, and loop training raises detection F1 to 0.933.

arXiv cs.CR · 8d agoAI safety & security

Moonshot serves Claude instead of Kimi and collects exchanges for model training

Moonshot AI reportedly served users Anthropic's Claude instead of its own Kimi model and collected the exchanges for model training.

A Hacker News post claims Moonshot AI served user requests with Anthropic's Claude rather than its in-house Kimi model, and stored those exchanges to train future models. The item is a social media link with no technical details, drawing 42 points and 31 comments. No confirmation from Moonshot or Anthropic is included in the source text.

Hacker News · AIupdated · 4d agofirst · 5d agoAI industry 15 sourcesHN 42↑ · 31 comments1

Perplexity Details Its GPU Embedding Stack: How Ivy, Tulip and ROSE Serve pplx-embed

Perplexity details its GPU embedding serving stack (Ivy, Tulip, ROSE), which reuses LLM prefill/decode kernels, CUDA graphs, and LazyTensors to cut launch overhead.

Perplexity engineers published a deep dive on the serving infrastructure behind pplx-embed, used across Perplexity Search and its API platform. The stack comprises Ivy (Rust HTTP gateway), Tulip (gRPC scheduling and batching), and ROSE (Runtime-Optimized Serving Engine), which reuses LLM prefill and decode kernels rather than running a separate embedding engine. Optimizations include whole-model CUDA graphs with lazy capture and a LazyTensor abstraction that overlaps CPU batch preparation with in-flight GPU work. Benchmarks are reported against vLLM v0.22.0 in BF16, with FlashAttention 4 generally fastest but FlashInfer 3 winning on Qwen-based models at very long sequence lengths.

MarkTechPost · 10d agoAI tools & infra1

[AINews] Fal’s H3 Max Live breaks the infinite videogen barrier

Fal post-trained MiniMax H3 into a 'Max' variant with 35x-faster inference, enabling faster-than-realtime AI video generation and infinite streams.

Fal post-trained MiniMax's H3 model into a 'Max' variant and optimized it for its in-house inference engine, achieving roughly 35x the speed of the official endpoint. The optimization enables faster-than-realtime video generation, demonstrated by an infinite interactive AI-generated stream productized by levels.io. The roundup also notes Meta Muse Code's general availability with an SDK, open DeepSeek-V4-Flash-Vision-Exp weights, GLM-5.3-Flash's strong agentic cost/performance rankings, and Tencent's 770B-parameter Hy4 Preview MoE with 49B active parameters.

Latent Space · 15d agoAI industry

deepseek-ai/DeepSeek-V4-Flash-Vision-Exp — new model trending #10 on Hugging Face

DeepSeek released DeepSeek-V4-Flash-Vision-Exp, its first experimental multimodal model, with large multimodal agent benchmark gains over V4-Flash-0731.

DeepSeek AI published DeepSeek-V4-Flash-Vision-Exp, an experimental multimodal model built on the DeepSeek-V4-Flash architecture with added visual modules and continued training. It scores 83.9 on Terminal Bench 2.1 and 36.5 on ApexBench Pass@1 versus 26.2 for DeepSeek-V4-Flash-0731, while remaining comparable to Opus-4.8 on several benchmarks. The MIT-licensed repository ships a tokenizer, OpenAI-style and TXT prompt encoding, and a minimal PyTorch inference implementation, with vLLM and SGLang deployment recipes.

Hugging Face trending models · 16d agoModel release1

Realtime-Venus: A full-duplex interaction system with asynchronous delegation

Realtime-Venus introduces two 9B full-duplex interaction models (Omni and Audio) that outperform Gemini 3.1 Live and GPT-4o on continuation metrics.

Realtime-Venus is a proactive full-duplex interaction system built on two separately trained 9B models: Realtime-Venus-Omni for audio-visual interaction and Realtime-Venus-Audio for spoken interaction. A dual-loop runtime lets foreground interaction continue while Realtime-Venus-Harness asynchronously executes background reasoning and tool tasks. Realtime-Venus-Omni leads on six of eight video benchmarks, including StreamingBench (70.2%), OVO-Bench (64.7%), and Daily-Omni (81.3%), while Realtime-Venus-Audio tops MMAU (78.0%) and MMAU-Pro (63.2%). On Full-Duplex-Bench v1.5, Realtime-Venus-Audio handles 75% of interruptions and exceeds Gemini 3.1 Live and GPT-4o on all three continuation metrics.

Hugging Face daily papers · 4d agoAI research

So you want to use OpenRouter?

Simon Willison highlights OpenRouter pitfalls: provider routing makes the same model endpoint behave inconsistently across backends.

Simon Willison discusses problems, raised by Mohamed Moustafa, with relying on OpenRouter's single API endpoint that automatically routes to the most cost-effective provider. Different backend providers run different serving software with different optimizations and settings, so requests to the same endpoint can behave in different ways. Some providers lack vision capability for vision models, and the reasoning effort option can be processed differently across providers.

Simon Willison · 4d agoAI tools & infra

Programmable World Model

Programmable World Model decouples executable world-state evolution from video generation, reaching 94% Count Accuracy and 98% State Accuracy on new CombatStateBench.

An agent translates natural-language instructions into executable programs specifying entity states and transition rules, executed by a lightweight engine that maintains an explicit, persistent global world state including off-screen entities. State-augmented 3D oriented bounding boxes are deterministically compiled into pixel-aligned spatiotemporal conditioning signals for a pretrained video model acting as the generative renderer. On the new CombatStateBench benchmark it achieves 94% Count Accuracy and 98% State Accuracy, substantially outperforming existing interactive video world models.

Hugging Face daily papers · 7d agoAI research

AI spend per employee slumped at top firms in August — summer doldrums or a warning sign?

Ramp data across 70,000 companies shows August AI adoption grew just 0.4% while per-employee spend at top-spending firms fell nearly 10% to $7,205 amid declining token prices.

Ramp's spending data shows 56% of its customers paid for AI products in August, up only 0.4% month-over-month, with spend per employee in the top 1% of firms down almost 10% to $7,205. Average token costs have dropped to $0.68 per million tokens from a 2026 peak of $1.15 in March after price cuts by OpenAI and Anthropic, and the labs have not yet offset the cuts with volume, with customers shifting to cheaper models like ChatGPT 5.6-Terra and Claude Sonnet. The US Census Bureau's broader survey shows only 22% of businesses report using AI, suggesting Ramp's tech-heavy client base overstates adoption, while only 6.4% of AI-spending businesses used model-serving or inference platforms in August.

TechCrunch · AI · 7d agoAI industry

Agentic Visual Generation: From Generative Models to Agentic Control

Researchers propose an L0-L4 control taxonomy for agentic visual generation, classifying controllers from fixed conditioning to experience-adaptive decision-making.

This paper proposes a taxonomy for agentic visual generation organized by what the controller can directly control in the generation process, rather than by planning depth, tool count, or model size. Levels range from L1 Conditioning Control through L2 Execution Control, L3 Outcome-Adaptive Control, and L4 Experience-Adaptive Control, with L0 Fixed Support denoting systems without a deployed decision-making controller. The framework is applied across image, video, editing, 3D, world, slide, and user-interface generation to map how controller capabilities and mechanisms have evolved across the field.

Hugging Face daily papers · 10d agoAI research

Off Guard: Breaking LiteLLM from authentication bypass to cloud compromise

Wiz found LiteLLM auth bypass (CVE-2026-59822) and post-auth RCE (CVE-2026-59821) chainable to cloud compromise; the bypass is in CISA KEV with in-the-wild exploitation.

Wiz scanned roughly 3,074 internet-facing LiteLLM deployments and found 9.6% accepted the default master key sk-1234 or required no authentication, making post-auth attacks effectively pre-auth. The MCP endpoint accepts any Bearer token and grants a valid session (CVE-2026-59822), confirmed exploited in the wild via honeypots and added to CISA's Known Exploited Vulnerabilities catalog. Custom code guardrails allow post-auth root-level RCE via exec(compile(...)) (CVE-2026-59821), while pass-through endpoints lack URL validation, enabling cloud credential theft in post-auth scenarios. All assigned vulnerabilities have been patched; the research was presented at DEF CON 34.

Wiz Blog · 7d agoExploit / PoC in the wildCVE-2026-59822CVE-2026-598211

Autoencoder Is All You Need: Profiling and Detecting Malicious DNS Traffic

Palo Alto Unit 42 details an autoencoder-based method that profiles DNS traffic to detect C2 and malicious domains, blocking ~374,000 malicious DNS requests daily.

Unit 42 built an RNN-based autoencoder that compresses DNS traffic time series into fixed-dimensional 'DNS profiles' for each domain and device. Downstream classification, clustering, and anomaly detection modules flag suspicious domains, capturing 170 emerging suspicious domains in May 2024. Signatures block roughly 374,000 malicious DNS requests daily and run in the Advanced DNS Security service, with detections shared to Advanced URL Filtering. Case studies link DNS traffic patterns to C2 beaconing, dynamic DNS abuse, and DNS tunneling for data exfiltration.

Palo Alto Unit 42 · Aug 17, 2026Research

Implementation of Machine Learning Workflows with NVIDIA cuML, RAPIDS, GPU Benchmarking, Explainability, Clustering, and Model Inference

Hands-on tutorial implements NVIDIA cuML and RAPIDS to GPU-accelerate scikit-learn-style ML workflows with benchmarking, clustering, and inference.

The tutorial demonstrates NVIDIA cuML as a GPU-accelerated machine learning framework, using cuml.accel to speed up unmodified scikit-learn scripts with zero code changes and the native cuML API for CuPy/cuDF interoperability. It benchmarks CPU versus GPU implementations of PCA, K-Means, nearest-neighbor search, logistic regression, random forests, and DBSCAN on datasets up to 200,000 samples with 64 features. It also builds GPU pipelines with UMAP, t-SNE, and HDBSCAN, validates GPU-generated SHAP explanations, uses the FIL library for forest inference, and covers model serialization and GPU/CPU portability.

MarkTechPost · 3d agoAI tools & infra

Speculative Decoding in vLLM on AMD GPUs

vLLM benchmarks speculative decoding on AMD Instinct MI300X and MI355X GPUs across five drafting methods including EAGLE-3 and native MTP.

The vLLM project documents draft-and-verify speculative decoding support for AMD GPUs via ROCm, comparing native MTP, Gemma 4 MTP, EAGLE-3, DFlash, and DSpark drafting approaches. Output-token throughput effects varied with drafting method, proposal length, model family, draft checkpoint, workload, and acceptance behavior. The post also covers how to enable each method plus practical tuning and observability considerations.

Hacker News · AI · 9d agoAI tools & infraHN 141↑ · 52 comments1

[AINews] NVIDIA buys HuggingFace for $13B, as OpenAI publishes their HF incident retro

Z.ai released open-weight GLM-5.3-Flash (320B/18B active, 1M context, MIT) while Nvidia confirmed buying Hugging Face for $13B.

Z.ai formally launched GLM-5.3-Flash, the model previously previewed as Ox Alpha: 320B total parameters with 18B active, a 1M-token context window, natively multimodal, MIT-licensed, and claimed on par with Claude Opus 4.8 on coding. Artificial Analysis scored it 57 on its Intelligence Index at $0.09 per task, roughly 7.5x cheaper than GLM-5.3, and it scored 84.3% on Terminal-Bench 2.1. Nvidia's $13B acquisition of Hugging Face (~80x its $150M ARR) was confirmed, nearly double its initial $7B January offer. The roundup also notes Qwen shipping an impressive Flash model on Chinese chips as part of a broader open-model narrative.

Latent Space · 20d agoModel release1