Skip to content
NewA calmer, faster ZeroHour
StoryAI tools & infrainfo

DeepSeek open-sources DeepInfer inference stack for V4.1-Flash

4 sources · 4 articlesFirst reported Updated ()1 · 1 read
What's new

First merged summary.

Merged summary · x-ai/grok-4.7 · rewritten as coverage arrivesSummaries by Vani AI

DeepSeek has released DeepInfer under Apache 2.0, while KV-cache management and load balancing remain internal.

DeepSeek has open-sourced DeepInfer, the inference framework behind DeepSeek-V4.1-Flash, under the Apache License 2.0. The Decoder reports it builds on vLLM with optimized kernels and multi-chip distributed processing, and that the company is also releasing deepseek-recipe so self-hosted token processing matches the official API. Latent Space says DeepInfer supports speculative decoding and structured output on NVIDIA and Huawei Ascend, and that DeepSeek claims 155 trillion tokens a day, 96% cache hits, 400 ms average time to first token, and 230 tokens per second. KV-cache management and load balancing remain tied to internal infrastructure, according to DeepSeek. vLLM says the project was built primarily on vLLM and that decode throughput was within 6% of DeepSeek’s published GB200 results.

Key facts
  • Apache License 2.0; repository is github.com/deepseek-ai/DeepInfer.
  • deepseek-recipe aims to match official API token processing for self-hosting.
  • DeepSeek says future model code will also be open source.
  • Claimed serving: 155 trillion tokens a day and 96% cache hits.
  • Recommended shapes: eight-accelerator prefill nodes and 64–128-accelerator decode supernodes.
  • Coding generation cited at 300 tokens per second; $38 per accelerator-hour off-peak.
Explain this AI news with Vani AIWhat was announced, what is new and why it matters, in a new chat

Timeline

dated events in the reporting · oldest first
  1. Oct 20, 2026Hacker News links DeepInfer repository; 52 points, 6 comments. · Hacker News
  2. Oct 20, 2026Decoder reports Apache 2.0 release and deepseek-recipe. · The Decoder
  3. Oct 21, 2026Latent Space adds production claims and withheld KV-cache code. · Latent Space

What each outlet adds

beyond the shared facts
  • The DecoderApache 2.0, vLLM kernels, deepseek-recipe, and future open-source model code.
  • Latent SpaceThroughput claims, NVIDIA and Ascend support, incomplete KV-cache release.
  • Hacker NewsLinks github.com/deepseek-ai/DeepInfer; 52 points and 6 comments.
Vendors
DeepSeek
Products
DeepInfervLLMdeepseek-recipe
Organizations
DeepSeekOpenAIAnthropic
AI models
V4.1 FlashV4.1-FlashDeepSeek-V4.1-Flash

Coverage

oldest first · one row per article · UTC
  1. · 5d ago Hacker News · AIFirst report

    DeepSeek open-sources its DeepInfer inference engine

    The Hacker News item only links to deepseek-ai/DeepInfer and gives no release notes.

  2. · 5d ago MarkTechPost

    DeepSeek open-sources DeepInfer, inference stack for V4.1-Flash

    The feed contained only the headline; license, hardware targets, and release notes were not in the text.

  3. · 5d ago The Decoder

    DeepSeek open-sources DeepInfer inference engine under Apache 2.0

    DeepInfer serves V4.1-Flash on top of vLLM, and deepseek-recipe mirrors official API token processing for self-hosting.

  4. · 4d ago Latent Space

    DeepSeek open-sources DeepInfer, its production inference engine

    DeepSeek claims 155 trillion tokens a day and 96% cache hits; KV cache and load balancing are not fully released.