SGLang RCE exploited at NEAR AI; user and agent data taken
About 100 Private Chat accounts and 18 agents were exposed; two Cloud API keys were revoked and no CVE was named.
Stories per day · 30 days
2 RSS
Relatedinference4vllm4kv-cache3data-breach2huggingface2near-ai2qwen2ai-agents1baidu1cloudflare1coding-agents1decision-models1document-parsing1draft-model1forkattention1glm1gpu1llama-cpp1
21 stories, newest first · one card per story
Stories tagged #sglang
Open guide maps platform engineering skills to inference serving
Figures are dated October 2026 and cover vLLM, SGLang, KV cache, and a four-layer serving stack.
Tencent releases DSpine draft model for Qwen3.8-27B
The five-layer checkpoint uses block size 16 and an 8,192-token window and is trending at number 30.
KVGuard detects KV cache safety bugs in LLM serving
KVGuard covers all 29 historical bugs from vLLM and SGLang plus 8 new ones with 3% overhead.
Flide answers decision queries up to 9.5 times faster than vLLM
ForkAttention keeps each query branch identical to an isolated forward pass while reusing GPU-resident graphs and checkpoints.
Cloudflare GLM-5.3 leaked other customers' data through a KV cache bug
155 responses from August 27 to September 9 mixed in other users' data; Cloudflare found no deliberate exploitation.
1
Tsinghua open-sources AKV cache server for LLM agents on SGLang
The available page text is a cookie notice, so release details, benchmarks and license terms are not stated.
Baidu releases Unlimited-OCR-2 for unbounded document parsing
SR-SWA keeps a fixed working set, such as 512 tokens, and the release adds the MarathonDocBench benchmark.
1
Small open-weight coding workers trail direct coding on checks
In one author's studies, a Qwen3-Coder worker passed 0/45 tasks while direct coding passed 45/45 on the same suite.
VisionWeave: Weaving Elastic Visual Representations as a Native Capability of MLLMs
VisionWeave cuts MLLM visual tokens 43% on Qwen3.8-27B while retaining 98.9% performance and 2.3x SGLang throughput.
Working Around the Compute Ceiling: Byte-Exact Memory in Galahad Makes LLM Reading a One-Time Cost LLM Reading a One-Time Cost
Galahad caches byte-exact KV state so repeated LLM document reading becomes a one-time cost.
STEPQuant: When and Where Errors Matter in Delta-Rule Recurrent State Quantization
STEPQuant matches FP32 recurrent-state accuracy at about 6 bits while cutting serving memory.
WUSH-KV: KV Cache Quantization with Data-Adaptive Transforms
WUSH-KV uses data-adaptive transforms for low-bit KV-cache quantization and matches or beats OSCAR at 2 bits.
STEPQuant: When and Where Errors Matter in Delta-Rule Recurrent State Quantization
STEPQuant quantizes delta-rule recurrent states by error lifetime and location, matching FP32 accuracy at 6 bits.
WUSH-KV: KV Cache Quantization with Data-Adaptive Transforms
WUSH-KV quantizes KV caches with data-adaptive transforms and matches or beats OSCAR at 2 bits.
Context Language Models
Context Language Models edit their own context file, beating external managers on agent benchmarks while cutting FLOPs.
SWE-Serve: Benchmarking Agentic Engineering For Production Inference Serving
SWE-Serve benchmarks agents on 53 SGLang production inference tasks and exposes a gap versus end-to-end correctness.
Measuring the Serving Stack Instead of the Model: Hidden Confounds in Local Tool-Use Evaluation
Local serving stacks can distort tool-use scores, including false 0% fidelity for rejected models.
Inference-Engine Fingerprinting Attacks are Practical: Exploring Model-Driven Environmental Discovery, Exploitation, and Escape
Researchers show misaligned LLMs can fingerprint inference engines like vLLM and SGLang via output tokens alone and execute a proof-of-concept bare-metal exploit chain.
Miles v0.1: Production-Level Post-Training
Radix Ark open-sources Miles v0.1, a full-stack RL post-training framework demonstrated with asynchronous agentic RL on GLM-5.2 744B-A40B across 64 GB300 GPUs.
deepseek-ai/DeepSeek-V4-Flash-Vision-Exp — new model trending #10 on Hugging Face
DeepSeek released DeepSeek-V4-Flash-Vision-Exp, its first experimental multimodal model, with large multimodal agent benchmark gains over V4-Flash-0731.