LOWN skips image tokens in diffusion VLMs when predictions agree
Without training, LOWN keeps 99.6% of LLaDA-V's score at 35.3% of average decoding FLOPs.
Stories per day · 30 days
2 RSS
Relatedcosta1diffusion-vlm1llada-v1lown1speech-tokens1spoken-dialogue1token-compression1visual-tokens1
21 stories, newest first · one card per story
Stories tagged #inference-efficiency
CoSTA compresses speech tokens for faster spoken dialogue models
On bilingual benchmarks, generation speed rises about 1.7 times while response quality is preserved.
Jag: Direct Box Prediction for Efficient Visual Grounding
Jag predicts bounding boxes in one forward pass, specializing a compact multimodal model for faster visual grounding.
Judging in Latent Space: Efficient Generative Reward Modeling via Semantics-Preserving Compression
LatentGRM judges preferences in compressed latent space, cutting 8B judge time about 6–7x while matching SFT accuracy.
Stateless Language Agents: Scaling Long-Horizon Automated Research
Stateless Language Agents beat three research-agent frameworks at up to one billion tokens, using 84% fewer tokens on kernels.
Decoding Looped Transformers Better for (Almost) Free
LoopCD uses earlier recurrent states for contrastive decoding, boosting looped Transformers while cutting inference FLOPs.
Selection-Based Structured Reasoning: Toward Efficient Multimodal Search Agents
SSR selects pre-specified reasoning traces for small multimodal agents, cutting per-turn latency over 90%.
JevSpawn: Adaptive Agentic Inference through Compositional Action Spaces
JevSpawn turns language instructions into finite action spaces for faster agentic inference without extra training.
DeltaWAM: Delta World Action Models for Bimanual Manipulation
DeltaWAM predicts visual deltas and robot actions, raising RoboTwin success while cutting training and inference cost.
Video DeltaNet: A Video-Native Hybrid Attention for Livestream Video Generation
Video DeltaNet combines softmax and linear attention for video generation, cutting DiT denoising of a 14.3-second 768p video to 6.7 seconds.
Video DeltaNet: A Video-Native Hybrid Attention for Livestream Video Generation
Video DeltaNet combines local Softmax attention with linear memory, enabling 14.5x faster 768p video generation on eight NVIDIA B200 GPUs.
DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression
DeepSeek releases V4.1-Flash, a 552B-parameter multimodal MoE with 1M-token context and 890-byte-per-token KV cache for cheaper agentic workloads; weights on Hugging Face.
Sample Count Is Not Enough: Candidate-Generation Strategy Shapes the Energy and Performance of LLM Test-Time Scaling
Study shows LLM test-time scaling cost depends on generation schedule: eight serial A100 calls use ~4.7x more GPU energy than one batched call.
[AINews] DeepSeek v4.1-Flash: 763B-P8B-D16B novel causal Encoder–Decoder architecture with vision marks the Return of the Whale
DeepSeek released V4.1-Flash, an open-weight 763B-parameter model with a novel causal encoder-decoder architecture, 1M context, vision input, and MIT license.
4
Attention Quantization for Tabular Foundation Models
FP8 quantization of attention queries, keys, and values speeds tabular foundation model inference up to 1.7x with no accuracy loss.
New Deepseek model V4.1-Flash cuts memory needs for AI agents
DeepSeek released V4.1-Flash, a 552B-parameter open-weight model cutting KV cache needs to a quarter of its predecessor for cheaper million-token AI agents.
TRACE: Trajectory-robust Admission with Evidence Ordering for Efficient GUI Agents
TRACE, a training-free visual token pruning framework, cuts GUI agent inference latency and memory while keeping trajectory-wide visual evidence reusable.
Why Is Video Still So Expensive? A Survey of Inference-Efficiency Mechanisms in Video and Audiovisual LLMs
A survey catalogs inference-efficiency techniques for video and audiovisual LLMs, mapping bottlenecks in sampling, encoding, token reduction, and LLM decoding.
Grouped Value Attention: Efficient KV Caching via On-Demand Key Reconstruction
Grouped Value Attention stores grouped values and reconstructs content keys via a learned linear map, cutting KV-cache size about 45-47% versus GQA.
Signed Rescue Routing: Harm-Aware Cascades for Efficient LLM Inference
Signed Rescue Routing improves LLM cascade efficiency by predicting when a larger model actually corrects a smaller one rather than uncertainty.
ShallowStream: Index Shallow then Answer Deep for Streaming Video Understanding
ShallowStream builds streaming-video retrieval indexes from shallow MLLM layers, cutting per-frame prefill latency by up to 52.1x.