Heartbeat Thinking keeps voice-agent tools running while listening
On FDBench v3, tool coverage rises 16 percentage points and mean final-response latency falls 36.4 percent versus a windowed baseline.
Stories per day · 30 days
2 RSS
Relatedagents1bidirectional-models1calibration1decision-models1inference1interactive-streaming1intern-decision1multimodal1speculative-execution1speculative-sampling1tool-use1video-generation1voice-agents1
17 stories, newest first · one card per story
Stories tagged #latency
Intern-Decision: Efficient Multimodal Decision Making with Autoregressive Masked Language Modeling
Intern-Decision's 4B model hits 90.02% decision accuracy with 44 ms latency on an RTX 4090.
Speculative Sampling in Interactive Video Generation
Speculative Sampling caches large bidirectional video windows so interactive generation can stay low-latency without shrinking chunks.
Hiding Tool Latency in On-Device Cascaded Voice Agent through Speculative Execution
Speculative tool execution cuts on-device voice-agent median time-to-first-audio from 5.79s to 4.60s.
Intent Interpretation at RIC Timescales: Jev Decision Models versus Large Language Models in 6G Open RAN
Jev-1.13.0 meets a one-second Open RAN intent budget far more often than hosted LLMs.
cua-speedrun: Standardized Benchmarking of the Speed of Computer-Use Agents
cua-speedrun standardizes virtual-machine infrastructure to measure computer-use agent speed, cost, and efficiency.
Decision-Oriented Recommendation Reranking: An Empirical Study of Jev
TypeSafe AI's Jev holds recommendation quality while latency grows more slowly than pointwise Qwen rerankers.
Perplexity Introduces Photon: A Rust-Based Retrieval Engine That Cuts p99 Latency From 800 ms to 65 ms
Perplexity's Rust engine Photon cuts retrieval p99 latency from about 800 ms to 65 ms.
2
Decision-Oriented Recommendation Reranking: An Empirical Study of Jev
Jev matches strong recommendation rerankers with slower latency growth than pointwise Qwen models.
Report: Progressive Disclosure of Agent Skills
Workday finds progressive disclosure of agent skills improves retrieval quality but slightly increases latency.
CacheRepair: Learning to Repair Cross-Chunk Context in RAG for KV Cache Fusion
CacheRepair restores cross-chunk KV context in RAG, speeding first token by 1.69-4.61x.
Replacing Large Language Models with Jev Decision Models for Low-Latency Edge Service Orchestration
Jev decision models cut median edge-orchestration latency 23-65% versus the fastest LLM on 8,280 requests.
How we made claude.ai 3x faster in two weeks
Anthropic cut core claude.ai and desktop latency about 3x in two weeks, using Claude to find and ship fixes.
Fresh-Challenge VDF Attestations for Model-Relative Response Latency
Fresh-Challenge VDF Attestations bind verifiable delay functions to unpredictable public challenges, yielding succinct evidence of model-relative response latency.
PACE: Perceived-Latency-Aware Cascading Service Routing and Filler Control for QoE-Efficient Retrieval-Augmented Dialogue Serving
PACE cuts perceived latency in retrieval-augmented dialogue serving via cascading routing and filler control, reaching 0.41s P95 under load.
OpenAI’s GPT-5.6 Sol runs up to 14× faster with Ultrafast mode
OpenAI launched GPT-5.6 Sol Ultrafast mode in limited preview, running up to 14x faster at 750 tokens per second via Cerebras inference.
Previewing Ultrafast mode: GPT-5.6 Sol at up to 14X the speed
OpenAI previews Ultrafast, an API service tier running GPT-5.6 Sol up to 14x faster via Cerebras at up to 750 output tokens per second.