30
55
PACE: Perceived-Latency-Aware Cascading Service Routing and Filler Control for QoE-Efficient Retrieval-Augmented Dialogue Serving
PACE cuts perceived latency in retrieval-augmented dialogue serving via cascading routing and filler control, reaching 0.41s P95 under load.
PACE is a serving framework for retrieval-augmented dialogue that optimizes Perceived Time-to-First-Response (PTFR) under quality and cost constraints. It combines a load-adaptive cascading router, a joint path-filler controller, and volatility-aware cache admission, deployed on a humanoid-robot sales service. On 75k CarQA requests, the cascade halved pure-LLM P95 PTFR (0.29s vs 0.53s) and the adaptive controller reached 0.41s P95, 2.4x better than RAG at high load; filler calls dropped 94% and stale answers fell from 86% to 0%.
20
30
30
55
55
55
Netskope acquires Infiot to provide users with optimized connections between any enterprise location
55
30
30
30
30
30
55
30
42
55
55
30
30
NS1 adds new capabilities and video streaming support to its Pulsar Active Traffic Steering solution
55
30