Evolving Infra adapts LLM serving to agent workloads at runtime
Authors report up to 4.1x lower mean latency and 14x lower P95 TTFT than NVIDIA Dynamo on GLM-5.2 and Kimi K2.6.
3 stories, newest first · one card per story
Authors report up to 4.1x lower mean latency and 14x lower P95 TTFT than NVIDIA Dynamo on GLM-5.2 and Kimi K2.6.
PEARL adapts prefill-decode execution and elastic GPUs, raising agentic RL throughput by up to 2.79x.
EAServe raises multimodal LLM serving goodput by controlling encode, prefill placement, and GPU sharing.
Today 2.5K readers · 2.7K views · 144 reads · all time 206K views