NVIDIA claims codesign lifts agentic throughput per dollar up to 30x
Original title: More Performance From Every Byte: How Full-Stack Codesign Solves AI’s Memory Challenge
From Blackwell Ultra to Vera Rubin, HBM per GPU rose about 3% while claimed agentic throughput per dollar rose up to 30x.
NVIDIA says growing models, long contexts and agentic workloads are tightening GPU HBM supply, and that adding HBM alone hits a memory-bandwidth wall. The company describes codesign across GPUs, Vera CPUs, NVLink-C2C, Dynamo software and CMX context storage to shrink weights and KV cache with NVFP4, offload idle context, and keep RAG indexes out of HBM. It cites up to 30x higher agentic throughput per dollar from Blackwell Ultra to Vera Rubin on SemiAnalysis AgentX, against roughly 3% more HBM per GPU.
Unconfirmed: The up-to-30x throughput-per-dollar figure is NVIDIA’s claim based on SemiAnalysis AgentX, not independently confirmed in the post.
- Frontier models have trillions of parameters; some agentic trajectories process up to 3 million tokens.
- NVFP4 reduces weight and KV-cache footprint versus FP8, the lowest precision on Hopper.
- NVL72 can spread model weights across up to 72 GPUs, freeing HBM for KV cache.
- Dynamo moves inactive context to shared CMX storage and returns it before reuse.
- Named CMX partners include Dell Technologies, HPE, IBM, NetApp, VAST and WEKA.
- Vendors
- NVIDIA
From the original article
5 min read at blogs.nvidia.com
Larger models, longer contexts and agentic applications are increasing GPU memory pressure while supply tightens and prices rise — a trend expected to continue through the near term. NVIDIA works closely with memory partners to rapidly adopt the latest memory technologies. NVIDIA was first to adopt HBM3e and HBM4 technologies into an accelerated computing stack.
By Amr Elmeleegy · blogs.nvidia.com
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.