Cerebras describes serving trillion-parameter models across WSE-3T wafers
Original title: Cerebras Deploys Trillion-Parameter AI Models
Using Kimi K3 as an example, Cerebras maps layers onto 59 wafers and keeps weights on chip.
Cerebras explains how it runs multi-trillion-parameter models on WSE-3T wafers that each have 44 GB of on-chip memory. The example is open-source Kimi K3, about 3 trillion parameters, with 69 linear-attention layers and 24 dense layers, 4-bit weights, and an 8-bit KV cache. Dense layers take one wafer each and linear layers two, for 59 wafers, with only activations passed by RDMA over Ethernet. Concurrent requests fill the pipeline; KV-cache SRAM limits how many can be in flight.
Unconfirmed: Several wafer-count and concurrency figures in the post are illustrative and are not fully shown in the excerpt.
- Each WSE-3T wafer has 44 GB of on-chip SRAM.
- The example assumes a 256k average request and 4-bit weights.
- Cerebras says activations cross wafers in about 2 microseconds per hop.
- Weights stay on chip; only activations move between pipeline stages.
- The post says production uses finer-grained stages than one layer.
From the original article
5 min read at cerebras.ai
One of the most frequently asked questions we get is how we deploy large models on Cerebras wafer-scale chips. A single Cerebras WSE-3T wafer has 44 GB of on-chip memory, and trillion-parameter frontier models need terabytes for their weights alone. In this post, we’ll explain how we split a large frontier model across multiple WSE-3T wafers, to run multi-trillion parameter models at 1000-2000+ tokens per second.
By gmays · cerebras.ai
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.