[AINews] Hot Chips: OpenAI’s Jalapeño, Cerebras CS-5, Groq 3 LPX, Apple M6
OpenAI unveiled Jalapeno custom inference chip claiming 1.5-1.9x better perf-per-watt than NVIDIA GB200/GB300, deploying in-house by year-end.
At the 37th Hot Chips conference, OpenAI published first benchmark details for its custom Jalapeno inference chip, claiming 1.5-1.9x more work per watt, 1.7-3.6x lower end-to-end latency, and 2.1-4.1x higher interactive-workload performance versus NVIDIA GB200/GB300, with the 700W-rated part staying at or below 550W in tests. Deployment into OpenAI's own infrastructure begins by year-end, with Gen 2 deep in development and Gen 3 underway. OpenAI also said GPT-Astra and Codex helped write low-level kernels, reportedly 1.5-1.8x faster than human-expert code for selected attention and MoE blocks. Cerebras CS-5, Groq 3 LPX and Apple M6 were also featured at the conference.
- Jalapeno claims 1.5-1.9x work per watt and 1.7-3.6x lower latency than GB200/GB300
- Rated 700W but ran at or below 550W on tested workloads
- Deployment into OpenAI infrastructure starts by year-end; Gen 2 and Gen 3 in development
- Model-assisted kernel work brought three open-weight models to high performance in about two months
- SemiAnalysis called it unusually strong for a first-generation chip against Blackwell-class systems
From the original article
8 min read at latent.space
By far the biggest announcement at the 37th Hot Chips conference was OpenAI’s stunning progress on their own chip, less than a year after the Broadcom announcement … and that it isn’t an ASIC; but a full on Blackwell-beating alternative. The key metric now is shifting to performance per watt, and Jalapeno delivers: The full Hot Chips presentation is not yet out but various takes are below.
latent.space
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.