[AINews] Hot Chips: OpenAI’s Jalapeño, Cerebras CS-5, Groq 3 LPX, Apple M6

By far the biggest announcement at the 37th Hot Chips conference was OpenAI’s stunning progress on their own chip, less than a year after the Broadcom announcement … and that it isn’t an ASIC; but a full on Blackwell-beating alternative.

The key metric now is shifting to performance per watt, and Jalapeno delivers:

The full Hot Chips presentation is not yet out but various takes are below. For a fuller breakdown, watch along with the rest of OpenAI:

AI News for 8/24/2026-8/25/2026. We checked 12 subreddits, 544 Twitters and no further Discords. AINews’ website lets you search all past issues. As a reminder, AINews is now a section of Latent Space . You can opt in/out of email frequencies!

AI Twitter Recap

OpenAI’s Jalapeño Inference Chip and the Shift in the Inference Stack

Jalapeño’s published numbers are the day’s biggest technical story : OpenAI released first benchmark details for its custom inference chip Jalapeño , claiming materially better efficiency and latency than NVIDIA GB200/GB300 systems on real model workloads. In OpenAI’s tests, Jalapeño delivered 1.5–1.9× more work per watt at peak throughput and 1.7–3.6× lower end-to-end latency , with 2.1–4.1× higher performance for highly interactive workloads; the chip is rated at 700W but reportedly stayed at or below 550W on the tested runs. OpenAI says deployment into its own infrastructure begins by year-end , with Gen 2 already deep in development and Gen 3 underway ( OpenAI announcement , deployment roadmap , Sam Altman ).

Why engineers care : the claim is not just raw perf, but a more balanced inference architecture that reduces the usual throughput/latency tradeoff. Multiple technical reactions highlighted that some comparison points are especially notable because Jalapeño reportedly performed well even without tricks like aggressive prefill/decode disaggregation or speculative decoding in some setups, while beating systems that did use them ( gdb , kimmonismus summary , eliebakouch analysis , You Jiacheng ). SemiAnalysis framed it as unusually strong for a first-generation ASIC and compared it directly against Blackwell and Rubin -class systems ( SemiAnalysis , dylan522p ).

A second-order story is model-assisted systems optimization : OpenAI’s post also said GPT-Astra + Codex helped write and optimize low-level kernels, bringing three previously unplanned open-weight models to high performance on Jalapeño in about two months; for selected attention and MoE blocks, these implementations reportedly ran 1.5–1.8× faster than existing human-expert-written code ( kimmonismus , eliebakouch ). That is a meaningful signal that compiler/kernel work is increasingly being folded into the model improvement loop, not just application-layer coding.

Broader infra implication : several posts tie Jalapeño to a larger industry transition in which frontier labs may no longer be strictly downstream of NVIDIA for inference economics, even if packaging and foundry capacity remain a hard bottleneck ( Liam Fedus , teortaxesTex reaction , LearnOpenCV caveat on TSMC/CoWoS capacity ).

Agent Harnesses, Memory Systems, and Eval Engineering Becoming First-Class

Harness quality is increasingly as important as model choice : several papers and launches converged on the same theme: agent performance depends heavily on the surrounding system. A new Microsoft-led paper on AutoSaddler treats the harness as code and patches prompts, tool configs, and control logic offline using failure traces, reporting gains of +9.0 on GAIA2 , +9.6 on SWE-Bench Pro , and +10.0 on Terminal-Bench 2.0 over base harnesses ( paper summary ). In parallel, another paper quantified harness variance directly, finding that swapping harnesses could move scores far more than swapping models, with model-pair rankings flipping across scaffolds; the proposed fix is a structured Harness Card disclosure standard ( analysis , “There Is No Neutral Harness” ).

Long-horizon software engineering remains very unsolved : SWE Refactor Bench measures whole-repository migration tasks like C→Rust , Maven→Gradle , and POSIX→WebAssembly across real projects including SQLite , zlib , and libsodium . Across 520 runs , only 28 survived all three stages, for a 5.4% survival rate, and 13/20 tasks were solved by nobody ( EinsiaAI ). This is a useful corrective to strong bug-fix numbers on more local coding benchmarks.

Memory systems are being redesigned as programmable state, not compressed chat history : one Alibaba paper summarized by DAIR backs agent sessions with an append-only event log plus a persistent Python kernel , binding tool outputs and derived state to typed variables instead of continually serializing them into prompts. Reported results include 94.8% on LongMemEval_S , 73.1% on BEAM_10M (+5.1 over the previous best published memory system), and 86.7% on LOCA_256K with Qwen3.8-Max ( summary ). Related work on Knowledge Triage showed that naive context compaction destroys exact-rule retention; after five rounds of compaction, one setup preserved only 10% of safety rules, while type-aware retention policies preserved 2–4× more ( summary ).

Practical eval-engineering is moving from ad hoc to productized workflows : LangChain/partners shared a concrete loop for turning traces and human feedback into task specs , synthetic environments, and evals that can be used to measure and post-train agents over time ( Vtrivedy10 , hwchase17 ). LangSmith Engine also shipped >2× better performance on key internal benchmarks with better issue detection/clustering, SaaS and self-hosted support, Slack/Linear integrations, and cost-tiered analysis modes ( LangChain ).

Local-First Agents, On-Device Inference, and the New Personal Compute Stack

Perplexity’s Portable Computer is the clearest local-agent product launch of the day : Perplexity launched Portable Computer on NVIDIA DGX Spark , positioning it as a fully local version of Perplexity Computer where the orchestrator LLM , subagent LLM , and agent harness all run on local hardware with no cloud dependency ( Perplexity launch , model details , NVIDIA , Arav Srinivas ). The initial local stack uses a post-trained PPLX 27B with Qwen 3.8 27B also available; Nemotron 3.5 Lightning support is coming.

The deeper trend is persistent, always-on local agents : Srinivas explicitly sketched a future of background processes that continuously ingest context from connectors, perform multi-hop reasoning in a perpetual loop, and run on your own hardware ( Arav Srinivas ). Community reactions were split between excitement about privacy/control and skepticism that “local-first” should mean a $5k DGX Spark rather than commodity consumer devices ( theo critique , theo follow-up ).

Apple/macOS local AI tooling is also maturing : exo said Apple featured it on new M5 Ultra Mac Studio and M6/M5 Pro Mac Mini pages, emphasizing low-latency RDMA over Thunderbolt 5 to cluster Macs and run models like Kimi K3 and GLM-5.3 at API-like speeds, with 4× M5 Ultra scaling to about 4.8 TB/s aggregate memory bandwidth ( exo ). Related posts pointed to Apple’s faster PCIe storage and ANE-based vision pipelines as making small local clusters and mixed CPU/ANE/GPU inference more practical ( anemll , onirenaud ).

Tooling continues to fill in around local runtimes : Ollama v0.33 added one-toggle integration to let Claude Desktop use Ollama as a third-party gateway for cloud and local models ( Ollama ); OpenCode v2 was shown running inside a Cloudflare Durable Object , illustrating how small agent runtimes are becoming embeddable in edge environments ( fayazara ).

Models, Retrieval, and Search Infrastructure

Qwen 3.8 is showing up across the stack : enthusiasm around the Qwen3.8 release was visible in both deployment and evaluation posts, with Together adding fine-tuning and dedicated inference support for Qwen3.8-27B ( Together ) and Unsloth claiming full QLoRA fine-tuning of the 27B model on free 2× Tesla T4 Kaggle instances using optimized kernels ( danielhanchen ). On the application side, Qwen3.8-27B reached #1 among open models in the Image-to-WebDev Arena and #7 overall , while priced at $0.40 / $3 per million input/output tokens ( arena ).

Search and retrieval infra got multiple substantive updates : Hugging Face published a detailed architecture writeup for the Papers with Code search engine: PostgreSQL + pgvector , Qwen 3 Embedding 0.6B , hybrid retrieval, embeddings generated on an NVIDIA L4 via Hugging Face Jobs, artifacts in buckets, and live serving via Inference Endpoints; the same stack powers “related papers” on paper pages ( Niels Rogge ). Keenable came out of stealth with a Web Search API and Web Query Language for AI, built by former Yandex Search leaders and backed by a $26M seed , explicitly targeting agent-scale web retrieval ( styskin ).

Retrieval model design remains active territory : there was renewed discussion around late interaction / multivector retrieval , with claims that scaling behavior is finally becoming visible in retrieval workloads and that model+DB co-design matters at least as much as storage format ( mixedbread perspective , Silvio Martinico ).

Robotics, Physical World Models, and Embodied Data

Figure’s “Index” is a major robotics data announcement : Figure introduced Index , described as the largest and most diverse robot dataset in the world, with reported ingestion at 30 minutes of video uploads per second , 16M video uploads , $15M already paid out for data, and 264k downloads . The company also says it will spend $1B over the next 12 months on data and compute ( Brett Adcock , follow-up ). That scale matters because many robotics labs still appear more bottlenecked on demonstration and perception data than on architecture novelty.

Large-scale physics/world modeling continues to push context limits : Anima Anandkumar highlighted Accelerated Understanding , a startup building large AI models for physical simulation across modalities and 4D spacetime, claiming 1T parameters during pretraining , 1T context during training, and >5T context at inference without subsampling or patching ( Anima Anandkumar ). The details are sparse, but the post is notable as a statement of where some frontier non-language modeling work is heading: massive-context multimodal simulation rather than only text/video generation.

Embodied policy generalization remains an active benchmark target : a separate robotics post introduced S1 , a manipulation model that can complete tasks from a single demonstration outside its training distribution ( anag004 ). Google Research also shared AgentHands , an XR system that augments conversational agents with synchronized hand gestures for spatial guidance during physical tasks ( Google Research ).

Top tweets (by engagement)

OpenAI chip launch : @sama on Jalapeño , @OpenAI benchmark announcement drove the largest technical conversation by far.

Local agent launch : @perplexity_ai launching Portable Computer was the biggest product release outside the chip story.

Developer platform / agent-native web : @OpenAIDevs announcing the WebMCP Challenge and WebMCP support in ChatGPT desktop signal OpenAI pushing websites toward explicit agent interfaces.

Open-source local task agents : @AndrewYNg on OpenWorker stood out for combining open harnesses, local models, and security-focused workflows.

Benchmark realism for coding agents : @EinsiaAI on SWE Refactor Bench is one of the more useful benchmark releases in the set because it targets whole-repo migrations instead of local edits.

AI Reddit Recap

/r/LocalLlama + /r/localLLM Recap

1. Qwen3.8 Flash/27B Benchmarks and Local Fit

Read more

Text extracted automatically; images, tables and formatting may be missing. Original: https://www.latent.space/p/ainews-hot-chips-openais-jalapeno