Retrospectively Reverse-Engineering Apple's Neural Engine
A developer reverse-engineers Apple's M1 Neural Engine architecture, mapping compute cores, MAC datapaths, and schedulers to explain the NPU's decline as transformers displaced CNN workloads.
A developer who previously maintained a reverse-engineered Linux driver for Apple's Neural Engine (ANE) published a retrospective deep dive mapping the M1 ANE's full internal architecture: compute, datapath, scheduler, memory, and execution model. The M1 ANE has 16 compute cores with 128 FP16 (or 256 INT8) MAC lanes each, totaling 2048 parallel MAC lanes, using 32-bit Q16.16 fixed-point accumulation with FP16 readout and an accumulator that saturates at 2^15. The author argues the ANE's dataflow was architected around the predictable reuse patterns of 2017-era CNN workloads (dating to the A11 Bionic), which autoregressive transformer decode broke, limiting its usefulness for general ML. With Apple's M5 folding ANE cores into GPU cores to tout LLM performance, the post frames this as the beginning of the end for the standalone NPU.
- M1 ANE contains 16 compute cores with 128 FP16 or 256 INT8 MAC lanes each, totaling 2048 parallel lanes.
- MAC accumulation is 32-bit Q16.16 fixed-point read out as FP16, saturating at 2^15.
- ANE dataflow assumed predictable CNN reuse patterns; autoregressive transformer decode broke this assumption.
- Author previously released a reverse-engineered ANE Linux driver (github.com/eiln/ane) but abandoned it as impractical for general workloads.
- Apple's M5 folds ANE cores into GPU cores, which the author reads as the standalone NPU's decline.
From the original article
13 min read at eiln.github.ioI stopped working on the reverse-engineered Apple Neural Engine (ANE) driver three years ago, upon a sad mini realization that the ANE block is just not that useful, and I could be doing more useful things, and moved onto upstreaming other, more useful, blocks.
By zdw · eiln.github.io
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.