Retrospectively Reverse-Engineering Apple's Neural Engine(eiln.github.io)
230 points by zdw 1 day ago | 32 comments
tl;dr: A reverse engineer returns to Apple's Neural Engine (ANE) on the M1 to map its full architecture, motivated by Apple's M5 folding ANE cores into the GPU—signaling the end of the standalone NPU. The teardown reveals ANE as a fixed-function dataflow engine with 16 cores of 128 FP16 MACs, a serialized register-configuration task descriptor model (no ISA), and a memory hierarchy (per-core KMem, shared 2MB L2) optimized for CNN weight reuse. The design's assumption that kernels are static and loaded infrequently—reasonable in 2017—cripples it for transformer decode, where serial kernel/tile DMA caps bandwidth below the GPU's.
HN Discussion:
  • Article conflates the still-active ANE with the new GPU-integrated Neural Accelerators, questioning its premise
  • Article's technical analysis of ANE's CNN-oriented design explains long-standing questions about its limited impact
  • Firsthand experience confirms ANE requires shoehorning transformers into CNN-shaped workloads
  • Apple's broader AI software stack (Core AI) and early NPU investment provide important context beyond the teardown
  • Praise for the quality and depth of the reverse-engineering work itself