Samsung's Processing-in-Memory (PIM)(chipsandcheese.com)
278 points by ingve 12 days ago | 106 comments
tl;dr: Samsung's LPDDR5X-PIM embeds MAC units at each of 16 DRAM banks, exploiting internal bandwidth (614 GB/s vs 76.8 GB/s external) for low-precision ML compute, while cleverly staying within the standard LPDDR5X protocol by repurposing special row addresses as mode-switching MMIO. However, integration with real systems is painful: PIM mode changes the semantics of read/write commands, breaking caching, prefetching, and speculative execution, and forcing memory regions to be uncacheable and effectively single-threaded. The author argues practical adoption would require extending DRAM interfaces with dedicated compute commands and cache-coherent memory controllers rather than the current mode-hijacking approach.
HN Discussion:
  • PIM is too constraining for general applications and better suited to ASICs for specific domains
  • PIM/dataflow architectures are the inevitable long-term future for AI compute
  • Skepticism that this specific implementation will succeed, like many exotic accelerators before it
  • This implementation is flawed because matrix multiplication requires extensive data movement PIM doesn't solve
  • Rather than patching around limitations, the entire computer architecture should be redesigned