Cerebras Deploys Trillion-Parameter AI Models(cerebras.ai)
358 points by gmays 1 day ago | 137 comments
tl;dr: Cerebras runs trillion-parameter models like the 3T-parameter Kimi K3 by pipelining layers across multiple WSE-3T wafers (59 in this example), with each 44GB wafer holding one dense layer plus KV cache, or two linear-attention layers. Weights stay resident on-chip and only activations pass between wafers via RDMA over Ethernet (~2µs per hop). Concurrency is maximized by keeping the pipeline full with micro-batched requests at each stage, with KV cache capacity on dense-layer wafers setting the ceiling on simultaneous requests.
HN Discussion:
  • ~Cerebras serves outdated models but should catch up soon with faster, cheaper alternatives to frontier models
  • ↓Cerebras can't scale to meet demand due to hardware manufacturing constraints and limited funding
  • ↓Available Cerebras models are too limited or low-quality for practical use despite impressive speed
  • •Technical curiosity and questions about the chip architecture, PE mesh, and KV cache handling details
  • ↓Criticism of the company's execution, team quality, or misleading benchmark practices (e.g. 4-bit quants)