AMD acquires Taalas to boost inference performance by etching models in silicon(theregister.com)
923 points by itvision 15 days ago | 693 comments
tl;dr: AMD has acquired Toronto-based startup Taalas, which etches AI model weights directly into silicon rather than storing them in HBM, creating what amounts to model-specific integrated circuits. Taalas' first test chip reportedly ran Llama 3.1 8B at ~17,000 tokens/sec (48x faster than Nvidia GPUs), and its upcoming HC2 chip targets 20B parameters per accelerator. The tradeoff is inflexibility—model changes require a chip re-spin (though only two metal layers)—making the tech best suited for stable, high-volume inference workloads from major model providers, likely paired with AMD's Instinct GPUs in disaggregated deployments.
HN Discussion:
  • Baked-in silicon LLMs will become ubiquitous in consumer devices like decoder chips did for video
  • This is a strategic win that undercuts Nvidia and competitors should have made this move first
  • The speed unlock is transformative for robotics, IoT, and enables new UX paradigms hard to predict
  • Fast-to-deprecation concerns are overblown because faster inference enables cheaper iteration on errors
  • Skeptical of the approach given reliability issues with LLMs and questions whether this hardware is needed