Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots(cactuscompute.com)
509 points by HenryNdubuaku 11 days ago | 171 comments
tl;dr: Cactus has released Needle 2, an open-source (Apache 2.0) 45M-parameter model for tool calling and structured extraction, shipped as a 14MB binary with a 28MB RAM ceiling that runs on microcontrollers, cheap phones, and Raspberry Pis at 300–1,500 tokens/sec. It uses a custom Simple Attention Network architecture with 2-bit quantization baked into training (not post-hoc) and a Hadamard MLP plus hashed n-gram engram tables to cut FLOPs. On function-calling benchmarks like Mobile Actions and Seal-Tools, it trades wins with FunctionGemma 270M, LFM2.5 230M, and Apple FM despite being 5–70× smaller.
HN Discussion:
  • Appreciates the micro-LLM concept and potential for hierarchical model architectures
  • Web demo produces nonsensical or incorrect tool-calling results, undermining claims
  • Questions the technical innovation and differentiation from existing solutions like LiteRT
  • ~Impressed by size achievement but skeptical of practical usefulness at this scale
  • Already finding real integration uses or curious how such micro-LLMs are built