Bonsai 27B: A 27B-Class model that runs on a phone

	Bonsai 27B: A 27B-Class model that runs on a phone(prismml.com)
	626 points by xenova 18 hours ago \| 219 comments
	tl;dr: PrismML released Bonsai 27B, a compressed version of Qwen3.6 27B using ternary (1.71-bit, 5.9GB) or binary (1.125-bit, 3.9GB) weights, allowing a 27B-class multimodal model to run on an iPhone 17 Pro or laptop. The ternary variant retains ~95% of full-precision benchmark performance and the 1-bit version ~90%, with math and coding least affected. Weights are Apache 2.0 licensed and run via MLX on Apple devices and CUDA on NVIDIA GPUs, hitting up to 163 tok/s on a 5090.
	HN Discussion: •Requests comparisons to other small quantized models like Gemma 4 12B QAT to validate claims ~Skepticism about benchmark comparisons and how it stacks against native small models ~Shares hands-on benchmarks showing CPU inference for ternary is unoptimized ↑Fascination with binary-weight networks and curiosity about CPU efficiency implications ↑Enthusiasm for ternary model scaling finally arriving, enabling local inference