Bonsai 2 27B: Near-Lossless Compression in a 9x Smaller Footprint

	Bonsai 2 27B: Near-Lossless Compression in a 9x Smaller Footprint(prismml.com)
	576 points by JonSchneider 26 days ago \| 197 comments
	tl;dr: PrismML released Ternary Bonsai 2 27B, a compressed version of Qwen3.8 27B using ternary {−1, 0, +1} weights with FP16 scaling (1.76 bits/weight), shrinking the model to 5.9GB while retaining 98.2% of full-precision benchmark performance. It hits 143 tokens/sec on an RTX 5090 and 46.8 tokens/sec on M5 Max, supports 262K context and multimodal input, and runs via CUDA or MLX under Apache 2.0.
	HN Discussion: •Practical setup instructions for running the model locally with required forks ↓The 'near-lossless' claim is overstated; model struggles on real tasks ↓Linguistic pedantry about '9x smaller' phrasing being mathematically incorrect ↓Skepticism about lack of comparison to existing quantization methods like Q2 or Unsloth ↑Sharing benchmark results on personal hardware confirming impressive performance