| Benchmarking Qwen3.8 27B quantizations: 4-bit holds up, 1-bit collapses(quesma.com) | |
| 283 points by stared 2 days ago | 131 comments | |
tl;dr: Benchmarking Qwen3.8 27B quantizations on GPQA Diamond, IFBench, and Terminal-Bench 2.1 shows that 4-bit Q4_K_M (17GB) matches full BF16 performance and fits on a 24GB GPU, while 2-bit degrades modestly but remains usable. 1-bit quantizations collapse to random-chance performance, with longer reasoning making results worse as models exhaust token budgets. Takeaway: pick the largest quant that fits your VRAM plus context—Q4_K_M is the sweet spot, and quantization shouldn't be feared. | |
HN Discussion:
| |