Unsloth Dynamic 3.0 GGUFs(unsloth.ai)
317 points by jonesy827 2 days ago | 118 comments
tl;dr: Unsloth released Dynamic v3.0 GGUF quantizations for Qwen3-VL-27B, claiming >10% better top-1 accuracy at the same file size versus other providers, using improved imatrix calibration, smarter layer selection, and pure post-training quantization (no QAT/QAD). New features include sub-1-bit quants (UD-IQ1_S at 6.2GB retaining ~72% top-1 accuracy) and a Divergence-300 @32 benchmark for evaluating multi-token trajectory fidelity against BF16. The quants work with llama.cpp and other standard inference engines.
HN Discussion:
  • Praises Unsloth's quants but requests speed/performance improvements for local inference
  • ~Requests clearer versioning of GGUF files to distinguish Dynamic 3.0 from prior releases
  • ~Skeptical of KL-divergence metrics; wants real-world coding benchmarks to validate quality claims
  • Questions the decision to remove MTP support in the new quants
  • Excited about sub-1-bit quants and asks for additional formats (MLX, NVFP4) and comparisons