Run Kimi K3 using 29 GB of RAM at 0.50 tok/s

	Run Kimi K3 using 29 GB of RAM at 0.50 tok/s(github.com)
	262 points by marcobambini 21 hours ago \| 110 comments
	tl;dr: WASTE is a dependency-free C inference engine that runs the full 2.78T-parameter Kimi K3 MoE model on a 64GB MacBook Pro at ~0.5 tok/s by streaming experts directly from NVMe (982GB container) while keeping a 27GB trunk resident and using remaining RAM as a bounded expert cache. Experts are stored at 3 bits/weight via residual vector quantization, and reads bypass the page cache; the usable RAM window is narrow (~46GB optimal, collapses above 52GB due to OS paging). I/O—not arithmetic—dominates at 53% of decode time.
	HN Discussion: ↓Skepticism that the README/reasoning was LLM-generated rather than human-authored ↓Questions the novelty since llama.cpp already does mmap-based streaming with page cache ↓Concerns about the company behind it due to prior non-open-source licensing history ~Interest in the concept but wanting to try with smaller models or faster hardware •Practical cost/performance analysis of running inference this way