DSpark: Speculative decoding accelerates LLM inference [pdf]

	DSpark: Speculative decoding accelerates LLM inference [pdf](github.com)
	791 points by aurenvale 108 days ago \| 361 comments
	tl;dr: Summary not available
	HN Discussion: ↑DeepSeek is praised for genuine innovation and open publishing compared to American labs ↑Users share positive real-world experience with DeepSeek models and integrations ↑This technique likely explains DeepSeek's ability to offer dramatically lower prices •Questions whether this is meaningfully novel versus 2022 speculative decoding work •Speculates about future proliferation of specialized small models for speculative decoding