Apple ML Research: Muse Spark Open with MLX on Mac Studio

	Apple ML Research: Muse Spark Open with MLX on Mac Studio(machinelearning.apple.com)
	362 points by antipaul 9 days ago \| 205 comments
	tl;dr: Apple demonstrates running the 816B-parameter Muse Spark Open model locally across four M5 Ultra Mac Studios using MLX, achieving 36+ tokens/second via tensor parallelization over Thunderbolt 5 RDMA. The M5 Ultra's GPU Neural Accelerators and 1.2 TB/s memory bandwidth deliver up to 4.6× faster time-to-first-token and 1.6× faster decode versus M3 Ultra on models like Qwen 3.8 and Mistral Medium 3.5. MLX supports both pipeline and tensor parallelization via a simple `mlx.launch` command, enabling local agentic coding workflows with tools like OpenCode at roughly 300W peak power per machine.
	HN Discussion: ↓Benchmarks are misleading due to unfair quantization comparisons and cherry-picked models ↓Muse Spark Open is a poor model choice; other smaller models perform better ↓The demo context window is too small to be practically useful ↑Apple is well-positioned to profit from running others' open-weight models locally ~Impressive direction for local inference but currently too expensive for most users