Nvidia Nemotron 3.5 Lightning and NeMo Switchyard(blogs.nvidia.com)
257 points by droidjj 10 days ago | 133 comments
tl;dr: NVIDIA released Nemotron 3.5 Lightning, a 30B-parameter mixture-of-experts open model optimized for high-volume agentic tasks, claiming up to 4x faster output and 30% faster task completion versus peers, with customization support via NeMo. Alongside it, NVIDIA open-sourced NeMo Switchyard, a model routing library that directs prompts to the most suitable model across open, proprietary, and NVIDIA options—internal benchmarks show ~⅓ the cost of using Opus 4.8 alone. Partners including LangChain, Ramp, Cognition, and Kong report cost reductions of 27–74% while maintaining near-frontier accuracy.
HN Discussion:
  • MoE models like Nemotron underperform dense models for coding tasks despite speed claims
  • Small efficient models are the right direction and will drive future gains
  • Skepticism about benchmark honesty, notably excluding competitive Qwen models from comparisons
  • Enthusiasm for the small model release working well on consumer hardware like Apple Silicon
  • Practical questions about routing mechanics (prompt caching) and VRAM requirements