Jeff – Jev-compatible 0.8B decision models, trained at home, ~30 ms(github.com)
565 points by firelex 15 days ago | 218 comments
tl;dr: Jeff is a set of small (0.8B–2B) fine-tuned Qwen and Gemma models for zero-shot classification, compatible with the Jev API format, returning calibrated probabilities in ~22-30ms per decision on GPU/Apple Silicon. Trained entirely on local hardware using synthetic data from open models, it matches or beats Jev on classification/grounding benchmarks but lags on reasoning-heavy tasks. Limitations include a 26-option cap, English-only support, and no multi-step reasoning—though a short fine-tune reportedly boosted one task's accuracy from 31.7% to 95.8%.
HN Discussion:
  • ↑Showcasing related demo project built on similar API compatibility
  • ↓Questions whether a full LLM is needed when embedders could handle classification
  • ↓Skeptical of clones since Jev's actual architecture isn't public and may not even be generative
  • ↓Reports significantly worse accuracy than Jev in real use cases
  • ↑Speculates about broader implications for LLM spending and local model dominance