Jeeves. Reasoning improves Jev-like decision models(github.com)
241 points by nicowaltz 14 days ago | 94 comments
tl;dr: Jeeves is a 9B reasoning-augmented classifier (Qwen3.5-9B with LoRA and a pointer head) trained via SFT and CISPO that "thinks" before producing calibrated decision probabilities for yes/no, multiple-choice, and rating questions. It outperforms comparable Jev and Kev-9B models on out-of-domain test data (0.889 vs 0.857/0.822) and JevBench hard (0.865 vs 0.730), though it trails Jev on knowledge benchmarks like MMLU. A block-4 diffusion drafter provides ~1.6× speedup, with ~0.3s latency without thinking and 3.3s median with it on an H100.
HN Discussion:
  • ↓Latency defeats the purpose of a fast/cheap Jev-class model
  • ↓Empirical testing confirms it's slow and underperforms Jev on accuracy
  • ↓Post-training is unnecessary; prompting an LLM with prefill achieves the same
  • ↑Hybrid pipelines mixing Jev-style and LLM approaches are the right direction
  • •Newcomers questioning terminology and use cases of Jev models