Memdoor: A team of 5 Qwen3.8-27B models beats Fable 5 on LiveCodeBench Hard(github.com)
279 points by guregodevo 13 hours ago | 127 comments
tl;dr: A multi-agent workflow (planner, two ideators, solver, reviewer, repair worker) using five Qwen3.8-27B instances communicating via files on disk matched the Fable 5 paper's harness on the first 10 LiveCodeBench Hard problems (9.8 vs 9.6 out of 10, five passes each), both beating a single Fable 5 call (8.0). The sample is too small to claim improvement over the original harness, and training contamination can't be ruled out. The setup is reproducible via Memdoor's YAML-based workflow, with runs averaging 40 minutes and $0.39 each on OpenRouter.
HN Discussion:
  • ↑Local models matching frontier models is critical to avoid dependence on hyperscalers
  • ↑Results suggest high local performance ceiling achievable with clever harnessing and fast hardware
  • ↑Multi-agent brute force scaling is a promising emerging axis for improving results
  • ~Benchmark sample is too small and possibly saturated to draw conclusions
  • •Asking tangential questions or noting prior related discussions without taking a stance