| SlopCodeBench 10/2026 – Opus 5.5, Astra wins overall, xhigh considered harmful(github.com) | |
| 304 points by dhorthy 10 days ago | 120 comments | |
tl;dr: On the full 36-problem SlopCodeBench, GPT-6 Astra XHigh led strict pass rate at 16.3%, narrowly beating GPT-6.1 SOL Medium (14.8%) and Opus 5.5 Medium (14.3%), while Opus 5.5 Medium topped the looser "core" rate at 74%. Higher reasoning effort (XHigh) cost 2-4x more but didn't reliably improve strict scores—Opus 5.5 actually did worse on XHigh than Medium, though its XHigh output was ~5x cleaner by slop metrics. Models still routinely break regression tests as requirements evolve, with only 0-3 of 36 final checkpoints passing strictly. | |
HN Discussion:
| |