The Normalization of Inexplicable Failures(ihatethefuture.com)
276 points by pxx 16 days ago | 123 comments
tl;dr: The author critiques the rise of LLM-based tools like "Jev" that return typed values with confidence scores, arguing developers are shipping them without evals, calibration checks, or failure-mode analysis—treating confidence scores as cargo-cult justification for errors. The deeper concern isn't that AI-driven systems fail more, but that "sometimes it just sucks" is becoming an acceptable end-state for debugging, normalizing inexplicability and eroding accountability in software.
HN Discussion:
  • ↑Normalizing LLM failures in libraries, infrastructure, and compilers would degrade the entire software ecosystem
  • ↑The erosion of accountability and inexplicability extends beyond software into consumer products broadly
  • ↑Confidence scores are anthropomorphic misnomers that mislead non-technical stakeholders about ML reliability
  • ↓The author is being pessimistic; with proper evals, LLMs are magical tools that outperform humans
  • ↓Jev's confidence scores actually validate well in practice and critics haven't shown real failure examples