Measuring the sloppiness of code(earendil.com)
272 points by doppp 1 day ago | 231 comments
tl;dr: LLM-generated code is often formally correct but "sloppy"—verbose, duplicated, or overly complex—and measuring this is hard because LLM-as-judge approaches are unreliable and human review doesn't scale. Using metrics from SlopCodeBench (verbosity and erosion based on cyclomatic complexity), the author found agent-generated code is roughly twice as verbose and eroded as human-written code in established repos. On iterative benchmarks where context resets between rounds, even state-of-the-art models score 0% strict pass rate, suggesting agents can't clean up their own accumulated slop.
HN Discussion:
  • ~Global architectural issues matter more than local sloppiness metrics
  • Quality metrics and dedicated tooling are needed to enforce good code structure for agents
  • AI labs will eventually address maintainability once correctness gains plateau
  • Human devs and mental models remain essential; coding isn't solved by AI
  • ~Questions whether code complexity even matters if LLMs can handle it themselves