Benchmarking Opus 5 on SlopCodeBench

Opus 5 achieved only a 24% strict pass rate on a subset of SlopCodeBench, a new long-horizon benchmark from UW Madison that tests models on evolving codebases with undisclosed requirements across multiple checkpoints. All tested models (Opus 4.8, Sonnet 5, Opus 5) failed to complete any challenge defect-free, with Opus 5 writing five times more functions than Opus 4.8 and showing increased code smell over time. The benchmark remains unsaturated—GPT-5.4 and Opus 4.6 scored 11% and 17% respectively in the original paper—suggesting current models cannot reliably handle real-world, iterative software engineering without human steering.

humanlayer