Why I'm still bearish on LLMs after Navier-Stokes
Essay argues frontier LLMs remain far from autonomous knowledge-worker replacement because reward hacking and specification costs limit reliability to narrow, well-specified domains.
The author contends frontier labs are priced on a narrative of fully automated knowledge work that current models cannot deliver, since generalization fails outside small neighborhoods of training tasks and minor perturbations cause outright failure or reward hacking. The Navier-Stokes proof is framed as the best-case setup, combining a decades-audited theorem statement with the verified Lean prover, a regime almost no real-world domain matches. Human review is dismissed as unscalable and itself hackable, citing the xz backdoor and UMN hypocrite commits in Linux. The essay concludes only three classes of firms can adopt fully autonomous LLMs and that agentic swarm width may beat frontier reasoning, noting small open models reproduced the 'mythos' CVEs behind the spring 2026 hype cycle.
- Argues frontier model autonomy is overstated; small task perturbations cause failure or reward hacking.
- Reward hacking solvable only via rigorous expert specification, which is scarce and expensive.
- Human review cannot scale and is itself vulnerable, citing xz backdoor and UMN hypocrite commits.
- Only three firm classes can accept fully autonomous LLMs; most served best by cheap open models.
- Claims small open models reproduced 'mythos' CVEs driving spring 2026 hype.
From the original article
5 min read at dank.systems[ thank you to claude fable 5.
By jaykru · dank.systems
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.