Why the creator of Jepsen distrusts documentation written with LLM help

The creator of Jepsen, Kyle Kingsbury, raised a warning: LLMs have made it easier to fake maturity in software projects. For those who choose dependencies day to day, this changes what's worth checking before trusting.

One sentence from someone who built a career distrusting databases

Kyle Kingsbury is known for one thing: not taking the vendor's word for it. He created Jepsen, the testing framework that simulates network partitions and hardware failures to see whether a database actually delivers the consistency guarantees it promises on its website. The project became known precisely for subjecting distributed databases to this kind of adversarial testing, regardless of what the vendor's documentation promised. When this guy says he no longer trusts the signals he used to evaluate software maturity, it's worth paying attention.

The line was collected by Simon Willison on his blog on November 1, 2026, taken from a Jepsen report on the ParticleDB database, version 1.1.4. It sums up a discomfort that had already been brewing:

More broadly, the use of LLMs erodes trust in documentation, in software, and in the organizations which produce them. Perhaps engineers will begin to treat any new software project with suspicion—the signals we used to rely on as markers of a mature system are now easier to counterfeit.

Kyle Kingsbury, creator of Jepsen, in a ParticleDB 1.1.4 test report

I agree with the diagnosis. I disagree a little with the implicit conclusion of distrusting everything, because that also punishes whoever is on the right side of history. But let's take it step by step.

The signals we used to measure maturity no longer work

Before LLMs became a work tool, a well-written README.md, with tested code examples, a troubleshooting section, and a detailed changelog, was a reasonable proxy for care. Writing that took work. A project that invested time in documentation had usually also invested time in the code.

That proxy broke. Today any generative AI tool writes a complete README.md, with an installation section, examples, FAQ, and even a text diagram, from a thirty-second prompt. The text turns out good. It says nothing about whether the code works, whether the tests actually cover the hard cases, or whether the maintainer knows what they're promising.

The same goes for other signals the dev community learned to read:

• Detailed commit messages: today they come out of a coding assistant in seconds, with no direct relation to the quality of the diff
• Well-argued pull request descriptions: can come from an LLM that never ran the test it claims to have run
• Changelog organized by category (Added, Fixed, Breaking): a trivial format to generate, hard to verify without opening the diff

None of these signals was definitive proof before. But they cost enough time to work as a rough filter. That filter no longer exists.

The paradox left for the honest maintainer

This is where the hardest part of Kingsbury's provocation lies, which he makes explicit in the second question of the same post:

In the age of LLMs, what can maintainers do to ensure correct software and accurate documentation? Having done that work, how can they demonstrate its quality to users?

Kyle Kingsbury, creator of Jepsen

A serious maintainer also uses an LLM to write documentation faster. That's legitimate and, in most cases, results in better docs than they would write alone, with less time spent. The problem is that, from the outside, their README.md and the README.md of a poorly tested package look alike. Prose no longer distinguishes the two cases.

That's not a reason to dismiss AI-generated documentation across the board. It would mean throwing out legitimate use along with abuse, and most of the projects I use day to day already use generative AI at some point in the writing. The right cut isn't who wrote the text, it's what can be verified beyond the text.

What changes in practice for those choosing a dependency

If prose stopped carrying a reliable signal, the evaluation work needs to shift to verifiable artifacts. In practice, before adding a new dependency to a project, I would prioritize:

1. Running npm test (or equivalent) locally and checking whether what passes matches what the README.md promises, instead of trusting the coverage badge
2. Looking at the history of .github/workflows (or an equivalent pipeline): CI that actually runs, with real execution logs, is harder to fake than a well-written paragraph
3. Looking for third-party adversarial analysis, Jepsen-style: someone from outside tried to break the tool's specific promise, whether consistency, security, or performance under load
4. Checking whether closed issues have real technical discussion, with bug reproduction and trade-offs explained, not just generic replies

None of these criteria is new. The difference is that, before, polished documentation still counted as a secondary signal. Now it counts for little, because it costs almost nothing to produce regardless of the real quality behind it.

The counterargument I take seriously

One could argue this has always been the case: pretty documentation was never a guarantee of anything, projects with strong marketing and weak engineering have existed since before LLMs, and what changes is only the cost of producing the veneer. That point is valid, and I won't pretend the problem of companies selling trust they don't have is new.

The difference in degree has become a difference in kind. Before, faking maturity cost real time: writing a convincing README.md, maintaining a changelog, answering issues with apparent competence. That cost worked as a filter, even if an imperfect one. Today the cost has dropped close to zero for anyone, honest or not, right at the moment when more dependencies enter production via npm install with no human review in between.

What remains open

Kingsbury ends with a question that has no easy answer, and I won't pretend I have one. The most honest path I see is shifting trust from prose to reproducible artifacts: a test that runs, a pipeline that fails when it should fail, independent analysis that tried to break the promise. This doesn't solve the scale problem, because verifying an artifact takes work that no one has time to do for every transitive dependency in package.json. But it's better than continuing to trust a text that, today, any model can write the same way, regardless of whether the project is solid or not.

Source: Simon Willison, "A quote from Kyle Kingsbury", Simon Willison's Weblog, November 1, 2026 (https://simonwillison.net/2026/Nov/1/kyle-kingsbury/).