Jepsen tests ParticleDB, an LLM-assisted database for AI agents: 96 bugs, invented docs, and fixes that weren't

Kyle Kingsbury's Jepsen analyses usually end with a list of isolation anomalies. His report on ParticleDB, published on October 31, also has a section called "LLM Tradeoffs", and it is the part people are quoting.

ParticleDB is a Rust database from Orkes, the company behind the Conductor workflow engine, pitched as a single engine for AI agents and applications: transactional SQL, a key-value store, vector search, live analytics and user-defined code. Orkes paid Jepsen to test it, as it did earlier for Conductor. Kingsbury tested versions 1.1.4 through 1.1.18.11.rc-22 and found 96 issues. Most needed just a single server, a single client and no faults.

What broke

The list reads less like a consistency report and more like a catalogue of SQL behaving at random. Versions up to 1.1.17 treated WHERE something = x as a primary-key lookup of x, so WHERE 4 = 5 returned the row with key 5. WHERE TRUE could match nothing. A high-precision literal 0.90... was apparently equal to 5. Constraints were violated, primary keys could become null or be duplicated, rows from dropped tables re-emerged "in mangled forms", strings came back as eight-bit integers, and whole columns occasionally vanished. Each transaction and write-ahead-log mode showed essentially every anomaly Jepsen's checker can detect, from lost committed writes to G2.

The marketing said otherwise. The home page claimed "Full ACID. 159K TPS.", with "snapshot isolation" in the text right below, which is not full isolation. It also said ParticleDB was "formally proven" with TLA+, but no TLA+ specification was available during the engagement. It claimed to be "1087x faster than PostgreSQL"; in Jepsen's runs it was nowhere near that, and throughput declined linearly over an hour, even on small datasets.

By the last build tested, all but ten issues appeared resolved. The ten that remained are not small: the default occ mode still lost committed writes in normal operation, and the new serializable mode still allowed stale reads, plus internal anomalies and Lost Update when the process was killed. Jepsen recommends that users "halt further deployment of ParticleDB and migrate to other databases as soon as possible".

The documentation made things up

This is the part that will look familiar to anyone who has read unchecked model output. According to the report, the Java SDK documentation "referred to non-existent dependency coordinates and seemed to make up class and package names which were nowhere to be found in the library itself". The quickstart and client examples "made up a variety of default usernames for ParticleDB, all of which were wrong". The stored-procedure documentation "even fabricated an imaginary email address for the author of this report". Running particledb --version on builds 1.1.16 through 1.1.18.11.rc-18 reported "spanner 1.1.16".

The report calls the docs "riddled with omissions, inconsistencies, and confabulations" and recommends Orkes work with a human technical writer to rewrite them.

"Perhaps no one knows"

The LLM section is careful about what it attributes. Kingsbury writes that "with LLM assistance, a handful of engineers at Orkes were able to produce a highly sophisticated database system", and a footnote sets that against MongoDB's 962 employees in 2018 and Cockroach Labs' 400 in 2022. Then come the costs. "LLM tooling repeatedly claimed that bugs had been fixed when, in fact, they had not." Orkes engineers had explained the Conductor bugs in detail, but here they "seemed unable to explain the causes or resolutions of nearly every issue". New builds "simply materialized, sans changelogs". On why ParticleDB turned strings into integers, and why it stopped: "Perhaps no one knows. With LLMs, perhaps no one needs to."

The testing story is just as odd. Orkes said it had run Jepsen's existing Postgres harness and SQLancer before the engagement, yet that harness "failed immediately" against 1.1.4. Kingsbury offers three possibilities: tests weren't run, the suite didn't exercise basic functionality, or "engineers entrusted Large Language Models with the work of testing". He doesn't pick one.

What actually found the bugs

The most reusable lesson is about method. Jepsen traditionally analyses a small set of carefully chosen transactions. Here, roughly two thousand lines of generative tests found a wealth of bugs: a compare workload generated random schemas and simple SQL, ran the same statements on ParticleDB and PostgreSQL, and searched for a minimal sequence that reproduced any divergence. Kingsbury writes that this "should not have worked", since two Postgres-compatible databases can legitimately differ in many ways. It worked because ParticleDB failed at "low-hanging fruit" an ordinary test suite should have caught, and Jepsen has since extracted part of the SQL generator into a shared library for other tests. Using a mature database as the oracle for a new one is a cheap check that anyone building systems with coding agents can borrow.

Orkes' side

Orkes says ParticleDB 1.1.19-rc.13 resolves every remaining issue in the report; Jepsen hasn't tested anything after 1.1.18.11.rc-22. In its own launch post on October 30, Orkes engineer Nikhil Kala writes that "as of ParticleDB 1.1.19, we report that all identified issues have been addressed". The post doesn't mention LLMs.

According to the report, Orkes and some of its customers ran ParticleDB 1.0.0, a limited build for Conductor that passed Jepsen's cursory tests, while the refactored 1.1 line was soft-launched on particledb.ai. From May through July 2026 the download page offered at least 1.1.4 through 1.1.18.11.rc-18, and the Docker page explained how to deploy them "for production". Only after testing did Orkes relabel those builds "unsupported pre-release builds" and the transaction modes "experimental".

Kingsbury's closing questions are aimed at the whole industry: "In the age of LLMs, what can maintainers do to ensure correct software and accurate documentation? Having done that work, how can they demonstrate its quality to users?" In ParticleDB's case, the evidence came from someone else's test suite.