PlanetScale launches full-text search extension for Postgres, and ParadeDB rebuts benchmark in two weeks

What PlanetScale launched, and why it draws attention

Two weeks before October 1, 2026, PlanetScale released TIN, a full-text search extension for Postgres. The curious detail is that PlanetScale is historically known for MySQL (via Vitess), not for Postgres. TIN, however, is specifically an extension for the elephant database, and the launch post brought benchmarks showing a significant advantage over ParadeDB, a rival Postgres search extension based on the Tantivy library.

According to the account published by Ming Ying, a ParadeDB engineer, on the project's blog (republished on the Planet PostgreSQL aggregator), TIN showed up with strong numbers: at least 8 times faster than ParadeDB 0.25 in BM25-ranked search and in document counting, using the same StackExchange dataset.

TIN is fast (at least 8x faster than ParadeDB 0.25 in every PlanetScale benchmark).

Ming Ying, ParadeDB engineer

This kind of gap usually points to a difference in architecture, not just in implementation. And that is exactly what the TIN post claimed: using Postgres's native ctid as the document identifier, eliminating an intermediate mapping that ParadeDB needs to maintain.

How ParadeDB closed the gap without changing its architecture

The ParadeDB team decided to investigate before accepting the ctid thesis as the root cause. Running EXPLAIN (ANALYZE, BUFFERS) on a simple BM25 Top K query, they found something revealing: 83% of Postgres page accesses were to read fieldnorms, the structure that normalizes the score by document length.

The problem wasn't the size of the fieldnorms (each one takes up a single byte), but the access locality: Tantivy stores fieldnorms in an array indexed by DocId, separate from the postings lists, which produces scattered reads across the page. The fix was conceptually simple: duplicate the fieldnorms array alongside each postings list, in the same DocId order, trading random reads for sequential reads.

The result: fieldnorms page accesses dropped from about 1,500 to 30 for the same query. The cost is additional storage (the Hacker News index of 28.7 million documents grew by about 9%), but the I/O gain more than made up for it.

The second optimization: switching the pruning algorithm

Even after the fieldnorms fix, disjunction queries with many terms (of the type termA OR termB OR termC...) remained slow. Profiling pointed to the bottleneck in Blockmax WAND, the algorithm Tantivy uses to skip posting blocks that cannot exceed the current Top K's minimum score.

There are two families of pruning algorithms: WAND, which skips more but spends more CPU deciding what to skip, and MAXSCORE, which skips less but with less overhead. When a query has many terms, WAND's cost grows and starts to outweigh the gain. Lucene had already gone through this same migration in 2023, adopting MAXSCORE for certain query formats.

ParadeDB implemented a simple selector: use MAXSCORE for disjunctions with three or more terms and dense postings, and keep WAND for the rest. On a ten-term query, p50 latency dropped by about 6 times and p95 by about 8 times, enough for ParadeDB to become 2 times faster than TIN on the Hacker News dataset.

The original benchmark had two configuration problems

Beyond the code optimizations, the ParadeDB team found two biases in how PlanetScale's benchmark was set up.

The first is syntactic: the queries used ParadeDB's @@@ operator without qualifying the search field (<query> instead of <field>:<query>). In the StackExchange dataset, this makes ParadeDB scan two indexed text columns per query, while TIN scanned only one. Switching to the native |||, &&&, and ### operators fixed the distortion.

The second is more interesting for anyone evaluating correctness, not just speed. TIN uses a technique called dense-term elision: at query time, it ignores the score of any term that appears in more than 10% of the corpus (the dense_ratio parameter). It's a performance shortcut, but it has a price.

In short: when the ParadeDB team measured the impact of this elision on ranking precision, the numbers are concerning for anyone who treats search as something that needs to be correct, not just fast:

In 6.4% of queries, none of the Top 10 results matched exact BM25. And part of the benchmark's own test queries were sequences of common words ("is it", "to a"), precisely the scenario where elision distorts ranking the most.

The final comparison: exact BM25 against exact BM25

With the configuration fixes and optimizations applied, ParadeDB republished the numbers comparing exact BM25 on both sides, along with the scenario with elision enabled in TIN (as in the original benchmark):

In other words: when both engines compute the same score, ParadeDB comes out ahead. TIN's advantage only appears when it's answering a slightly different question (approximate BM25).

ctid versus DocId: the open question

TIN's central thesis was that using Postgres's ctid as the document identifier is superior because it eliminates a translation step. ParadeDB disagrees, and the technical argument matters to anyone designing a search index on top of a relational database.

A ctid concatenates a block number (in the millions) with a tuple position (at most 291), two different numeric domains inside the same 48-bit integer. This compresses worse than a dense, sequential 32-bit DocId, which is the standard for most search engines (including Lucene). Postings list compression depends directly on this.

There's also the problem of columnar storage, needed to sort by field, apply a range filter, or generate facets, three operations common in product search. With a dense DocId, document 42 maps directly to row 42 of each column. With ctid, you need to map the physical (block, slot) position to a columnar position before fetching any attribute, the same translation cost that TIN claims to have eliminated, just in the opposite direction.

What changes for those choosing between native Postgres and Elasticsearch

For those considering taking Elasticsearch out of the stack and concentrating full-text search inside Postgres itself, this episode leaves three practical lessons.

• A launch benchmark is a starting point, not a verdict. In two weeks, the competitor closed an 8x gap without changing its architecture, just by adjusting data locality and switching pruning algorithms. Review the configuration (dense_ratio, field qualification, stopwords) before deciding on any migration.
• Correctness weighs more than throughput. An approximate BM25 ranking may be enough for much of the use cases, but the decision to use common-term elision should be explicit and documented, not a default setting hidden in a marketing benchmark.
• Code openness matters in the evaluation. ParadeDB is an open source extension that runs on any self-managed Postgres; TIN, as of this publication, only exists inside PlanetScale's managed infrastructure, which limits portability for those already operating their own cluster.

ParadeDB has already cut a release candidate, 0.26.0-rc.2, with these optimizations, and promises the stable 0.26.0 release the week after publication. The changes are backward compatible, but require reindexing to inherit the locality gains. Ming Ying's post is Part I of a series; Part II is expected to cover optimizations in count queries (COUNT), which use a different code path than BM25 Top K.