Perplexity swaps DynamoDB for in-house database and cuts latency by 5x

Perplexity, the company behind an AI-answered search engine, migrated the serving layer that powers its searches from Amazon DynamoDB to CobbleDB, a distributed key-value database built in-house in Rust. The information was published by InfoQ on September 25, 2026, based on details disclosed by Perplexity itself. The reason wasn't an engineering whim: at volumes above 200,000 requests per second, DynamoDB's pricing model (which charges for every byte transferred) had become an unsustainable bill.

Why DynamoDB stopped making sense

The read pattern of an LLM-answered search engine differs from that of a traditional search. Each query made to Perplexity generates between 100 and 120 target page keys, which the retrieval service splits into parallel batches of 10 to 20 keys. Instead of returning just a metadata snippet, as a conventional search does, the system needs to extract full text passages and dense vector embeddings per record, which produces average payloads of about 50 KB, much heavier than a typical metadata database read.

Beyond the cost, there was an opacity problem: DynamoDB acts as a black box when it comes to partition placement, in-memory cache policy, and replica routing. The engineering team couldn't prevent tail-latency spikes caused by cache-missed reads, cross-availability-zone network hops, or lagging replicas. To make things worse, reprocessing jobs triggered by changes to the chunking algorithm or by newer embedding models threw heavy writes directly at DynamoDB, competing (the classic "noisy neighbor" problem) with production users' reads.

Three systems, three responsibilities

Instead of optimizing inside DynamoDB, Perplexity split the problem into three specialized pieces:

• Pillar: runs on YTsaurus over high-capacity spinning disks, maintaining versioned table families for page metadata, passages, and vector representations. YTsaurus's atomic transactions ensure that crawl updates, state mutations, and export queues are committed together.
• Lorry: a stateless queue consumer that groups Pillar's exports into partition-aligned batch files, storing the payloads in Amazon S3 and posting metadata notices to CobbleDB.
• CobbleDB: the serving nodes independently pull and ingest these batches from S3, fully isolating the low-latency read nodes from the crawler's heavy write pipeline.

This separation is, in itself, an architecture lesson: instead of a single database trying to serve intensive writes and low-latency reads at the same time, Perplexity decoupled durable storage from hot retrieval.

How CobbleDB works internally

CobbleDB is a distributed key-value store optimized exclusively for batched lookups. Each partition maintains three replicas spread across independent compute nodes. The main daemon uses RocksDB as its embedded storage engine, combining memory-mapped caching with local NVMe disks.

A stateless query router hashes page identifiers to partitions and coordinates read execution, prioritizing replicas in the same availability zone to reduce network overhead. If a target replica shows elevated response time, the router speculates: it fires a parallel read to an alternative replica on another node (the technique known as hedged reads). Within each node, CobbleDB retrieves keys simultaneously through RocksDB's batched MultiGet interface, eliminating round-trip overhead:

pub struct BatchedPageRequest {
 pub keys: Vec<PageKey>,
 pub zone_affinity: AvailabilityZone,
}

impl StorageEngine {
 pub fn multi_get_pages(&self, keys: &[PageKey]) -> Result<Vec<Option<PageRecord>>, Error> {
 let rocksdb_keys: Vec<&[u8]> = keys.iter().map(|k| k.as_bytes()).collect();
 self.rocksdb.batched_multi_get(&rocksdb_keys)
 }
}

Another deliberate decision: the database abandons distributed transaction protocols and synchronous consensus algorithms. Since search serving tolerates a small replication delay, replicas apply updates asynchronously, at their own pace, which greatly reduces operational overhead.

The numbers behind the switch

In production measurements, CobbleDB reduced median batch read latency from 31.4 ms to 5.60 ms, p90 dropped from 56.7 ms to 9.77 ms, and p99 (tail latency) plunged from 123 ms to 24.2 ms. Synthetic benchmarks with payloads up to 100 KB confirmed consistent throughput of up to 500,000 requests per second. On storage, savings came to at least 20% compared to the previous DynamoDB cost.

The price of trading managed cloud for in-house engineering

The architecture has trade-offs that Perplexity doesn't hide. Replacing a fully managed database shifts the entire node lifecycle onto the company's own site reliability engineers (SREs): backup verification, partition rebalancing, hardware failure management. Applications also have to live with eventual consistency, since replicas ingest batch files at different intervals. This isn't a trivial trade-off, and that's exactly why it only makes sense at extreme scale: for most teams, the cost of maintaining dedicated infrastructure engineering still outweighs what's saved by leaving a managed service.

Two engineers, two months, and AI agents

One detail caught attention in the announcement: according to Aravind Srinivas, CEO of Perplexity, the 40,000 lines of Rust that make up CobbleDB were written in two months by just two systems engineers, working alongside an autonomous swarm of AI coding agents that handled integration tests, build monitoring, and operational runbooks. The company also signaled plans to open-source CobbleDB's code, with no set date so far.

What it means for those building in Brazil

Perplexity's case isn't a blanket invitation to abandon managed databases: the decision only pays off above hundreds of thousands of requests per second, when the per-byte-transferred pricing of a service like DynamoDB stops being marginal and becomes a significant line item on the cloud bill. For most teams, migrating to an in-house database would trade a cost problem for an operations problem (node management, backup, rebalancing) that few teams have the bandwidth to sustain.

What's worth taking away is the architectural pattern: separating durable storage (here, Pillar) from a serving layer optimized solely for low-latency batch reads (CobbleDB) is a replicable strategy in any system that suffers from read hot paths competing with heavy write pipelines, even in stacks much more modest than Perplexity's. It's also worth studying CobbleDB's specific techniques: hedged reads to cut tail latency, RocksDB with batched MultiGet to eliminate round-trips, and the deliberate choice to trade strong consistency for availability and speed when the problem domain tolerates replication delay.

The use of AI agents as part of the systems engineering team, not just to generate application code but for integration tests and operational runbooks of a production database, is also a sign of what's already happening in leading-edge infrastructure teams and should keep making news here. It remains an open question when and how Perplexity will actually open-source CobbleDB's code, something that, if and when it happens, deserves coverage of its own.