How Vercel cut 91% of edge metadata latency (and what it means for you)

Every request that reaches Vercel passes through the company's CDN, which executes, on average, more than 80 million routing instructions per second. Part of that work is looking up route metadata: figuring out which paths exist and how to serve them. When that metadata isn't cached, the CDN has to fetch it before responding, and that's exactly the step Vercel's engineering team targeted in the changelog published on September 10, 2026. The result: metadata lookup P99 dropping from 215.8 ms to 19.1 ms, a 91% reduction.

For anyone building with Next.js and deploying on Vercel, this gain is invisible in the code but real at the edge. And for anyone running their own edge (Cloudflare Workers, a homegrown CDN, route caching at any layer), the how is replicable.

Where the latency comes from: cache misses per path

The problem isn't obscure. A request's path doesn't always match the path of the content that serves it. /blog/hello-world might resolve to the dynamic route /blog/[slug]; that page's React Server Component payload might resolve to /blog/[slug].rsc. The CDN applies routing rules (declared by the framework via the Build Output API) and, in the process, has to check several target paths before finding the right response.

Before doing the exact lookup, Bloom filters in global routing discard paths that definitely don't exist. What's left needs an exact metadata query. And here was the bottleneck: Vercel stored metadata as a separate object per path, fetched and cached individually.

This worked fine for small projects that deployed infrequently. The problem shows up at the opposite end of the scale: large deployments can have hundreds of thousands of paths, each with its own cache entry. Since every new deploy generates a fresh set of cache keys, the first query to each path always resulted in a cache miss. Large sites that deploy frequently paid that cost over and over.

The idea: fetch in groups, parse only what's needed

The solution was to group the metadata of many paths into files called shards. Fetching a shard brings the metadata of all its paths into the cache at once. A per-path cache fill warms up one path; a shard fill warms up every path assigned to that shard.

The detail that keeps this from trading one problem for another: each shard carries an internal index that lets the CDN locate a specific path's metadata without decoding, decompressing, or parsing the other entries. Each fetch warms many paths, but each lookup still processes only the record it needs.

The structure reuses layers Vercel already had for other routing datasets. Shards are built in JSONL (one JSON value per line), with sorted, interleaved key-value records inherited from Bulk Redirects, and directly addressable Base64 structures borrowed from Bloom filters. The team deliberately separated the data layout from the lookup structures sitting on top, so each workload can choose what it needs:

• sorted, inspectable, randomly accessible JSONL key-value records;
• optional inline indexes for low-overhead binary search;
• embedded Base64 data with offset-based decoding;
• capped shards that keep transfer and cache costs in check.

Index pointers are stored as fixed-width numbers, each an exact multiple of six-bit Base64 characters. That way, a pointer can be decoded in place without first parsing the index line as JSON or decoding the entire Base64 blob. Routing runs a binary search over the encoded paths using byte offsets calculated at build time, finding the path in O(log n) pointer reads and string comparisons. Only then does it parse the JSON value on the following line. The rest of the shard stays untouched.

Why 200 KB and not one giant shard

The most instructive part of the post is the size experiment. The team's initial intuition was that most deployments' metadata would fit into a single shard, so they started with shards of several megabytes, betting that the index and binary search would keep parsing cheap.

What broke that bet was cache topology. Each routing process keeps a small in-memory LRU cache of recent shards, in front of a larger regional cache shared by all processes in the region. In testing, the regional hit rate was high, but the LRU hit rate was low: requests spread across many processes in each region, so each process rarely encountered the same shard twice. Transferring multi-MB shards also turned out more expensive than expected.

Large shards made LRU misses too slow, while tiny shards made regional misses too common.

The practical balance, found in production, landed around 200 KB: a high regional hit rate and a cheap LRU miss to fill. The numbers, measured on production traffic between August 5 and 12, 2026:

| Lookup metric | Before (per path) | After (indexed shards) | Gain | |---|---|---|---| | P99 | 215.8 ms | 19.1 ms | 91% lower | | Average | 8.59 ms | 1.81 ms | 79% lower | | Standard deviation | 44.9 ms | 19.0 ms | 58% lower |

It's worth noting what was left out: the team evaluated shrinking shards further with front-coding of sorted paths, JSONL documents that deduplicate metadata, and a more compact custom serialization format. All of them produced considerably smaller shards, but simulations predicted only modest latency gains. The conclusion was honest: the extra work of encoding, compatibility, and rollout wasn't worth it for this migration. The optimization is shelved until another workload justifies the cost.

How to swap routing without serving 404s

This lookup runs on every request for every deployment. If the sharded metadata disagreed with the per-path version, the result would be a stale route, a wrong status code, or a 404 on a path that exists. The validation process is a good playbook for any change on a critical path.

First, offline: the team set up test deployments and ran a harness that queried each path through both paths and compared the responses. Then, in production, behind a feature flag, routing made both queries on a random sample of requests, but kept serving the old result. The comparison ran in the background, for several weeks, without slowing down production. It's the pattern Vercel calls shadow mode.

Shadow mode found discrepancies, and they were exceedingly rare. One was a bug in the old encoding, which packed paths into RFC 2047 encoded words to fit non-ASCII text into ASCII fields, and it only showed up when an emoji was split across two words. The new format stores paths as plain UTF-8, so the bug can't happen. Finding that edge case increased confidence in the comparison, and the shards started serving real traffic.

The bonus: faster builds

With shards in production, the team went back to the build pipeline and removed the work that had become redundant:

• skipping the per-path metadata upload saves ~9.7 s;
• writing route-group metadata directly into the manifest saves ~4.5 s;
• not uploading the files that ended up empty saves ~2.4 s.

That's about 16.6 s per deploy. In aggregate, the deploy step got ~10% faster; on metadata-heavy deployments, where these steps dominate, the estimate approaches 25%.

What this changes for you

If you deploy Next.js on Vercel, the Build Output API contract hasn't changed: the framework still describes what the application needs, Vercel improved how the CDN serves it. Deployments made after July 17, 2026 already use the new shards. For an older deployment, a redeploy is enough to pick up the fast lookups. On Vercel's own marketing and docs sites, lookup P99 dropped from 203 ms to 31 ms, and route resolution P99 on large sites got roughly twice as fast.

The more lasting value here is the architectural pattern, not the feature. If you maintain any per-key metadata cache layer (routing in Workers, a homegrown proxy, tenant resolution in a multi-tenant SaaS), the lesson is the same: group the fetch without grouping the parsing. An internal index with fixed-width pointers and binary search lets you bring in many records in one fetch and still parse just one. And block size can't be deduced on a whiteboard: it depends on how your cache distributes across processes, something only production measurement reveals. Vercel only landed on 200 KB because the LRU behaved differently than expected.