Cloudflare launches automatic router that picks the AI model per request

What the Auto Router is

Cloudflare has put the Auto Router into public beta, an AI Gateway feature that automatically chooses which language model handles each request. Just point the model parameter to cloudflare/auto instead of pinning a specific provider. A classifier decides, request by request, which model in the available pool offers the best ratio between expected quality and cost.

The news, published by InfoQ on October 26, draws attention to an uncommon detail in this kind of announcement: Cloudflare itself published a benchmark in which its router loses to a more expensive model on some tasks. That number, more than the launch itself, is what matters to anyone running agents in production who needs to decide whether it's worth trading model predictability for cost savings.

How the two-stage routing works

AI Gateway processes each request in two stages. In the first, it builds a pool of candidates: it filters out models that can't meet the requested format and applies credentials, billing configuration, access control, and spending limits set by the account.

The second stage brings in the classifier, a multi-head model running on Workers AI over Cloudflare's edge GPUs. It assigns probabilities across 14 task categories and scores the request from 1 to 5 on four dimensions: complexity, ambiguity, stakes (how much an error costs), and dependency on prior context. A scoring matrix combines these scores with benchmark results under the formula stated by the company: utility equals expected quality minus an adaptive cost penalty.

One technical point matters for anyone integrating this: the classifier only looks at the most recent turns of the conversation, with an 8 KiB text limit. The harder the task, the less weight price gets in the final decision.

The benchmark Cloudflare itself loses

Cloudflare ran 97 tasks, with three samples per model and 10,000 bootstrap resamples, comparing the Auto Router against two models used as a baseline: Opus 5.5 and GPT-6 Sol. The result, in the announcement's own words: ten more reliability points for roughly a third of the spend, in favor of the reference models when compared to the router.

In other words: the Auto Router is cheaper, but delivers less reliability than simply pinning an expensive model. Jacques B, a CTO who works on agent engineering, summed up the trade-off in a LinkedIn post cited by Cloudflare itself:

A cheaper model isn't much of a saving if someone has to spend another hour fixing the result.

Jacques B, CTO in agent engineering

For anyone deciding LLM cost architecture, this is more honest information than most routing announcements tend to give: the team knows exactly which side of the curve it's choosing to stand on, instead of buying into the promise of 'same quality, less cost' without verification.

The blind spot: assessing risk only from recent turns

The most criticized part of the technical design is exactly the one that matters most to anyone building agents with real-world actions (payment, refund, changes to sensitive data). Because the classifier only sees the most recent turns, a short confirmation instruction can, in practice, mask the real risk of the action.

Igor Pyrin, a tech lead with experience in banking and healthcare platforms, pointed out the problem:

In a long agent session, the user may say 'this is a customer refund' early and later just say 'go ahead.' I'd carry an application-owned risk label across turns and use it to decide whether the request can use cloudflare/auto at all. A short final instruction should not silently lower the risk classification of the underlying action. The turn that commits the action is the cheapest-looking turn in the conversation.

Igor Pyrin, tech lead

This is the most important practical takeaway of the piece for anyone enabling cloudflare/auto in production: don't delegate the entire risk classification to the router in flows where the final action has financial, medical, or legal consequences. A risk label maintained by the application itself, rather than recalculated at every turn by the classifier, looks like the simplest defense against this blind spot.

Auditing and pool control via headers

Cloudflare documented two mechanisms that help mitigate part of the problem above. The response to each routed request carries the headers cf-aig-routed-model and cf-aig-routing-reason, with values such as cost_optimal_within_pool and pinned_by_turn, which makes the choice auditable after the fact.

It's also possible to restrict the candidate pool per request using the cf-aig-allowed-models and cf-aig-allowed-providers headers. What's still missing, and was requested by Adam Natad in the launch thread itself, is the ability to save these pools as reusable routing profiles, instead of declaring the headers on every call.

The model pool changes, and that affects anyone reproducing the benchmark

One detail that any team trying to reproduce Cloudflare's numbers needs to check before trusting them: the default pool today lists Claude Opus 5, Sonnet 5, and Fable 5, alongside GPT-5.6 Luna, Sol, and Terra, and Grok 4.5. The two models used as a baseline in the benchmark, Opus 5.5 and GPT-6 Sol, are not in the default pool: they're only accessible via opt-in, using the restriction headers cited above.

Cloudflare states that the pool "may change over time," so this could be a side effect of the evolution of available models rather than a deliberate decision. But it means a team running the same test today might get a different result than the one published, simply because the router is choosing among a different set of models than the one used in the original comparison.

Session cost: prompt caching and the switching penalty

There's an extra cost that doesn't show up in the main benchmark: session management. Without a session ID, requests from the same conversation can land on different models on each call and lose the chance to reuse the prompt cache.

Cloudflare also applies a switching penalty that grows with context size, because a new model may have to redo reasoning tokens billed at output pricing. In practice, this means that in long conversations with multiple turns, keeping the same model throughout the session (instead of letting the router switch on every message) tends to come out cheaper than a quick read of the announcement suggests.

The debate over the unit of cost

On Reddit, commenter reachjalil rejected the product's premise, arguing that automatic routers end up costing more by delivering less intelligence, since a more capable model tends to be cheaper when what matters is the final outcome. User yangqi replied that the correct unit of comparison isn't the request, but the complete task.

In the reply, reachjalil touched on something Cloudflare itself acknowledges: classifying a task's difficulty is, itself, easier work for a large model, and the classifier's weights go stale quickly, in their words, "especially with models changing all the time." Indeed, Cloudflare's announcement confirms that routing weights come from benchmark results and that new models need to be added manually to the scoring matrix, with no automatic update.

What's still missing

The Auto Router is free during the beta, with no price set for afterward. WebSockets aren't supported, so it only works with Chat Completions and the Responses API, which may exclude some bidirectional streaming flows already in production.

On the roadmap is cloudflare/auto-best, a variant that uses the same classifier and the same pool, but optimizes only for the highest expected quality, without factoring in cost. For anyone deciding to adopt either one today, the practical takeaway from this round of criticism is clear: use the automatic router for low-risk traffic, and keep a pinned model, with a risk label defined by the application itself, in any flow where the final action has a real consequence.