Vercel's AI Gateway unifies private models and catalog models in a single API

- Reading
- 5 min
- Code
- 2
The AI Gateway's Private Inference feature lets you plug fine-tuned, self-hosted, or prerelease endpoints into the same authentication, SDKs, and observability used for Vercel's catalog models.
What changes: one gateway for everything
Until recently, teams that maintained fine-tuned, self-hosted, or prerelease models needed a parallel infrastructure: separate API keys, their own client code, and a logging pipeline isolated from the rest of the stack. That was the price of stepping outside a provider's ready-made catalog.
Vercel's AI Gateway now offers the Private Inference feature, which connects these in-house endpoints to the same authentication layer, SDKs and APIs, and observability used by catalog models (OpenAI, Anthropic, and so on). In practice, the model you trained or host becomes just another option for model in your call, alongside what's already available.

In short: AI Gateway stops being just a router for third-party models and also becomes the single entry point for any model your company controls, whether bought or trained in-house.
How to connect your own endpoint
The flow described in Vercel's documentation follows three steps:
- Open the Private Inference page in the AI Gateway dashboard and register the endpoint as a "private provider," providing the base URL, the API format (Chat Completions, Responses, or Messages), and the credentials.
- Register the models that provider serves, giving each one an internal slug.
- Call the model by its
private/<slug>ID anywhere you already use a catalog model today.
The same process can be done via the Vercel CLI, version 65.16.0 or later, which matters for teams automating infrastructure in CI/CD:
# Connects your endpoint as a private provider
vercel ai-gateway private-inference providers create acme-vllm \
--base-url https://vllm.example.com/v1
# Exposes the endpoint's model as private/support-fine-tune
vercel ai-gateway private-inference models create support-fine-tune \
--provider acme-vllm \
--api-model acme/support-llama-ftNote that the endpoint can be anything compatible with the supported formats, including your own vLLM server, which covers the common scenario of teams that fine-tune models and host inference themselves.
From the command line to code: the practical example
Once registered, the private model behaves like any other in the AI SDK. Vercel's own example shows usage with streamText, including a fallback to a catalog model in case the private endpoint fails:
import { streamText } from 'ai';
const result = streamText({
model: 'private/support-fine-tune',
prompt: 'Draft a reply to a customer who was charged twice.',
providerOptions: {
gateway: {
// Optional fallback to a catalog model
models: ['anthropic/claude-sonnet-5.5'],
},
},
});
for await (const text of result.textStream) {
process.stdout.write(text);
}Switching from a private model to a catalog model (or vice versa) literally becomes just changing the value of model. This matters for teams that currently maintain separate code branches or wrappers just to switch between an internal fine-tuned model and an off-the-shelf model: the calling logic becomes the same.
Observability and fallback: what it actually solves
The most practical point of the announcement, for those running production workloads, isn't the connection itself, but what it avoids duplicating:
- Unified logs: requests to private models appear alongside catalog requests in AI Gateway logs, trace drains, and custom reports, with the same volume, token, and time-to-first-token charts broken down by model.
- Scoped by team: private models stay restricted to the Vercel team that registered them, don't appear in the public catalog, and credentials are stored encrypted.
- Zero Data Retention per provider: you can enable, provider by provider, a zero-retention policy, something regulated teams (healthcare, financial) tend to require contractually.
- Coding agents: private models become available to Claude Code, Codex, Cursor, and other agents already connected to AI Gateway, without separate reconfiguration.
This directly addresses the fragmented observability problem: previously, a fine-tuned model running outside the catalog created a blind spot in the dashboard, forcing the team to build its own telemetry or accept operating in the dark.
The trade-offs nobody announces in the changelog
Not everything here is a free gain. Putting your own endpoint behind AI Gateway means every call passes through a Vercel format-translation layer before reaching your server, which adds a network hop and one more dependency in the availability chain.
There's also a trust aspect: even with encrypted credentials and optional Zero Data Retention, traffic from your proprietary model passes through Vercel's infrastructure before reaching the endpoint you control. For teams with strict data sovereignty requirements, this can be a point of friction to check with the compliance team before migrating.
And there's the opportunity cost of lock-in: the more your application depends on the AI SDK and AI Gateway's call format to orchestrate your own models, the more expensive it becomes to switch gateways in the future. This isn't a problem exclusive to Vercel, but it's compounded when you also move the private part of your AI stack into that same router.
When it's not worth it
If your team only uses catalog models (GPT, Claude, Gemini via public API) and never intends to host or fine-tune anything, Private Inference simply doesn't apply: there's no gain in unifying what's already unified. The feature also matters less for those who don't use Vercel's AI SDK or AI Gateway today, because the central advantage is precisely reusing an observability and authentication layer that already exists in your stack.
It's worth a closer look if your company already maintains (or plans to maintain) fine-tuned models in production, especially those served by something like your own vLLM: in that case, trading duplicated logs and authentication for a single dashboard is a real reduction in operational overhead, not just a marketing promise. The full documentation on supported model types, requirements, and billing is on Vercel's site, and it's worth reading before migrating anything that's already in production.
Translated from the Brazilian Portuguese original · Read the original
Seedance 2.5 Flash joins Vercel's AI Gateway catalog for cheaper AI video
ByteDance launched a lighter version of Seedance 2.5, and Vercel has already made the model available in the AI Gateway, integrated with AI SDK 7. For those building generative video features, this means less latency and lower cost per clip, without switching providers.





