DeepSeek open-sources DeepInfer, an inference framework that runs on Nvidia GPUs and Huawei Ascend chips

What DeepInfer is

DeepSeek open-sourced, on October 20, 2026, the inference framework that runs behind the company's own large-scale model services, named DeepInfer, according to TechNode. The framework is the piece that decides how a trained model responds to each request in production: how to distribute processing across machines, how to accelerate text generation, and how to deliver output in structured format. Until the day before the announcement, this layer remained closed within DeepSeek's own infrastructure.

Context: from the model to the engine that runs the model

DeepSeek already had a track record of open-weighting its models, as happened with the R1 family and with versions of the V series, a move that helped popularize the idea of open-weight models competing with closed products from OpenAI and Google. What changes now is the layer below the model: it's no longer just "here's the weight for you to download," it's "here's the engine we use to serve this weight at scale." This is usually the part kept under lock and key, because it's where the efficiency gains that define the cost per token charged to the end customer live.

Two chips, one inference stack

Unlike inference frameworks designed only for Nvidia's architecture, DeepInfer supports both Nvidia GPUs and Huawei's Ascend processors. Support for Ascend has a concrete reason: United States export restrictions limit Chinese companies' access to Nvidia's most advanced chips, which pushes part of the local ecosystem toward Huawei hardware as a viable alternative at scale.

Among the features released are distributed inference, speculative decoding (a technique that uses a smaller model to predict tokens and speed up generation from the main model), and structured output, a format designed for integrations that require JSON or predictable schemas in the response, such as function calling. According to DeepSeek, the stated goal is to help developers and companies improve throughput and latency while reducing computational cost and energy consumption.

The framework currently supports the DeepSeek-V4.1-Flash-1017 model, and the company says it will keep publishing inference code and optimizations for upcoming models in the family through the same project.

The package comes with deepseek-recipe

Along with the main framework, DeepSeek also open-sourced deepseek-recipe, a component that converts API requests and handles model input and output following the same pattern used in the company's commercial service. In practice, it's the layer that speaks the language of DeepSeek's public API, useful for anyone who wants to locally reproduce the exact behavior of the hosted service, without rewriting prompt and response parsing from scratch.

Why this matters for those running AI in Brazil

In short: those building GPU inference infrastructure in Brazil face two problems of the same size: Nvidia GPU prices inflated by import taxes and global demand, and dependence on a single hardware supplier. An open inference framework that is already tested on Ascend doesn't solve the availability problem for this hardware here (Ascend still has no significant distribution in the Brazilian market), but it sets a technical precedent: the code that turns a trained model into a production service is no longer tied to a single chip architecture.

For teams that already run DeepSeek's own models, such as DeepSeek-V4.1-Flash-1017, or that plan to adopt the family in production, the most direct gain is being able to swap the serving layer without depending on third-party forks built on top of projects like vLLM, SGLang, or TensorRT-LLM, today the most widely used options in this space. Having the official code from the team that trained the model reduces the risk of divergence between the behavior DeepSeek publishes as a reference and what actually runs on the production server.

What's still unclear

TechNode doesn't detail the open-source license DeepSeek chose, nor does it provide performance comparisons between DeepInfer and competing frameworks. The publication also doesn't include figures for throughput gains or computational cost reduction: the company talks about improving these metrics, with no benchmark published so far. It remains to be seen whether the inference community will adopt the project as a viable option alongside vLLM and SGLang, or whether it will stay limited to those specifically running DeepSeek's own models.