DeepSeek open-sources DeepInfer, the framework it uses to cut AI inference costs

DeepSeek, the Chinese artificial intelligence startup, open-sourced on Tuesday (October 20, 2026, Beijing time) the code for DeepInfer, an inference framework developed in-house to run its models more cheaply. The news was reported by SCMP Tech. Inference is the stage of serving and running an already trained model, as opposed to training itself, and it's where most of the operational cost falls for any company putting AI into production.

What DeepInfer Does in Practice

The framework was built specifically to run DeepSeek-V4.1-Flash, a model launched in September 2026 with an architecture that reduces the amount of memory needed to operate, surpassing the performance of the company's earlier V1 Pro. An update to V4.1-Flash released the previous Saturday (October 17) made the model one of the most powerful and cost-efficient open-weight models on the market, according to independent benchmarking firm Artificial Analysis.

Since V4.1-Flash is an open-weight model, any cloud provider can serve it on its own infrastructure. Open-sourcing DeepInfer alongside it, DeepSeek says, helps the community "reduce computational cost, lower energy consumption, and make AI more accessible."

The Numbers DeepSeek Disclosed

In a technical article published on the Chinese Q&A platform Zhihu on Tuesday, DeepSeek said a single computing cluster running V4.1-Flash on DeepInfer can generate revenue equivalent to $38 per GPU per hour, based on off-peak pricing. The company did not disclose its operating costs or profit margin, so this figure comes from a revenue estimate, not direct savings verified by third parties.

The efficiency gain, according to the company, comes in part from retaining already-processed information for more than 72 hours, which avoids repeated recomputation of context the model has already seen. This type of context caching follows the same logic as techniques like prefix caching, already used in inference frameworks such as vLLM and SGLang, but applied here to the specific architecture of V4.1-Flash.

Nvidia and Huawei in the Same Play

DeepInfer was designed to support both Nvidia's Blackwell SM100 GPUs and Huawei Technologies' Ascend 950 chips. This dual compatibility matters because it reduces dependence on a single hardware vendor to run the model in production: whoever serves V4.1-Flash isn't locked into Nvidia's CUDA ecosystem if they have access to Huawei clusters, something more relevant today for Chinese providers than for infrastructure available in Brazil, but which signals where the AI chip market is heading.

A Verbal Declaration of War

The launch was accompanied by an unusually combative statement from Chen Deli, senior researcher at DeepSeek, in a social media post:

Open source vs closed source … A decisive battle between two futures, two destinies for humanity

Chen Deli, senior researcher at DeepSeek

He went on, in an even more direct tone:

However long they want to fight, we'll fight. Until total victory.

Chen Deli, senior researcher at DeepSeek

Cui Tianyi, who leads DeepSeek's Harness team, also commented on the launch in his own post, saying he hopes more companies and organizations around the world will "deploy DeepSeek models and profit from doing so" (in the original English: "deploy DeepSeek models and profit from doing so").

The Context of the China-US Dispute

Chen's statements and the launch of DeepInfer come at a time when Chinese AI companies, such as Moonshot AI and Z.ai, continue releasing updates to open-weight models, challenging American rivals like Anthropic and OpenAI, which publicly advocate for the closed-source model. Anthropic CEO Dario Amodei has repeatedly stated that open-source AI poses risks to humanity, in addition to accusing Chinese companies of "stealing" American AI technology, including a recent accusation that Chinese competitors secretly use Claude.

At the same time, Beijing proposed export controls last week for 23 AI-related technologies. A spokesperson for the Chinese government said, however, that the country will continue to encourage open-source AI and promote its adoption worldwide, with a particular focus on the Global South.

What Changes for Those Building with AI in Brazil

For those already running open-weight models in production, such as Llama, Qwen, or DeepSeek's own models, DeepInfer is one more piece in the inference framework ecosystem, alongside vLLM, SGLang, and TensorRT-LLM. The difference is that it was optimized specifically for the V4.1-Flash architecture, which tends to yield better performance on that specific model than a generic framework, but also means less flexibility for anyone who wants to serve other models with it.

The practical point is the 72-hour context cache: teams that make many repeated calls with the same prompt prefix (agents that reuse extensive system context, for example) are the ones who feel this type of optimization most in their GPU bill. It's worth looking at the project's repository to assess whether the integration pays off compared to what's already in production, but pricing for V4.1-Flash via API remains the simplest way to test the model before investing in your own infrastructure.

What Remains Open

DeepSeek has not published independent benchmarks comparing DeepInfer to competing frameworks, nor detailed the actual operating cost behind the cited $38 per GPU per hour. There are also, so far, no reports of major Western cloud providers adopting the framework in production. The move is more a sign of DeepSeek's aggressive open-source strategy than an immediate change in the day-to-day work of those already doing LLM inference outside China.