DeepSeek v4.1-Flash separates prefill and decode, driving down cost per task

DeepSeek has once again published a cutting-edge architecture. According to the roundup by Latent Space, the new DeepSeek v4.1-Flash retired V4 Pro and swapped the classic MoE design for a causal encoder-decoder architecture with an explicit separation between the reading (prefill) and generation (decode) phases. The internal name sums it up: 763B-P8B-D16B, that is, 763 billion total parameters, 8B active on input and 16B active on output. That yields a sparsity of 1% to 2%.

Sebastian Raschka summed up the reaction of much of the research community: despite the modest version number, this "should have been called DeepSeek V5." The change is not a 0.1 bump, it's a reformulation of the backbone, with native vision built into the same release, without waiting for a separate model.

Why separating prefill and decode matters

In a traditional LLM, the same set of active parameters processes both the input prompt and each generated token. The problem is that these two phases have very different cost profiles: prefill reads thousands of tokens at once (compute-bound), while decode generates one token at a time and is bound by memory bandwidth, mainly because of the KV cache that grows with context.

DeepSeek attacked exactly this point. By using 8B active for prefill and 16B for decode, plus tricks like Sliding-Window Attention Bounded Replay, the technical report cited by Latent Space claims a KV cache up to 1/8 the size it was in V4 Flash. Analyst nrehiew highlighted a concrete number: about 890 bytes per token of KV in the evaluated regime, and called the model a case study in "how obsessing over KV cache compression gets you a hyper-efficient frontier model."

In practice, for those building long-running agents, this is the variable that hurts. Contexts of 1 million tokens and agent loops that reprocess history blow up memory consumption. Cutting the KV cache is what makes this type of workload financially viable outside a dedicated datacenter.

The numbers: cheap despite being verbose

Independent benchmarks tell a consistent story. Artificial Analysis put v4.1-Flash at 40 on the Intelligence Index, above V4 Pro 0813 and just below GLM-5.3-Flash. It's not the raw top of intelligence, and it's worth noting that honestly. What changes is the cost axis.

| Metric | v4.1-Flash | Comparison | |---|---|---| | Input price | US$ 0.30 / 1M tokens | input cache at US$0.006/1M | | Output price | US$ 1.20 / 1M tokens | +50% off-peak discount | | Cost per task (AA Index) | US$ 0.27 | ~7x lower than GLM-5.3 (US$ 2.01) and Kimi K3 (US$ 2.00) | | AutomationBench-AA | 69% | ties GPT-6 Astra, above Grok 4.6 (67%) | | AA-LCR v1.1 | 84% | ties GPT-5.6 Sol and Gemini 3.8 Flash | | Context | 1M tokens | text + image, MIT license |

The important caveat: v4.1-Flash is one of the most verbose models ever measured, averaging 89,000 tokens per task on the Intelligence Index, 25% above GLM-5.3. Even so, the total cost per task remains extremely low because the token is so cheap. Vals AI also ranked it #1 among open-weight models in their index, at US$0.30 per test, the cheapest in the open top 10.

There's a skeptical warning that deserves weight. TeortaxesTex, who closely follows DeepSeek, called some results "very strange," especially the first place in AutomationBench, and pointed to DeepSeek's historical pattern of showing high internal evals with weaker external robustness, because it "ships internal research artifacts, not products." He also reported that the multi-agent mode (DSH agent teams) can degrade quality if the project lacks clear modularity, with solo v4.1 outperforming team mode in at least one case. In other words: a benchmark number is a starting point, not a verdict for your use case.

Running locally became surprisingly accessible

The part that most excited the infrastructure crowd was the model's servability. Fraser Price reported v4.1-Flash at full precision at 200 TPS on 4 Max-Qs with just 64GB of system RAM, offloading a 200GB hash table/Engram to NVMe. He then improved that to 300+ TPS on 4 RTX Pro, still at full precision, with a peak of under 32GB of system RAM, using a custom vLLM fork with SSD support.

Antirez showed DwarfStar running the model on a 128GB M5 Max, saying that streaming via SSD made everything unexpectedly fast, and speculated that v4.1 might "reuse the same experts more often," which would help the disk access pattern. On the open ecosystem side, vLLM's recent release included DeepSeek-V4's shared experts merged into MegaMoE, and Mooncake Store now allows offloading decode KV.

The technical point behind this: with 1-2% sparsity and compressed KV, the fraction of weights that needs to be hot in VRAM at each step is small. This is what allows pushing the bulk of the structure to NVMe while still keeping usable throughput. Stochastic Chasm even inferred QAT on the KV cache, which would explain the better-than-peers performance under FP4 KV cache.

What changes for those building in Brazil

For Brazilian developers, the cost-benefit calculation of running AI moves out of the abstract. Two routes become more concrete:

• Via cheap API: at US$0.30/1M input and US$1.20/1M output, with cache at a fraction of a cent, you can prototype long-context agents without the dollar-denominated bill (already hit by the exchange rate) making the project unviable. High verbosity is the counterpoint to monitor, since it inflates output, but the price per token compensates in most scenarios.
• Via self-hosting with offload: reports of running on a 128GB M5 Max or on machines with 32-64GB of system RAM plus GPUs, with the rest on SSD, take the frontier model out of the exclusive territory of those with a cluster. It's not plug-and-play (Fraser Price needed a custom vLLM fork), but the path exists and is being documented in the open ecosystem.

When it's not worth it

It's not a silver bullet. If the application needs the absolute ceiling of intelligence, v4.1-Flash trails models like GLM-5.3-Flash on some indices, and DeepSeek itself positions it as the smallest of the new family. If the pipeline is sensitive to verbosity (short responses, first-token latency, output cost dominance), the average 89k tokens per task weighs heavily. And whoever depends on proven robustness in production should take seriously the criticism that DeepSeek's high evals don't always translate into reliability outside the lab, testing in their own domain before migrating.

The strategic message is bigger than the model itself. DeepSeek reinforces the thesis that open models today compete less on available weight and more on servability: fitting into offload pipelines, quantized KV, local deployment, and open inference servers. The lab itself, according to Latent Space, signaled that the ROI of improving data quality today outweighs that of inventing new post-training algorithms, a debate that remains open and worth following in the original report and reactions.