Breaking

NVIDIA Says Software Tuning Made DeepSeek V4 Up to 5x Faster on Blackwell

June 30, 2026 at 11:31 EDT

Infra & Chips
Foundation Models
Open Source

On June 30, 2026, NVIDIA said software optimizations on its Blackwell platform had improved inference performance of the open-source DeepSeek V4 model by up to 5x in a single month, cutting cost per token to roughly one-fifth of previous levels. The announcement emphasized that the economics of already-deployed infrastructure can keep improving through software-stack gains alone, without swapping hardware.

NVIDIA Blackwell · Inference Optimization

One Month of Software Tuning Cut DeepSeek V4 Token Cost to One-Fifth

NVIDIA says software alone — no hardware change — boosted DeepSeek V4 inference up to 5× in a single month, and up to 20× across the full stack on the same GPU.

5×

Faster after one month — token cost down to ~1/5

20×

Full-stack throughput on the same GPU

30×

Power efficiency vs H200 at same interactivity

Token long-context window

Partner Result: SGLang on GB300

5× throughput at the same interactivity via disaggregated serving — measured in tokens per second.

~2,200 t/s

Before

→

~11,200 t/s

After

What Makes V4 Lean

−73%

Per-token FLOPs via hybrid attention (CSA + DSA + HCA)

−90%

KV cache memory vs the prior generation

99.8%

FP8 accuracy retained with 4-bit NVFP4 quantization

Praise

Rapid day-0 optimization and large throughput gains; broad adoption — DeepInfra, Baseten, Cognition, Together AI and Cursor among the listed partners.

Caveats

Some saw slow early responses on NVIDIA endpoints, plus an early TensorRT-LLM bug on hidden-size handling — later fixed via a community PR.

The pitch: with inference cost the key operational bottleneck, NVIDIA argues efficiency keeps improving after launch through software alone — co-designing runtimes, kernels, networking and hardware so new models run optimized from day zero.

Continue reading

The rest of this article is for AI News Blitz readers. Choose an option below to keep reading.