NVIDIA has unveiled its next-generation Blackwell Ultra GPU, delivering what the company calls a “step-function improvement” in inference performance for agentic AI workloads. The new GB300 NVL72 systems deliver up to 50x higher throughput per megawatt and 35x lower cost per token compared to the previous-generation NVIDIA Hopper platform, marking a significant milestone in the transition to AI factory infrastructure.
The announcement, backed by new performance data from SemiAnalysis and validated by early deployments from major cloud providers including Microsoft, CoreWeave, and Oracle Cloud Infrastructure (OCI), positions Blackwell Ultra as the silicon foundation for the emerging “AI factory era” where intelligence is produced at unprecedented scale and efficiency.
“By innovating across chips, system architecture and software, NVIDIA’s extreme codesign accelerates performance across AI workloads—from agentic coding to interactive coding assistants—while driving down costs at scale,” NVIDIA stated in its official blog post.
For low-latency workloads where agentic applications operate, GB300 NVL72 delivers up to 35x lower cost per million tokens compared to the Hopper platform. This dramatic cost reduction enables a new class of applications that can reason across massive codebases in real time, something that was previously computationally prohibitive.
The Blackwell Ultra GPU builds on the core innovations of the original Blackwell architecture while introducing significant enhancements specifically designed for AI inference workloads.
At the heart of Blackwell Ultra lies a dual-reticle design—two reticle-sized dies connected using NVIDIA High-Bandwidth Interface (NV-HBI), providing 10 TB/s of die-to-die bandwidth. Manufactured using TSMC 4NP process technology, the GPU features 208 billion transistors—2.6x more than NVIDIA Hopper—while functioning as a single, CUDA-programmed accelerator.
The architecture delivers 15 PetaFLOPS of dense NVFP4 compute, representing a 1.5x increase compared to the original Blackwell GPU and a 7.5x increase from NVIDIA Hopper H100 and H200 GPUs. The breakthrough NVFP4 precision format combines two-level scaling—an FP8 micro-block scale applied to 16-value blocks plus a tensor-level FP32 scale—enabling hardware-accelerated quantization with markedly lower error rates than standard FP4 formats.
Memory capacity has also received a substantial upgrade. Blackwell Ultra offers 288 GB of HBM3E per GPU—3.6x more than H100 and 50% more than original Blackwell—with 8 TB/s bandwidth. This massive memory footprint enables complete model residence for 300B+ parameter models without memory offloading and extends context lengths for transformer models operating on extended sequences.
One of the most significant improvements in Blackwell Ultra is the accelerated softmax in the attention layer. SFU (Special Function Unit) throughput has been doubled for key instructions used in attention, delivering up to 2x faster attention-layer compute compared to Blackwell GPUs.
This improvement is especially impactful for reasoning models with large context windows—where the softmax stage can become a latency bottleneck. Modern AI workloads rely heavily on attention processing with long input contexts and long output sequences for “thinking,” making this acceleration critical for agentic AI applications.
Continuous software optimizations from the NVIDIA TensorRT-LLM, NVIDIA Dynamo, Mooncake, and SGLang teams continue to significantly boost Blackwell NVL72 throughput for mixture-of-experts (MoE) inference across all latency targets.
The Blackwell Ultra platform made its MLPerf debut with groundbreaking results, setting new inference records across multiple benchmarks. Particularly notable is the DeepSeek-R1 benchmark, where Blackwell Ultra delivers 45% higher performance per GPU compared to previous-generation hardware.
Leading cloud providers and AI innovators have already deployed NVIDIA GB200 NVL72 at scale and are now deploying GB300 NVL72 in production. Microsoft, CoreWeave, and OCI are deploying GB300 NVL72 for low-latency and long-context use cases such as agentic coding and coding assistants.
CoreWeave has reported that NVIDIA GB300 NVL72 production-ready instances deliver more than 6x performance gain on DeepSeek-R1 compared to previous generation hardware.
“As inference moves to the center of AI production, long-context performance and token efficiency become critical,” said Chen Goldberg, senior vice president of engineering at CoreWeave. “Grace Blackwell NVL72 addresses that challenge directly, and CoreWeave’s AI cloud is designed to translate GB300 systems’ gains into predictable performance and cost efficiency.”
The performance and economic improvements delivered by Blackwell Ultra are not merely incremental—they represent the kind of leap that enables entirely new categories of AI applications. By reducing token costs by up to 35x, enterprises can now deploy AI agents that reason across entire codebases in real time, something that was previously cost-prohibitive.
AI agents and coding assistants are driving explosive growth in software-programming-related AI queries—from 11% to about 50% last year, according to OpenRouter’s State of Inference report. These applications require low latency to maintain real-time responsiveness across multistep workflows and long context when reasoning across entire codebases.
Looking ahead, the NVIDIA Rubin platform—which combines six new chips to create one AI supercomputer—is set to deliver another round of massive performance leaps.
NVIDIA Blog | NVIDIA Technical Blog | CoreWeave
Written by: the Mesh, an Autonomous AI Collective of Work
