Verified, then logged.
Sun 22 Nov 2026 · 00:24 KST / 15:24 UTC한국어 | ENSearch /

Breaking08 / Industry

Rubin adds about 3% HBM per GPU, up to 30x output per dollar

A close-up of an accelerator module in an open data center server tray, with rows of stacked high-bandwidth memory chips flanking a large processor die. AI-generated image

NVIDIA said on 6 Nov 2026 that its Vera Rubin NVL72 system delivers up to 30x higher agentic throughput per dollar than the previous generation. The company presented full-stack co-design as its answer to tight memory supply.

High-bandwidth memory (HBM) capacity per GPU on the system, which links up to 72 GPUs, rose about 3% over the previous generation, NVIDIA said. Its blog gives the 30x figure for the DeepSeek V4 Pro workload on SemiAnalysis AgentX at 125 tokens per second per user, against the Blackwell Ultra-generation GB300 NVL72. At 100 tokens per second per user, the GB300 NVL72 delivers up to 10x higher throughput per dollar than the H200 platform, with HBM per GPU roughly doubling.

Memory wall. NVIDIA described the memory wall as the point at which bandwidth can no longer keep pace with compute, however much HBM is added. Beyond that point, extra capacity goes unused while still adding cost and power draw. The metric it put forward for operators is the number of useful tokens delivered within a power budget and response-time target.

HBM savings. According to NVIDIA, the 4-bit precision format NVFP4 makes model weights and the KV cache, the working memory that holds each request's context, smaller than FP8 does. On NVL72, weights are spread across up to 72 GPUs, which leaves more HBM free on each GPU. The inference software Dynamo moves inactive context to the CMX storage tier and returns it before reuse.

NVIDIA said it was the first to bring HBM3e and HBM4 into an accelerated computing stack. Its blog lists Cloudian, DDN, Dell Technologies, Everpure, HPE, IBM, NetApp, VAST and WEKA as CMX partners.

NVIDIA said tight memory supply and rising prices are expected to continue in the near term. The blog does not disclose the system prices used to calculate the up-to-30x figure, or HBM yields.