Estimate a Vera Rubin NVL72 deployment from two inputs: a system-specific quote and sustained workload measurements on the serving stack you plan to run. NVIDIA’s published peak throughput and vendor comparisons are useful reference points, but they do not tell you how many useful tokens per second your model will deliver or what each token will cost. A defensible estimate ties measured output under your latency and quality requirements to the full cost of acquiring, powering, cooling, and operating the chosen deployment.
Start by defining what you are estimating
Decide whether the boundary is one NVL72 rack or a larger platform. A single rack is not the same cost or capacity unit as a deployment that also includes LPX systems, storage or context-memory systems, scale-out networking, and adjacent racks. Record what is inside the boundary before comparing quotes or performance figures.
As an Amazon Associate I earn from qualifying purchases.
Then define the workload and service target. For an estimate to be useful, it should describe the model and version, precision, prompt and output lengths, context and KV-cache assumptions, concurrency, serving framework and version, topology, target latency and output quality, and expected utilization. Treat prefill and decode separately for long-context or interactive workloads: their resource demands and bottlenecks can differ.
What NVIDIA publishes for an NVL72
Rack configuration and peak arithmetic figures
NVIDIA describes Vera Rubin NVL72 as an integrated rack-scale system with 72 Rubin GPUs, 36 Vera CPUs, ConnectX-9 SuperNICs, BlueField-4 DPUs, and NVLink 6. Its product information names Quantum-X800 InfiniBand and Spectrum-X Ethernet for scale-out networking. NVIDIA’s technical description specifies 18 compute trays, 9 NVLink switch trays, a sixth-generation copper NVLink spine, 3.6 TB/s bandwidth per GPU, and 260 TB/s scale-up bandwidth per rack. Those are vendor-described system characteristics, not independent measurements.
#1 Best Overall
- NVIDIA Volta GV100 Architecture — 4,608 CUDA Cores, 640 1st-Gen Tensor Cores delivering 14 TFLOPS FP32 and 112 TFLOPS deep learning performance for AI training, inference, HPC, and scientific computing workloads
- 32GB HBM2 ECC Memory — 900 GB/s Bandwidth — High-bandwidth memory on a 4096-bit bus with ECC error correction provides the memory capacity and throughput required for the largest AI models, simulations, and datasets
- PCIe 3.0 x16 Interface — 250W TDP — Standard PCIe Gen3 connectivity with passive cooling designed for enterprise rack server deployment in HPE ProLiant, Dell PowerEdge, and Supermicro platforms with adequate chassis airflow
- NVLink — Scale to 96GB Unified Memory — Connect two V100 GPUs via NVLink at 300 GB/s bi-directional bandwidth to scale GPU memory from 32GB to 96GB for larger AI training and HPC workloads
- Multi-Precision Computing — Supports FP64 (7 TFLOPS), FP32 (14 TFLOPS), FP16 (112 TFLOPS) and INT8 precision modes for flexible deployment across training, inference, and scientific simulation workloads
| Published figure | Value | How to interpret it |
|---|---|---|
| NVFP4 inference | 3,600 PFLOPS per NVL72 rack | Peak, format-specific arithmetic throughput; not an application token-rate forecast. |
| NVFP4 training | 2,520 PFLOPS per NVL72 rack | Training specification, not an inference capacity estimate. |
| FP8/FP6 training | 1,260 PFLOPS per NVL72 rack | Training specification. |
| FP16/BF16 | 288 PFLOPS per NVL72 rack | Published rack figure; workload results depend on implementation and conditions. |
| TF32 | 144 PFLOPS per NVL72 rack | Published rack figure; not equivalent to measured serving throughput. |
NVIDIA also describes a 100 MW AI-factory configuration with 40,000 Rubin GPUs and MaxLPS, listing 2 ZFLOPS of NVFP4 inference and 12 PB of HBM4. That factory-scale configuration is a different boundary from a single rack.
Why peak PFLOPS does not answer “tokens per second”
Tokens per second depends on the model, numerical precision, input and output lengths, batch and concurrency, memory and cache behavior, software, topology, and whether the measure is aggregate throughput or latency for an individual request. The service-level target matters too: a system driven to maximize offline throughput may not meet an interactive latency target. Use published arithmetic figures to understand the scale and architecture, then benchmark the actual workload.
Use vendor comparisons as scoped evidence, not multipliers
NVIDIA’s GB200 comparison
NVIDIA says NVL72 can deliver one-tenth the cost per million tokens and up to 10 times more tokens per megawatt than GB200 NVL72 in a stated Kimi-K2-Thinking scenario with 32K input and 8K output tokens. These are NVIDIA-reported comparisons for that model and context setup; the product information says inference performance is subject to change. They are not universal ratios for other models, prompt mixes, utilization levels, or customer installations. NVIDIA’s separate claim that a projected MoE training scenario needs one-fourth as many GPUs concerns training a 10T-parameter model on 100T tokens within a fixed month, not inference economics.
The Tool Desk
Outbyte Driver Updater FREEScan for outdated or missing drivers - takes under a minuteDriver Scan →Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →MLPerf Inference v6.1 preview results
In a September 16, 2026 post, NVIDIA reported preview MLPerf Inference v6.1 results of up to 3.7 times GB300 throughput for Qwen3-VL across offline, server, and interactive scenarios using vLLM and NVIDIA Dynamo, and up to 2.5 times for DeepSeek-R1 using TensorRT-LLM. NVIDIA identified entries 6.1-0106 and 6.1-0074 and said optimization continued after submission. These are vendor-reported preview submission results for specified models and software stacks—not an application-wide multiplier or a substitute for testing your configuration.
NVIDIA’s FY2026 Sustainability Report also presents modeled performance-per-megawatt scenarios using internal DLSim analytical projections for Vera Rubin NVL72 (NVFP4), Groq 3 LPX (FP8), and GB200 NVL72 (NVFP4), including GPT-MoE 2T / 400K and a separate Kimi-K2-Thinking 32K/8K scenario. The report cautions that projections may differ from measured silicon and other deployments. Do not treat modeled efficiency as a measured customer result.
How much does an NVIDIA Vera Rubin NVL72 rack cost?
There is no universal public selling price established by the sources available as of October 4, 2026. Two dated secondary reports illustrate the uncertainty rather than establish a price:
| Reported figure | Attribution and date | What it establishes |
|---|---|---|
| About $7.8 million | Tom’s Hardware, May 22, 2026, relaying a Morgan Stanley Research estimate | An attributed estimate, not an NVIDIA price or a delivered buyer quote. |
| Up to $8.8 million | Tom’s Hardware, March 24, 2026, based on secondary reporting | A separately reported upper figure; configuration and included scope are not established as comparable to the other estimate. |
For a buyer estimate, request a dated, configuration-specific delivered quote. It should identify rack scope, delivery, support, software, networking, storage, installation, and any required adjacent systems or facility work. A headline rack estimate that omits these boundaries cannot be used as a complete deployment cost.
Crashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minuteWindows Errors? Fix Them Before They Spread
Repair common Windows errors and clear accumulated junk for a smoother, more stable PC - no reinstall needed.Free scan · no reinstallHow much power does a Vera Rubin NVL72 rack use?
NVIDIA’s product information reviewed for this estimate does not publish a universal rack input-power figure. A 2026 Pegatron datasheet for its RA4803-72N3 Vera Rubin NVL72 implementation lists six 18.3 kW power supplies, “Max Q = 188kW,” and “Max. TDP Support Max P = 228kW,” with liquid cooling and 415V/480V input. Pegatron says its specifications are subject to change. These differently labeled figures describe one manufacturer’s system; they should not be collapsed into a single sustained operating draw or generalized to every NVL72 installation.
Ask the selected system builder or integrator to confirm sustained and maximum rack input power, liquid-cooling capacity, electrical distribution, networking loads, and redundancy requirements for the quoted configuration. Do not substitute GPU TDP for facility input power or assume an existing rack’s power and cooling envelope is adequate. Add facility overhead and cooling energy using the site’s actual design and operating assumptions.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Build an auditable cost-per-token estimate
Use a matched service target and measured useful output—not peak FLOPS or a vendor headline ratio. One practical annualized structure is:
Annual total cost = annualized hardware and facility cost + annual energy and cooling cost + annual operations and support cost.
Cost per million useful output tokens = annual total cost ÷ (annual useful output tokens ÷ 1,000,000).
Rank #3
- Professional GPU with Blackwell Architecture in Compact Small Form Factor (SFF)
- Blackwell Architecture
- 24GB GDDR7 with PCIe 5.0 & Ray Tracing
- AI Workstation
Useful output means tokens delivered while meeting the defined quality, latency, and availability target. If a low-cost scenario depends on queueing beyond the target or lower output quality, it is not comparable to a scenario that meets the target.
Inputs to collect
- Quote and scope: dated delivered price, included system components, deployment services, software, support, storage, networking, and facility fit-out or allocated facility cost.
- Measured serving performance: model/version, precision, input/output distribution, context and cache assumptions, serving framework/version, topology, concurrency, sustained useful throughput, latency distribution, and achieved utilization.
- Facility and operating costs: integrator-confirmed rack input power and cooling needs, local electricity tariff, operating hours, facility overhead, staffing, maintenance, networking, and financing or depreciation assumptions appropriate to your accounting view.
- Useful life: expected service period and replacement or residual-value treatment, stated consistently with the hardware and facility cost allocation.
Run low, base, and high cases
Vary the rack quote, electricity price, utilization, measured useful throughput, and service life. Keep the workload, quality, and service-level definition visible in each case. The result should make clear which assumption drives cost per million tokens and how much capacity remains when demand is below the rack’s practical operating range.
Is it cheaper to own an NVL72 or rent cloud GPUs?
The available public information does not establish a general winner or current generally available customer price for Vera Rubin cloud instances. NVIDIA lists cloud providers and system builders deploying Vera Rubin systems, including CoreWeave, Google Cloud, Microsoft Azure, Oracle Cloud Infrastructure, and Nebius, but listing a provider does not establish a customer’s region, delivery date, instance terms, or price.
Compare ownership and rental only with a current provider quote and a matched workload. Use the same model and quality, context lengths, precision, useful throughput, latency target, expected utilization, and service assumptions. For ownership, include delivered capital cost, facility power and cooling, operations, support, and the cost of unused capacity. For cloud, verify the precise hardware offering, region, availability, billing units, storage and networking charges, and any minimum or commitment terms. If those inputs are unavailable, report the break-even as undetermined rather than deriving it from NVIDIA’s benchmark multipliers.
Availability and deployment boundary
NVIDIA’s May 31, 2026 newsroom release said Vera Rubin was ramping into full production and production shipments were set to begin starting in fall 2026; its product and technical pages similarly described production ramp and shipment plans in the second half of 2026. Those statements do not confirm a specific buyer’s delivery slot or regional availability. Confirm schedule, configuration, and site readiness directly with the supplier and integrator before basing a capacity plan on a delivery date.
Request mix can also alter the required capacity. Jensen Huang said in NVIDIA’s May 31, 2026 release: “Agentic AI is a new kind of workload. One prompt can launch a thousand-step journey of reasoning, retrieval, tool use and response generation.” This is NVIDIA’s framing rather than benchmark evidence, but it highlights why a single average prompt length may fail to represent an agent workload: tool calls and multi-step reasoning can change the number and shape of model requests per user task.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
