Some links on this page are affiliate links: if you buy through them we may earn a commission, at no extra cost to you.
MLPerf can help you shortlist AI infrastructure, but it cannot pick a data center for you. Its standardized results show what a complete system demonstrated on specified training, inference, storage, or power tests. The buying decision still depends on whether those tests resemble your workload—and on cost, availability, software, and facility constraints.
The practical approach is to use MLPerf to narrow the field, normalize the contenders against your service and budget targets, then validate finalists with a production-like pilot.
What MLPerf tells a data-center buyer
MLPerf is a suite of benchmarks published by MLCommons. It provides common workloads and rules for comparing submitted systems. Results are useful evidence, not a universal ranking: each number belongs to a particular benchmark version, workload, scenario, configuration, and metric.
Free tools Windows power users keep installed
One-click scans. No signup required.
That distinction matters because a data center is more than its accelerators. CPUs, memory, interconnects, storage, software, scaling behavior, power delivery, and cooling can all affect the work completed. A chip’s theoretical peak performance does not tell you how quickly a cluster will finish a model run or meet an inference service-level objective.
#1 Best Overall
- NVIDIA Ampere Streaming Multiprocessors: The all-new Ampere SM brings 2X the FP32 throughput and improved power efficiency.
- 2nd Generation RT Cores: Experience 2X the throughput of 1st gen RT Cores, plus concurrent RT and shading for a whole new level of ray-tracing performance.
- 3rd Generation Tensor Cores: Get up to 2X the throughput with structural sparsity and advanced AI algorithms such as DLSS. These cores deliver a massive boost in game performance and all-new AI capabilities.
- Axial-tech fan design features a smaller fan hub that facilitates longer blades and a barrier ring that increases downward air pressure.
- OC Mode : 1500 MHz (Boost Clock)/Default Mode : 1470 MHz (Boost Clock)
For current context, MLPerf Training v6.0, released June 16, 2026, added DeepSeek V3 and GPT-OSS 20B tests focused on sparse Mixture-of-Experts workloads. The round included 95 unique systems, 13 accelerator types, 19 host processors, and a majority of multi-node submissions; cloud-system participation more than doubled from v5.1. MLPerf Inference v6.0, released April 1, 2026, added or updated data-center tests including GPT-OSS 120B and advanced-reasoning coverage for DeepSeek-R1. These updates broaden the evidence available for current AI infrastructure, but they do not make every result representative of every production model. Training v6.0 results · Inference v6.0 results
Choose the benchmark that matches the decision
| Benchmark | What it measures | Useful for | What it does not settle |
|---|---|---|---|
| Training | Time to train a specified model to a defined quality target | Comparing end-to-end training systems and their scaling behavior | Your model’s exact training time, facility fit, or total cost |
| Inference | Serving performance under defined workload, latency, and quality conditions | Comparing systems for batch or interactive serving | Production tail latency, cost per request, or fit for your traffic mix |
| Storage | Whether a storage and data path can deliver training data at a rate associated with high accelerator utilization | Finding possible input-pipeline and storage bottlenecks | Your full data preparation, governance, security, or recovery workflow |
| Power | Energy or power for specified benchmark runs under documented measurement procedures | Comparing energy efficiency when submissions are comparable | Total facility energy, cooling needs, or utility capacity |
| Endpoints | An emerging end-to-end service comparison | Exploring deployed inference across cloud, neocloud, and managed services | A mature replacement for all service-level and procurement analysis |
MLPerf Endpoints v0.7, released July 28, 2026, is a foundation for broader comparisons of deployed AI services. Treat it as an emerging complement rather than a complete answer to cloud or managed-service selection. MLPerf Endpoints v0.7 release
Training: time to target quality, not theoretical peak
Training results measure how quickly a system reaches a specified quality target. The result reflects the full stack: accelerators, hosts, software, data movement, and communication. A system with more theoretical compute may still take longer if it has memory limitations, slow input delivery, synchronization costs, weak multi-node scaling, or substantial checkpointing overhead.
For large runs, examine multi-node results and scaling rather than relying on a single-accelerator score. Adding nodes does not guarantee a proportional reduction in elapsed time. Training v6.0’s majority of multi-node submissions underscores the relevance of cluster behavior, but a benchmark result still needs to be checked against the model size, interconnect, and node count you plan to deploy. MLPerf Training benchmark and results
Inference: match the scenario to the service
Inference measures serving under defined scenarios, latency requirements, and quality constraints. An Offline test emphasizes throughput when work can be batched. A Server test models requests arriving dynamically and requires the system to satisfy latency limits. User-facing conversational and reasoning workloads have different traffic patterns and metrics again. Multi-node serving may matter for models that do not fit or perform adequately on one node.
A high Offline throughput score is not evidence that a system will deliver a good interactive experience. Before comparing results, identify whether your business constraint is queries per second, samples per second, generated tokens per second, first-token latency, inter-token latency, or a tail-latency target such as p95 or p99. Do not convert one metric into another without supporting data. A benchmark’s quality requirement matters too: higher throughput achieved at a different quality target is not an equivalent result. MLPerf Inference rules
Storage: test whether the data path keeps up
Storage tests address a bottleneck that accelerator-only comparisons miss: whether the system can supply training data fast enough to keep accelerators busy. MLPerf Storage reports throughput, including samples per second and MB/s, and identifies details such as storage software and protocol, hardware, networking, capacity, compute-node count, and simulated accelerator type. The benchmark is designed around maintaining at least 90% accelerator utilization.
Outdated Drivers Are Slowing You Down
One free scan finds every outdated or missing driver and matches the right update for your exact hardware.Free scan · exact hardware matchPC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11A storage result can help answer whether improving the data path might be more effective than buying additional accelerators. But check dataset size and composition, caching assumptions, and whether the test includes checkpointing. MLPerf Storage uses synthetic populations intended to reflect real datasets’ file-size distributions and scales datasets to limit cache effects. That supports repeatability; it does not reproduce every organization’s preprocessing, metadata, access controls, or augmentation pipeline. Storage v2.0 added checkpointing tests aimed at recovery and forward progress in large training systems. MLPerf Storage scope and results · Storage v2.0 checkpointing tests
Power: useful evidence, not a facility energy bill
When comparable power submissions exist, they can help show performance per watt or energy for a benchmark run. Compare only results with equivalent workloads, quality targets, precision, system scale, and measurement boundaries. Not every performance submission has a directly comparable power result, and accelerator thermal design power is not total data-center energy consumption.
Rank #2
- Powered by the NVIDIA Blackwell architecture and DLSS 4
- Powered by GeForce RTX 5070
- Integrated with 12GB GDDR7 192bit memory interface
- PCIe 5.0
- NVIDIA SFF ready
For facility decisions, also account for server power, network and storage equipment, power-conversion losses, cooling overhead, peak demand, rack density, and the site’s available electrical and cooling capacity. MLPerf power methods are documented separately; inspect the method and scope before using a result in an energy model. MLPerf power measurement documentation
Read a result as a system record
Do not copy a headline score into a comparison spreadsheet without its context. Record at least:
- Benchmark suite, version, workload or model, and scenario
- Division and result metric, including any quality condition
- Accelerator model and count, host processors, memory, interconnect, and node count
- Software stack, framework, precision, and relevant optimization methods
- Power-measurement status and method, if applicable
- Submitter, system vendor, submission date, and availability status
Start with the closed division for a cleaner cross-vendor comparison: its workload and quality conditions are more constrained. Open or exploratory submissions can show useful implementation innovation, but may involve more extensive changes and are not a simple apples-to-apples ranking. MLPerf permits reimplementation of reference workloads to encourage software and hardware innovation, so inspect how a result was achieved as well as its score. The benchmark rules are the authoritative source for interpreting a run; use the results dashboard and supporting submission details to check the configuration. MLPerf Inference documentation and results
Also separate a submitted system from a purchasable product. Verify that the exact configuration—not merely a related product family—is available in your region and timeframe. Ask about general availability, quantity, lead time, chassis and rack requirements, software support, warranty, service terms, and cloud capacity or quota. A leaderboard result is not a supply commitment.
Translate results into workload economics
First define the outcome that matters, then normalize the result against it. A useful scorecard may include time per target-quality training run, throughput per node, inference latency and throughput at target concurrency, storage-fed accelerator utilization, and performance per watt. Add per-rack throughput if rack capacity is a constraint. Do not assume that throughput scales linearly with accelerator count.
Training cost
A starting estimate is:
Cost per training run = hourly infrastructure cost × elapsed training hours + storage, network, and support costs
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Use an elapsed time that reflects your target-quality run, including relevant data, networking, and checkpointing costs. For owned infrastructure, hourly cost should reflect depreciation or financing, facilities, power, cooling, staffing, and expected utilization—not just the purchase price divided by a theoretical number of hours.
Inference cost
For a request-based service:
Cost per million requests = (hourly total cost ÷ requests per hour) × 1,000,000
For generative AI, use tokens instead of requests if token volume is the relevant operating constraint. Include the same cost categories, plus idle capacity, software licenses, data transfer, egress, and any reservation or commitment discounts that actually apply.
Rank #3
- AI Performance: 767 AI TOPS
- OC mode: 2632 MHz (OC mode)/ 2602 MHz (Default mode)
- Powered by the NVIDIA Blackwell architecture and DLSS 4
- Axial-tech fan design features a smaller fan hub that facilitates longer blades and a barrier ring that increases downward air pressure
- A 2.5-slot design maximizes compatibility and cooling efficiency for superior performance in small chassis
For example, if two illustrative systems each cost $100 per hour, but one sustains 20,000 requests per hour and the other 25,000 under the same quality and latency requirements, their simple infrastructure costs are $5.00 and $4.00 per 1,000 requests, respectively. This arithmetic is illustrative, not an observed MLPerf result; real comparisons require comparable service metrics and a complete cost boundary.
Recommended Free Tools
Cloud prices are time- and region-sensitive. AWS’s Capacity Blocks page displayed $34.608 per hour for an eight-H100 p5.48xlarge and $82.368 per hour for an eight-B200 p6-b200.48xlarge when crawled in July 2026. Those are specific reservation price signals, not universal rates or a cost comparison with on-premises systems. Google Cloud lists GPU and machine-type pricing by region and pricing model; use its current pricing information for the configuration under consideration. AWS Capacity Blocks pricing · Google Cloud GPU pricing
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.What the benchmark means for deployment choices
Training clusters
For training, use MLPerf to shortlist systems that resemble your model and target quality, then inspect multi-node scaling, interconnect, storage throughput, and checkpoint behavior. A faster run on a much larger cluster may still be a worse fit if it needs more rack space, higher power, scarce networking, or an unavailable configuration.
Rented capacity can suit bursty projects or uncertain demand; owned infrastructure may make sense when utilization is high and sustained, or when data control and predictable capacity matter. Neither choice follows from leaderboard position alone. Compare the cost of equivalent completed work, including deployment lead time and idle periods.
Inference fleets
For serving, map each benchmark scenario to the service you intend to run. Batch document processing can value Offline throughput. Interactive APIs and chat experiences need evidence about latency under dynamic load, concurrency, and the relevant generative workload. Large models may require multi-GPU or multi-node serving. A realistic pilot should report first-token and inter-token latency where relevant, as well as p50/p95/p99 latency, throughput, quality, and cost at production-like traffic.
Data, power, and software are part of the system
Storage and networking can determine whether expensive accelerators stay busy. A storage upgrade may improve effective cluster utilization more economically than adding accelerators, but only if its design supports the real pipeline and its operational requirements. Include transformation, encryption, access controls, metadata operations, backup, replication, multi-tenancy, and recovery in the pilot.
Likewise, assess framework and compiler maturity, kernel and quantization support, distributed-training and serving libraries, observability, debugging, model portability, staff expertise, and vendor support. MLPerf captures the performance of a combined stack under benchmark conditions; it does not price the long-term engineering burden of adopting an unfamiliar one.
When a leaderboard comparison misleads
- Different models: Performance on one model does not establish superiority across model sizes or architectures. Memory capacity, bandwidth, sparsity, communication, and optimization can change the ranking.
- Different scales: An eight-accelerator server and a rack-scale system answer different questions. Normalize per accelerator, node, rack, dollar, watt, or completed task only when the underlying configuration and metric make that comparison meaningful.
- Different scenarios: Offline throughput cannot stand in for interactive server latency. Check the scenario, concurrency, and quality conditions.
- Missing submissions: A vendor’s absence may reflect timing, availability, engineering priorities, benchmark scope, or commercial strategy—not poor performance. A single submission is not proof of market-wide leadership.
- Specialized optimizations: Benchmark optimizations can be valuable, but ask whether they are upstreamed, open source or licensed, supported, portable, and available in your intended environment.
- Unverified availability: A record result is immaterial if the exact system cannot be obtained, supported, or deployed in the required location and timeframe.
- Incomplete power data: Different measurement conditions or boundaries can make apparent efficiency comparisons misleading. Treat results as evidence, not a complete facility-energy model.
The strongest check against benchmark overfitting is to run a proof of concept with representative models, real preprocessing and input distributions, target concurrency, production-size checkpoints, security controls, monitoring, and failure-recovery procedures.
Quick Recap
A practical buyer workflow
- Define the workload and service target. Document model and version, dataset, training quality target, inference quality target, input and output lengths, concurrency, latency and availability objectives, growth forecast, and security or data-residency requirements.
- Select the relevant suite. Use Training for time-to-quality, Inference for serving behavior, Storage for data delivery and checkpoint bottlenecks, and Power for comparable energy evidence. Treat Endpoints as an emerging service-level lens.
- Filter and shortlist. Match benchmark version, workload, scenario, division, accelerator and node count, system availability, power data, and deployment type. The Training results page links to the v6.0 results material.
- Inspect full configurations. Check host CPU, memory, network, storage, software versions, precision, framework, interconnect, power method, and availability—not only the accelerator name.
- Normalize economics and capacity. Estimate cost per completed training run or unit of inference, then include utilization, rack throughput, power, cooling, network, storage, support, staffing, and idle capacity.
- Run a representative pilot. Measure end-to-end training time, data-loader wait, accelerator utilization, communication overhead, checkpoint and recovery time, and inference latency and throughput at realistic concurrency. Record actual cost and operational effort.
- Choose the deployment model per workload. Compare on-premises, public cloud, neocloud, colocation, and managed inference against utilization, availability, control, support, data movement, and facility limits. A hybrid answer may be appropriate.
Questions to ask vendors
- Is this the exact configuration submitted to MLPerf, and can you provide its full result record?
- When will that configuration be available in our region and at our required scale?
- Which benchmark optimizations are included in the supported production software stack?
- What throughput and latency can you demonstrate on our model, quality target, and concurrency?
- What are the storage, network, checkpoint, and recovery characteristics of the proposed system?
- What does the quoted cost include, and which charges vary with region, utilization, data transfer, or reservation?
- What power and cooling capacity, rack density, and facility changes does deployment require?
- What support, warranty, replacement, and software lifecycle terms apply?
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
