Do these 3 things before closing this tab:
1Fix the driver behind crashes, sound loss and screen glitches2Clear out junk files and repair common Windows errors3Scan for outdated or missing drivers - takes under a minuteSize a multi-GPU AI server in three separate domains: the GPU-to-GPU links inside a server, the scale-out fabric between servers, and the paths used by storage, users and management. Then size storage twice—first for capacity, and separately for read/write performance. GPU count alone cannot determine either answer.
For a single server, verify the GPU topology and supported scale-up links. For a cluster, size the inter-node fabric for the distributed workload, while keeping storage and customer traffic on their own designed paths. As an initial reference, NVIDIA documentation cites approximately 12.5 Gb/s of storage bandwidth per GPU and an example of about 200 Gb/s for 16 GPUs, while its certified-system guidance includes 200 Gbps minimum for multi-node inference and up to 400 Gbps per GPU in specified configurations. Those figures are starting points tied to particular architectures, not universal requirements.
As an Amazon Associate I earn from qualifying purchases.
Start with the three network domains
A multi-GPU design has more than one “network.” Treating every link as interchangeable leads to overbuying some components and starving others.
Scale-up: links inside one server
Scale-up is the GPU-to-GPU domain within a node. NVLink is NVIDIA’s proprietary scale-up technology; its topology, switch placement and supported collective operations determine how quickly GPUs exchange gradients, activations and parameters. PCIe topology also matters because GPUs, NICs, NVMe devices and CPUs share root complexes and lanes. A server with the same GPU count can behave very differently when those devices are attached to different CPU sockets or PCIe switches.
#1 Best Overall
- MEET THE NEXT GEN: Consider this a cheat code; Our Samsung 990 PRO Gen4 SSD helps you reach near max performance with lightning-fast speeds; Whether you’re a hardcore gamer or a tech guru, you’ll get power efficiency built for the final boss
- REACH THE NEXT LEVEL: Gen4 steps up with faster transfer speeds and high-performance bandwidth; With a more than 55% improvement in random performance compared to 980 PRO, it’s here for heavy computing and faster loading
- THE FASTEST SSD FROM THE WORLD'S FLASH MEMORY BRAND: The speed you need for any occasion; With read and write speeds up to 7450/6900 MB/s you’ll reach near max performance of PCIe 4.0 powering through for any use
- PLAY WITHOUT LIMITS: Give yourself some space with storage capacities from 1TB to 4TB; Sync all your saves and reign supreme in gaming, video editing, data analysis and more
- IT’S A POWER MOVE: Save the power for your performance; Get power efficiency all while experiencing up to 50% improved performance per watt over the 980 PRO; It makes every move more effective with less consumption
NVIDIA describes NVLink as part of the scale-up domain and the cluster interconnect as the scale-out GPU-to-GPU network. The external cluster fabric may use Ethernet or InfiniBand. See NVIDIA’s Data Center Architecture and the HGX AI Factory network architecture.
Scale-out: links between servers
Scale-out carries collective communication when a training or inference job spans nodes. All-reduce, all-gather and parameter exchanges can make latency and bisection bandwidth as important as the advertised port rate. The required fabric depends on node count, GPU topology, the parallelism strategy and the utilization target. A single-node job has no inter-node traffic, while a distributed training job may continuously exchange data during every step.
Storage, customer and management paths
Storage traffic is not the same as the GPU compute fabric. In NVIDIA HGX architectures, storage networking is a separate path, so a storage-bandwidth figure must not be interpreted as compute-fabric bandwidth. Customer access, provisioning, monitoring and management should also have deliberate connectivity and failure isolation. NVIDIA’s architecture documentation identifies Ethernet management and service paths separately from the configurable compute interconnect; the Data Center Architecture guide states that “The TAN and SMN are always Ethernet; the CIN can be configured as either Ethernet or InfiniBand, and NVLink is a proprietary NVIDIA standard.”
Free tools Windows power users keep installed
One-click scans. No signup required.
Collect the inputs before selecting hardware
Write down these values for the exact server and workload. If one is unknown, label it as an assumption and test it before placing an order.
- GPU model and count, server model, number of CPU sockets, PCIe root-complex layout and NVLink topology.
- Single-node or multi-node deployment, current node count and the largest planned job.
- Workload type: pretraining, fine-tuning, batch inference, online inference or data processing.
- Parallelism and communication pattern: data, tensor, pipeline or expert parallelism; collective frequency; and expected GPU utilization.
- Dataset size, file format, average and peak read rate, object/file count, and whether the active set fits in system RAM or local NVMe cache.
- Model-artifact size, checkpoint size, checkpoint interval, retention count and acceptable restart time.
- Number of concurrent jobs and whether they share the same storage namespace.
- Required availability, redundancy, maintenance windows and the network path from compute nodes to shared storage.
This inventory prevents a common mistake: buying links for the nominal GPU count while ignoring data-loader bursts, simultaneous checkpoints or several jobs competing for the same array.
Size the compute network
Single-node servers
For one server, external bandwidth may be needed only for storage, data ingest and user access. Start by documenting which GPUs share each PCIe switch and CPU socket, which NICs can reach those GPUs without a cross-socket hop, and which NVMe devices share lanes. Confirm that the server’s firmware and software stack support the intended GPU peer-to-peer and NIC-to-GPU paths.
Rank #2
- BREAKTHROUGH PCIe 5.0 PERFORMANCE: Supercharge your workflow and gaming with PCIe 5.0, boasting up to 14,700/13,400 MB/s¹ sequential read/write speeds. Tackle massive files and power up your gaming with Gen5—twice as fast as the 990 PRO SSD.
- EVERY TASK, TURBOCHARGED: Speed past productivity limits. With random read/write speeds up to 1,850K/2,600K IOPS¹, enjoy fast game loads, seamless AI apps, and efficient multitasking. Virtually no lag, no limits—just nonstop performance.
- THINK FAST, CREATE FASTER: With random read/write speeds of up to 1,850K/2,600K IOPS¹, the 9100 PRO SSD fuels seamless AI content creation, swift loads, and smooth gameplay. Work, play, and create at lightning speed.
- SPEED, WHENEVER YOU NEED: From laptops, desktop PCs, to PS5 with added heatsink devices², experience blazing PCIe 5.0 speeds and up to 8TB of storage. Perfect for video editing, gaming, and creative tasks, with the compatibility to match your device.
- STAY COOL, RUN FAST: Push limits, not temperatures. A 5nm controller boosts power efficiency up to 49% over the 990 PRO SSD³, while advanced thermal control keeps performance smooth and reliable.
Do not add one external NIC per GPU by rule. A single-node workload may have no scale-out traffic, while a multi-node workload may need several ports to keep all GPUs busy. The correct count follows from topology, traffic pattern and target utilization.
Multi-node training and inference
For a cluster, calculate both aggregate fabric capacity and the bandwidth available to each GPU. Include the number of nodes, GPUs per node, ports per node, switch radix, uplink capacity and any oversubscription. A design with high-rate server ports can still underperform if the spine, leaf or uplinks provide less bisection bandwidth than the workload requires.
NVIDIA’s Certified Systems Configuration Guide includes guidance of at least 200 Gbps NIC capability for multi-node inference and up to 400 Gbps per GPU in the configurations it describes. These are configuration recommendations, not a universal minimum for every multi-GPU server. Apply them only after checking the GPU model, node design, NIC placement, switch fabric, redundancy and workload.
| Design question | What to measure or calculate | Why it matters |
|---|---|---|
| Per-GPU allocation | Usable fabric bandwidth divided by active GPUs, after reserving capacity for protocol overhead and other traffic | Shows whether one GPU is starved even when the node’s aggregate port rate looks large |
| Aggregate bisection | Total bandwidth that can cross the fabric between equal halves of the cluster | Reveals oversubscription during all-reduce and all-to-all operations |
| Latency | Measured application-relevant message latency, not only link speed | Small frequent collectives can be latency-bound |
| Topology | GPU-to-NIC affinity, switch tiers, hop count and placement across CPU sockets | Extra hops and cross-socket paths reduce effective throughput |
| Resilience | Capacity and behavior after a link, NIC or switch failure | A nominally fast fabric can become unusable after losing one path |
Ethernet or InfiniBand?
Both Ethernet and InfiniBand can serve as the cluster interconnect in the architectures cited above. Choose based on the complete design: switch and NIC availability, software support, congestion control, operations expertise, telemetry, failure recovery and measured application performance. A 400 Gbps port does not guarantee 400 Gbps of useful all-reduce traffic, and a lower-rate fabric may be adequate for a lightly communicating workload.
For Ethernet, specify the intended transport and loss-management features, such as RoCE configuration where applicable, rather than writing “Ethernet” as if every implementation behaved the same. For InfiniBand, verify the required host-channel adapters, switch fabric, fabric management and integration with the framework’s communication libraries. In either case, benchmark the largest planned job, not just a point-to-point test between two idle ports.
Quick wins for a faster PC:
Scan for outdated or missing drivers - takes under a minuteDriver Scan →Repair Windows errors before they cause bigger problemsFix Now →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Separate storage capacity from storage performance
Capacity model
Capacity is the space required to keep data and recoverable state. Sum the following categories, then add headroom for growth and temporary copies:
Rank #3
- Ideal for high speed, low power storage
- Gen 4x4 NVMe PCle performance
- Up to 6,000MB/s read, 4,000MB/s write
- Includes Acronis cloning software
- 5-year limited warranty
- Source datasets and transformed or sharded versions.
- Model weights, tokenizer files, configuration and evaluation artifacts.
- Checkpoints, including the largest in-progress checkpoint and the number retained.
- Local scratch, preprocessing output, logs and experiment metadata.
- Replicas or snapshots required by the availability policy.
Checkpoint retention can dominate capacity even when the training dataset is modest. Calculate the size of one complete checkpoint, multiply by the retained versions, and include a temporary write area if the application writes a new checkpoint before deleting the old one. Do not claim a universal “AI server capacity”: the required terabytes depend on the model, dataset, retention policy and whether copies are staged locally or kept on shared storage.
Performance model
Performance is the rate at which the workload can read samples and write state. It depends on data format, dataset size, cache hit rate, concurrent jobs, file or object layout and checkpoint behavior. Training often rereads data across epochs; a dataset that fits in RAM or local NVMe may generate little shared-storage traffic after staging, while a larger dataset can sustain reads throughout the run.
Synchronous checkpoint writes are a special case: forward progress can stop while the checkpoint is committed. Measure the peak write bandwidth and the pause time, not only the average daily throughput. NVIDIA notes that model, workload, data format and organizational needs all affect storage requirements, and advises characterizing workloads before finalizing performance and capacity. The DGX B300 SuperPOD storage architecture states, “It is best to characterize workloads and organizational needs before finalizing performance and capacity requirements.”
Use bandwidth references correctly
NVIDIA’s HGX AI Factory certified-storage guidance uses approximately 12.5 Gb/s per GPU as a storage-bandwidth planning guideline and gives an example of approximately 200 Gb/s aggregate for 16 GPUs. This is a reference-architecture example, not a universal minimum.
Convert units before comparing a requirement with a device or link. 12.5 Gb/s is 1.5625 GB/s as a theoretical byte rate, before protocol overhead, filesystem behavior, replication and other real-world losses. The 16-GPU example’s 200 Gb/s is 25 GB/s in the same theoretical conversion. A storage system advertised in GB/s and a NIC advertised in Gb/s are not directly comparable until you divide gigabits by eight and account for usable throughput.
The same document cautions that “the Storage bandwidth (BW) per GPU is highly variable based on workload, model, and performance requirements.” Use the figure to create a first-pass design and a benchmark target, then replace it with measured application demand.
Rank #4
- GROUNDBREAKING READ/WRITE SPEEDS: The 990 EVO Plus features the latest NAND memory, boosting sequential read/write speeds up to 7,250/6,300MB/s. Ideal for huge file transfers and finishing tasks faster than ever.
- LARGE STORAGE CAPACITY: Harness the full power of your drive with Intelligent TurboWrite2.0's enhanced large-file performance—now available in a 4TB capacity.
- EXCEPTIONAL THERMAL CONTROL: Keep your cool as you work—or play—without worrying about overheating or battery life. The efficiency-boosting nickel-coated controller allows the 990 EVO Plus to utilize less power while achieving similar performance.
- OPTIMIZED PERFORMANCE: Optimized to support the latest technology for SSDs—990 EVO Plus is compatible with PCIe 4.0 x4 and PCIe 5.0 x2. This means you get more bandwidth and higher data processing and performance.
- NEVER MISS AN UPDATE: Your 990 EVO Plus SSD performs like new with the always up-to-date Magician Software. Stay up to speed with the latest firmware updates, extra encryption, and continual monitoring of your drive health–it works like a charm.
Choose local NVMe, shared storage, or both
Local NVMe as cache or staging
Local NVMe can hold a working set, stage a dataset before a run, absorb temporary files and reduce repeated reads from shared storage. NVIDIA’s certified-system configurations describe one local NVMe device per CPU socket and a 1 TB minimum. Those are configuration-specific baselines, not proof that 1 TB is enough for your dataset or checkpoint plan. Check endurance, write-cliff behavior, namespace isolation and the server’s PCIe lane budget.
The Tool Desk
Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →Outbyte PC Repair FREEClear out junk files and repair common Windows errorsFree Scan →Shared high-performance storage
Shared storage lets multiple nodes access common datasets and centralizes checkpoint durability. Size it for simultaneous readers, metadata operations and peak checkpoint writes. Confirm that the storage network has independent capacity from the compute fabric and that the array, clients and switches can sustain the required rate at the expected concurrency.
Tiered design
A common pattern is shared storage for the authoritative dataset and checkpoints, with local NVMe used for staging or caching. Define cache warm-up time, eviction rules and behavior after a node reboot. If a local cache is lost, decide whether the job can refill it automatically or must restart from shared storage. For multimodal or image-heavy jobs, do not assume an ordinary language-model profile is representative: NVIDIA’s B300 architecture distinguishes workloads whose data fits in local cache from enhanced multimodal workloads needing more sustained I/O, and notes that checkpoint files can reach terabytes.
Worked sizing examples
Sixteen-GPU storage reference
Applying the cited 12.5 Gb/s-per-GPU planning value to 16 GPUs gives 200 Gb/s aggregate, or 25 GB/s theoretical. Treat that as the target for the specified reference architecture, then subtract expected protocol and system overhead when selecting usable storage throughput. If four jobs share the system, test the distribution of that bandwidth rather than assuming each receives exactly one quarter.
Checkpoint capacity
Suppose a workload creates a 3 TB checkpoint every four hours and retains five versions. The retained set requires 15 TB before snapshots, replication and the temporary space needed while a new checkpoint is written. If the application writes synchronously, measure whether the storage system can complete that write within the acceptable pause window; capacity alone does not answer that question.
Cache-fit decision
If a 600 GB active dataset and preprocessing output fit on a node’s local NVMe, stage them once and measure subsequent shared-storage traffic. If the active set is 20 TB and several nodes read it concurrently, local staging may be too slow or too small; size shared throughput and metadata performance for the sustained workload instead.
Procurement and validation procedure
- Map the topology. Obtain the server’s GPU, NIC, NVMe, CPU-socket and PCIe-switch diagram. Record GPU-to-NIC affinity and NVLink connections.
- Define the largest job. Record node count, GPUs per node, parallelism, expected utilization and the communication collectives used.
- Estimate compute-fabric demand. Select Ethernet or InfiniBand, calculate per-GPU and aggregate bandwidth, and check switch bisection and oversubscription.
- Estimate storage capacity. Add datasets, artifacts, checkpoints, retained versions, scratch and temporary-copy space; then apply the required growth and redundancy policy.
- Estimate storage performance. Measure sustained and burst reads, random and small I/O, metadata operations and peak checkpoint writes for the real formats and concurrency.
- Design tiers and paths. Decide what belongs on shared storage, local NVMe or both. Keep storage, compute, customer and management traffic on intentionally sized paths.
- Build a pilot. Run the largest realistic data-loader, training or inference job with production-like concurrency. Capture GPU utilization, step time, collective latency, cache-hit rate, storage throughput and checkpoint pause time.
- Test failure behavior. Remove a link or storage path in a controlled test, verify redundancy and record the performance available after failover.
- Finalize the bill of materials. Specify exact GPU and server platform, NIC count and placement, switch ports and tiers, NVMe devices, shared-storage capacity and usable throughput, plus cabling and support requirements.
Common sizing mistakes and fixes
- Using GPU count as the only input: add workload communication, node count, topology and concurrency.
- Confusing compute and storage bandwidth: document separate fabrics and test each path independently.
- Comparing Gb/s with GB/s: divide by eight, then account for protocol and system overhead.
- Buying one NIC per GPU automatically: derive port count from topology and measured traffic; the certified guide’s upper figure is configuration-specific.
- Ignoring oversubscription: calculate bisection bandwidth and test all nodes communicating simultaneously.
- Sizing only average storage throughput: include data-loader bursts, metadata operations and synchronous checkpoint peaks.
- Assuming 1 TB local NVMe is universal: treat the documented 1 TB baseline as configuration guidance and compare it with the actual active set and scratch requirements.
- Ignoring recovery space: include temporary checkpoint files, retained versions, snapshots and replication before declaring capacity sufficient.
- Benchmarking empty systems: test with production formats, concurrency, cache state and the largest planned job.
What a defensible design specification contains
A procurement-ready specification should state the GPU and server platform, node count, scale-up topology, scale-out fabric and usable bisection bandwidth, NIC placement, storage read and write targets, capacity by data category, local-cache policy, checkpoint interval and retention, redundancy behavior, and the benchmark used to validate each target. It should also identify which figures are measured, which are architectural references and which remain assumptions.
NVIDIA’s Inference Reference Architecture and the other architecture documents linked above can provide starting patterns, but no cited source establishes a universal network rate or storage capacity for every multi-GPU server. The final bill of materials requires the actual GPU and server design, workload, dataset, node count and checkpoint requirements.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
