Quick wins for a faster PC:
Clear out junk files and repair common Windows errorsFree Scan →Scan for outdated or missing drivers - takes under a minuteDriver Scan →A GPU that is not fully utilized is not automatically waiting on storage. Prove that data movement is limiting useful work, then correlate application timing with storage, network, CPU, PCIe and accelerator telemetry. Map the complete path, benchmark the suspected layer, and change one supported variable at a time. Consider GPUDirect Storage (GDS) only when the workload and software stack meet its direct-I/O requirements.
Start with the critical-path question
NVIDIA’s GPUDirect Storage design guidance states: “For IO to be a bottleneck, it must be on the critical path.” That is the rule that prevents the most common diagnostic mistake: treating a busy disk, a low GPU-utilization sample, or a high I/O rate as proof of causation.
As an Amazon Associate I earn from qualifying purchases.
Storage is a limiting factor only when delays in getting data to the accelerator reduce useful application progress. A compute-bound training step can leave storage active without storage determining its duration. Conversely, a small read stream can stall a pipeline if each request blocks the next operation and cannot be overlapped.
Do these 3 things before closing this tab:
1Repair Windows errors before they cause bigger problems2Scan for outdated or missing drivers - takes under a minute3Clear out junk files and repair common Windows errorsBegin with the workload rather than with a device specification. Record a stable baseline using the same dataset, batch size, software, cache state and system load that produce the symptom. Use an application metric such as:
#1 Best Overall
- MEET THE NEXT GEN: Consider this a cheat code; Our Samsung 990 PRO Gen4 SSD helps you reach near max performance with lightning-fast speeds; Whether you’re a hardcore gamer or a tech guru, you’ll get power efficiency built for the final boss
- REACH THE NEXT LEVEL: Gen4 steps up with faster transfer speeds and high-performance bandwidth; With a more than 55% improvement in random performance compared to 980 PRO, it’s here for heavy computing and faster loading
- THE FASTEST SSD FROM THE WORLD'S FLASH MEMORY BRAND: The speed you need for any occasion; With read and write speeds up to 7450/6900 MB/s you’ll reach near max performance of PCIe 4.0 powering through for any use
- PLAY WITHOUT LIMITS: Give yourself some space with storage capacities from 1TB to 4TB; Sync all your saves and reign supreme in gaming, video editing, data analysis and more
- IT’S A POWER MOVE: Save the power for your performance; Get power efficiency all while experiencing up to 50% improved performance per watt over the 980 PRO; It makes every move more effective with less consumption
- time per training step and examples per second;
- tokens per second for language or sequence workloads;
- inference requests per second and p50/p95/p99 latency; or
- startup, checkpoint or data-refresh time when those phases are the problem.
Also record accelerator model and count, storage location, filesystem or protocol, CPU and memory configuration, driver and framework versions, and the exact interval in which performance degrades. Separate cold-cache startup from steady-state behavior; a one-time cache fill is not the same bottleneck as recurring step-time stalls.
Build a synchronized baseline
Describe the symptom precisely
Write down what changed and when. “The GPU is at 40%” is an observation, not a diagnosis. “Step time rises from 180 ms to 420 ms when workers begin reading the next shard, while GPU kernels show idle gaps” is a testable symptom.
Capture at least one healthy run and one affected run. Keep the workload and system state as constant as possible, and note whether the dataset is local, on a shared filesystem, or reached over a network. If multiple accelerators are involved, determine whether all ranks slow together or only the ranks attached to one host, root complex or storage path.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Measure progress and idle time
Instrument the application around data acquisition, preprocessing, host-to-device transfer, and the compute step. The useful question is how much of each iteration is spent waiting for the next usable batch. If data loading overlaps completely with computation, reducing I/O time may not change step time even when the storage path is busy.
Compare data-loading duration with accelerator compute duration and with gaps in the accelerator timeline. A pipeline with 30 ms of visible I/O and 200 ms of compute may gain nothing from a faster drive if the 30 ms is hidden behind those 200 ms. A pipeline with 30 ms of compute and 80 ms of exposed loading is a different case.
Observe every layer during the same interval
Collect telemetry concurrently, using timestamps that let you align application events with device activity. NVIDIA recommends combining GDS statistics with Linux iostat and GPU tools such as nvidia-smi or DCGM’s dcgmi; a single utilization percentage cannot show where time is being lost.
Rank #2
- Ideal for high speed, low power storage
- Gen 4x4 NVMe PCle performance
- Up to 6,000MB/s read, 4,000MB/s write
- Includes Acronis cloning software
- 5-year limited warranty
Application and accelerator evidence
- Mark the beginning and end of input fetch, decode, preprocessing, transfer and compute.
- Record accelerator utilization, memory activity, copy-engine activity and kernel gaps.
- For multi-GPU jobs, compare ranks and devices rather than averaging them into one number.
- Note whether the accelerator stays busy while I/O is active. That pattern suggests overlap, not necessarily a storage problem.
Local-storage evidence
Use iostat or an equivalent device monitor to capture throughput, operations per second, request size, queue depth, await or latency, and utilization for each device. A high utilization value can mean the device is saturated, but it can also reflect many small requests at modest bandwidth. Compare the observed request pattern with the workload’s actual read/write mix.
Check every drive separately. One overloaded device can limit a job even when aggregate host throughput looks healthy. Conversely, low device utilization during accelerator starvation points toward another layer, such as CPU parsing, a filesystem lock, a network path, or application scheduling.
Network-storage evidence
For remote data, measure the NIC, network or fabric, filesystem or protocol, and storage service at the same time as the client. Look for link saturation, retransmissions, queueing, server-side contention and uneven traffic between hosts. A client-side disk graph cannot explain a bottleneck that occurs in a shared storage system or in the network between the client and that system.
GDS and profiler evidence
When GDS is installed and supported, enable its statistics before collecting them. NVIDIA documents that gds_stats collection is disabled by default; enable profile:cufile_stats in /etc/cufile.json, then collect the statistics during a representative run. Keep the configuration change documented and revert it if operational policy requires the default.
NVIDIA Nsight Systems can trace GDS API activity and expose GDS user-space metrics. Its options and metric availability depend on the installed release, so use the Nsight Systems User Guide that matches the version on the host. A profiler timeline is especially useful for distinguishing file-read time, CPU work, memory copies and accelerator kernel execution.
Outdated Drivers Are Slowing You Down
One free scan finds every outdated or missing driver and matches the right update for your exact hardware.Free scan · exact hardware matchPC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11Map the physical and software path
Draw the route that one sample takes from persistent storage to accelerator memory. The path often contains more limiting links than the drive itself.
Rank #3
- SPEED UP PROJECTS. Launch creator applications fast with uncompromising PCIe 4.0 read speeds up to 7,100MB/s,[2] (1TB and 2TB[1] models) and write speeds up to 6,700MB/s[2] (1TB[1]-4TB[1] models).
- CREATE AND STORE MORE. Make more room for your 4K videos and high-resolution images with capacities from 500GB[1] up to 4TB[1] on M.2 2280 built with our trusted 8th generation SANDISK BiCS QLC 3D CBA NAND.
- IT GOES WHERE YOU GO. With an all-new power efficient design, your drive delivers high performance with low power, giving you more time to be productive while on the go.
- UNCOMPROMISED RELIABILITY. With up to 1,200 TBW[3] (4TB[1] model) endurance rating, your drive is designed for creators.
- KEEP YOUR DRIVE UPDATED. Monitor your SSD’s performance and check for updates with the downloadable SANDISK Dashboard application.[5]
Local-drive route
- Identify the drive or drives and their interface, lane width, firmware and form factor.
- Record which PCIe slot, switch and root complex each drive uses.
- Map the accelerator’s PCIe connection and determine whether it shares a switch, root complex or link with storage.
- Check aggregate lane and switch capacity when several drives feed one accelerator.
- Confirm that the filesystem and application path are supported for the intended direct-I/O method.
Topology can dominate delivered performance. NVIDIA’s design material gives a platform-specific example in which at least four x4 PCIe drives are needed to saturate a x16 PCIe link. That is an illustrative design example, not a sizing rule for every server. A single drive with a high headline bandwidth cannot deliver that bandwidth if its slot, switch or upstream link is narrower.
Network-storage route
- Locate the storage service and identify its protocol and filesystem.
- Trace the client’s NIC to its PCIe slot, switch and CPU root complex.
- Trace the network or fabric path, including link speeds and oversubscribed switches.
- Identify server-side disks, caches and shared contention where you have access to them.
- Map the final path from NIC or filesystem buffers to accelerator memory.
For both local and remote storage, include CPU sockets and NUMA placement. A cross-socket transfer, an incorrectly placed worker, or an extra host-memory copy can consume the margin that a storage benchmark appeared to provide.
Inspect the application’s data path
Determine which component first touches the bytes and which component performs the final transfer. CPU-side parsing, decompression, augmentation, memory mapping, pageable buffers or migrating memory can make the application CPU-bound even when the drive and network have spare capacity. Unsupported filesystems or a failed direct-I/O setup can silently route data through a fallback path.
GDS guidance describes a suitable direct path as one in which the GPU is the first or last agent touching the transferred data, with explicit transfers and pinned GPU buffers in supported applications. If the workload must parse or transform records on the CPU before the GPU can use them, GDS cannot remove that required work.
Benchmark the suspected layer without fooling yourself
A component benchmark tells you what a path can deliver under specified conditions; it does not prove that the application will improve. Benchmark the relevant layer, then return to the original workload.
Choose a representative test
- Request shape: match block or record size, sequential versus random access, and read/write ratio.
- Concurrency: reproduce worker count, queue depth and the number of accelerators issuing requests.
- Dataset size: use enough data to avoid an accidental cache-only result and include the steady-state phase.
- Target path: test the same mount, device, NIC and filesystem used by the application.
- Metrics: capture sustained throughput, IOPS and average and tail latency, not just a peak bandwidth number.
Use the appropriate tools
Use NVIDIA’s gdsio where the GDS path applies. Use a storage benchmark such as fio for a local device or filesystem test, and a network test for the relevant fabric layer. Keep destructive tests away from production data and follow the storage provider’s test guidance.
Rank #4
- HUGE SPEED BOOST: Get random read/write speeds that are 40%/55% faster than 980 PRO; Experience up to 1400K/1550K IOPS, while sequential read/write speeds up to 7,450/6,900 MB/s reach near the max performance of PCIe 4.0*
- BREAKTHROUGH POWER EFFICIENCY: Use less power and get more performance; Enjoy up to 50% improved performance per watt over 980 PRO, plus optimal power efficiency with max PCIe 4.0 performance**
- SMART THERMAL CONTROL: Samsung's own nickel-coated controller delivers effective thermal control; With its slim size, 990 PRO is a perfect fit for desktops and laptops that meet the PCI-SIG D8 standard***
- THE CHAMPION MAKER: Up to 65% improvement in random performance enables faster loads for an ultimate gaming experience on PS5 and DirectStorage PC games****
- SAMSUNG MAGICIAN SOFTWARE: Get the most out of your SSD with Samsung Magician's advanced yet intuitive optimization tools; Monitor drive health, protect valuable data, and receive important updates for your 990 PRO
For each run, record queueing, concurrency, block size, cache state, start and end times, and the exact path tested. Compare the result with the hardware and link limits for that path. A benchmark that reaches the drive’s specification while the application receives a fraction of it indicates a higher-layer limit, not a failed drive.
Decide whether GDS addresses the measured limit
GDS provides a direct DMA route between storage and GPU memory that can avoid a CPU bounce buffer. It is not a universal “faster storage” switch. Evaluate it only after the critical-path test and topology mapping identify host-memory movement as material to the workload.
Conditions that make GDS worth testing
- I/O occupies a meaningful, exposed portion of the critical path.
- The application uses explicit transfers rather than relying on an opaque or unsupported memory path.
- GPU buffers are pinned as required by the supported API and application design.
- The GPU, CUDA and cuFile software stack, filesystem and storage path are supported for the installed GDS release.
- The workload can benefit from removing a CPU bounce buffer instead of requiring CPU parsing or transformation first.
If asynchronous transfers are already fully hidden behind compute, or if CPU processing dominates before and after the transfer, GDS may produce little or no application-level improvement. Confirm operation with the documented checks for the exact CUDA/GDS release, then benchmark and profile; a configuration flag alone does not prove that bytes are taking the direct path.
Use NVIDIA’s GPUDirect Storage Design Guide for suitability and data-path requirements, and the GPUDirect Storage Benchmarking and Configuration Guide for installation checks, gdsio, gds_stats and topology considerations.
Interpret common symptom patterns
| Observed pattern | Likely investigation direction | Next measurement |
|---|---|---|
| GPU idle gaps coincide with high storage latency and data-loader waits | Storage or an upstream filesystem path may be exposed on the critical path. | Correlate per-device latency and application wait spans; test request size and concurrency. |
| GPU remains busy while storage reads continue | I/O may be fully overlapped with compute; storage activity alone is not proof of a bottleneck. | Compare step time after a controlled storage change or reduced prefetch depth. |
| Drive bandwidth is low, CPU usage is high, and preprocessing spans are long | Parsing, decoding, augmentation or buffer handling may dominate. | Profile CPU stages and measure bytes delivered before and after preprocessing. |
| Only one GPU or rank is slow | Uneven drive, NIC, PCIe, NUMA or filesystem placement is possible. | Compare topology and per-rank I/O, CPU affinity and queue depth. |
| All hosts slow when a shared job starts | Network, storage-service or shared-fabric contention may be limiting. | Measure client links and server-side or fabric counters during the overlap. |
| GDS statistics show little direct I/O despite a configured path | The application may be using an unsupported filesystem, fallback path or non-GDS transfer. | Check release-specific support and trace the file and memory APIs with Nsight Systems. |
Compare storage options by delivered application performance
When evaluating a different local drive, a larger drive set or network storage, compare the complete path rather than isolated product specifications. The relevant dimensions are:
| Dimension | What to measure | Why it matters |
|---|---|---|
| Throughput and IOPS | Sustained bytes per second and operations per second at the workload’s request size, concurrency and read/write mix. | Peak sequential bandwidth may not predict many-small-request performance. |
| Latency | Average plus tail latency under realistic queue depth and shared load. | Training stalls can be caused by outliers even when averages look acceptable. |
| Topology | PCIe lanes, switches, root-complex and socket crossings, NIC placement and aggregate link limits. | The narrowest shared link controls delivered bandwidth. |
| Access pattern | Sequential streaming, random or fine-grained reads, cache reuse and preprocessing. | Different patterns stress different layers and caching behavior. |
| Scale and capacity | Dataset size, accelerator count, expected growth and shared-storage contention. | A path that works for one accelerator may not scale to a larger job. |
| Software fit | Filesystem or protocol support, cuFile/GDS compatibility, drivers and operational support. | Unsupported software can force copies or fallback behavior. |
| Application impact | Step time, examples or tokens per second, and inference throughput or latency. | This is the outcome that determines whether a change is worthwhile. |
No universal storage device or vendor winner follows from these dimensions. Select an option only after a representative test confirms compatibility and improves the application metric.
Best Value
- This product has been replaced by our latest generation. Please search for the SANDISK Optimus GX 7100 NVMe SSD
- HIGH-OCTANE GAMING. Experience speeds up to 7,250MB/s read and 6,900MB/s write (1-2TB models), with up to 35% faster performance than previous generation.
- PURPOSE-BUILT. Designed for serious on-the-go gamers, with a PCIe Gen4 interface and SANDISK’s next generation TLC 3D NAND.
- MORE TIME TO CLEAR THAT CHECKPOINT. Built with laptops and handheld gaming devices in mind, with up to 100% more power efficiency over the previous generation.
- DO MORE WITH DASHBOARD. Ensure your drive is optimized for prime performance with the downloadable WD_BLACK Dashboard (Windows only).
Apply one supported change at a time
Treat tuning as a controlled experiment. Preserve the baseline, alter one variable, repeat the same workload and keep the change only if the application improves without unacceptable latency, reliability or operational cost.
- Fix request shape: adjust batching, shard size, prefetch depth or worker concurrency if measurements show inefficient small requests or under-filled queues.
- Fix application buffering: change buffer sizes, pinned-memory use or CPU affinity only when profiling identifies those stages as limiting.
- Fix the saturated component: rebalance drives, move a NIC or accelerator, reduce a shared-link oversubscription, or address a storage-service limit when telemetry identifies that layer.
- Evaluate the direct path: test explicit cuFile/GDS transfers when the workload meets the suitability conditions and the installed stack is supported.
- Re-run the original workload: compare the same step-time, throughput or latency metric, not only the subsystem benchmark.
Retain provider defaults unless a storage provider recommends a change or your measurements demonstrate a benefit. NVIDIA’s configuration guidance documents platform-specific considerations; the exact requirements for settings such as ACS and IOMMU depend on the server, operating system and software release. Verify vendor instructions and operational constraints before changing system-wide settings. See the GPUDirect Storage Best Practices Guide, Release v1.19.1 for the documented considerations.
A practical diagnostic checklist
- Have you reproduced the slowdown with a stable workload and recorded a healthy baseline?
- Is the affected phase startup, cold-cache loading, steady-state training, checkpointing or inference?
- Does exposed I/O time coincide with application waits or accelerator idle gaps?
- Are storage, network, CPU, accelerator and profiler traces aligned to the same interval?
- Have you mapped drives or storage servers, NICs, PCIe switches, root complexes and NUMA placement?
- Are CPU parsing, decoding, augmentation or pageable-memory copies dominating the path?
- Does the benchmark use the workload’s request size, concurrency, cache state and target path?
- Have you compared average and tail latency as well as throughput and IOPS?
- If considering GDS, are GPU, cuFile, filesystem and CUDA versions supported, and are direct transfers verified?
- Did you change one variable and validate the result with the original application metric?
Further technical context
AI systems use a hierarchy that may include local NVMe, networked filesystems, shared storage, caches and preprocessing services. NVIDIA’s overview of this hierarchy is available in Tips on Scaling Storage for AI Training and Inferencing. Use it as architectural context, not as a substitute for measurements on your server.
For local NVMe planning, verify server form factor, interface, firmware, endurance and lane allocation before purchase or deployment. An NVMe SSD is relevant only when testing shows that local storage limits the workload; it will not correct a network, PCIe-topology, CPU-preprocessing or filesystem bottleneck.
Frequently Asked Questions
Does low GPU utilization prove that storage is slow?
No. Low utilization can result from CPU preprocessing, synchronization, insufficient parallelism, a software fallback or compute scheduling. Correlate accelerator idle gaps with application waits and storage or network latency before blaming storage.
When should I use GDS instead of ordinary host-memory transfers?
Evaluate GDS when exposed I/O is materially on the critical path, the application can use explicit supported transfers and pinned GPU memory, and the GPU, filesystem and cuFile stack are compatible. Benchmark the real workload after verifying the direct path.
Why can a high-end NVMe drive fail to improve training speed?
The limit may be a shared PCIe link, switch or root complex, CPU parsing, filesystem behavior, network storage, request shape or fully hidden asynchronous I/O. Measure the complete route and the application metric.
Free tools Windows power users keep installed
One-click scans. No signup required.
What should I collect before asking a storage vendor for help?
Provide the workload and access pattern, baseline and affected timings, drive or storage-service details, filesystem, accelerator and software versions, synchronized iostat/network/GPU data, topology, benchmark parameters and a profiler timeline showing where waits occur.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
