Do these 3 things before closing this tab:
1Fix the driver behind crashes, sound loss and screen glitches2Repair Windows errors before they cause bigger problems3Scan for outdated or missing drivers - takes under a minuteA powerful GPU or AI accelerator can sit idle when data cannot reach it fast enough. Every training batch, model load, checkpoint, cache entry and network request travels through links with finite bandwidth and latency. If storage, preprocessing, host-memory copies, an interconnect, or distributed communication is slower than computation, the accelerator waits and end-to-end performance falls.
The remedy is not automatically a faster GPU or SSD. First identify which path is limiting the workload, then change that path: reduce copies, improve locality, overlap transfers with computation, rebalance preprocessing, or place devices on a topology that provides the required bandwidth.
As an Amazon Associate I earn from qualifying purchases.
What a “data-movement bottleneck” actually is
AI performance is an end-to-end property of a system, not a specification printed on a processor. A model may require billions of arithmetic operations, but those operations only happen when inputs, weights and intermediate values are available in the accelerator’s memory. Moving them involves storage devices, CPU host memory, PCIe or another interconnect, GPU memory, network links and synchronization points.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Bandwidth determines how much data can pass per second. Latency determines how long a transfer takes to begin or complete, which matters especially for small requests, short sequences and synchronization. Shared links add contention and jitter: another process can consume bandwidth or force a transfer to wait. A pipeline can therefore have a high theoretical peak and still deliver low utilization.
#1 Best Overall
- Ideal for high speed, low power storage
- Gen 4x4 NVMe PCle performance
- Up to 6,000MB/s read, 4,000MB/s write
- Includes Acronis cloning software
- 5-year limited warranty
Underutilization is the visible symptom. A utilization graph may show the GPU repeatedly rising to full load and dropping to near zero, or remaining low while CPUs, storage or network interfaces are busy. Those patterns indicate waiting, but they do not identify the cause by themselves.
The common storage-to-GPU path and why copies matter
In a conventional loading workflow, an example or model shard is read from a drive into CPU host memory and then copied from host memory into GPU memory. NVIDIA’s interoperability material describes this host-mediated, two-copy route. The CPU may also coordinate I/O, decode files and prepare tensors while the copies consume memory bandwidth and an interconnect.
On supported systems, a direct-storage path can use a storage-side DMA engine to place data in GPU memory without the CPU bounce buffer. NVIDIA’s GPUDirect Storage design guide describes this as a way to reduce copies, CPU overhead and potentially transfer latency. “Direct” does not mean universally available: the drive, GPU, PCIe switches, firmware, drivers, filesystem and software stack must support the path, and the topology must allow it to operate efficiently.
Crashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minutePC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11Even when direct I/O is available, it may not improve a workload whose real limit is elsewhere. If preprocessing takes longer than the read, bypassing host memory will leave the accelerator waiting for the same preprocessing stage. If a remote filesystem is saturated, local DMA cannot remove the network queue. Measure the complete path before changing it.
Training is a pipeline, not one model step
Training repeatedly cycles through data acquisition, preparation, transfer, computation, synchronization and checkpointing. A batch is ready for the next stage only when the previous stage has finished. The University of Texas at Austin dissertation Accelerating Deep Learning Training: A Storage Perspective (December 2021) identifies fetching examples, preprocessing and writing checkpoints as data-pipeline concerns that can constrain training.
Rank #2
- MEET THE NEXT GEN: Consider this a cheat code; Our Samsung 990 PRO Gen4 SSD helps you reach near max performance with lightning-fast speeds; Whether you’re a hardcore gamer or a tech guru, you’ll get power efficiency built for the final boss
- REACH THE NEXT LEVEL: Gen4 steps up with faster transfer speeds and high-performance bandwidth; With a more than 55% improvement in random performance compared to 980 PRO, it’s here for heavy computing and faster loading
- THE FASTEST SSD FROM THE WORLD'S FLASH MEMORY BRAND: The speed you need for any occasion; With read and write speeds up to 7450/6900 MB/s you’ll reach near max performance of PCIe 4.0 powering through for any use
- PLAY WITHOUT LIMITS: Give yourself some space with storage capacities from 1TB to 4TB; Sync all your saves and reign supreme in gaming, video editing, data analysis and more
- IT’S A POWER MOVE: Save the power for your performance; Get power efficiency all while experiencing up to 50% improved performance per watt over the 980 PRO; It makes every move more effective with less consumption
Fetching and staging examples
Random reads from a large dataset create more seeks, metadata operations and small transfers than sequential reads. A remote dataset adds network latency and shared-service contention. If workers cannot stage the next batch while the current batch is computing, the accelerator encounters bubbles between steps.
Preprocessing and decoding
Image decoding, tokenization, augmentation, compression handling and format conversion can consume CPU cores and memory bandwidth. Increasing the number of data-loader workers helps only until the CPU, storage or memory bus becomes saturated. More workers can then increase contention and make latency less predictable.
The Tool Desk
Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →Host-to-device transfers
Prepared tensors still have to cross the host-to-GPU link unless they are produced directly in device memory. Large batches may be bandwidth-bound; many small tensors may be latency-bound. Pinning, batching or asynchronous transfers can help only when the framework and hardware support overlap and the host has enough memory to stage data.
Checkpoint writes
A checkpoint can pause or slow training while model, optimizer and metadata state are serialized and written. On shared storage, a checkpoint from one job can interfere with input reads from many others. Checkpoint frequency, format and destination therefore affect throughput even though no neural-network arithmetic changed.
Distributed communication
Data movement also occurs between accelerators. Gradient reductions, parameter exchanges and pipeline synchronization use an interconnect or network. The slowest link, a misplaced process or a congested switch can hold all participants at a barrier. Faster local computation does not compensate for time spent waiting for peers.
Rank #3
- BREAKTHROUGH PCIe 5.0 PERFORMANCE: Supercharge your workflow and gaming with PCIe 5.0, boasting up to 14,700/13,400 MB/s¹ sequential read/write speeds. Tackle massive files and power up your gaming with Gen5—twice as fast as the 990 PRO SSD.
- EVERY TASK, TURBOCHARGED: Speed past productivity limits. With random read/write speeds up to 1,850K/2,600K IOPS¹, enjoy fast game loads, seamless AI apps, and efficient multitasking. Virtually no lag, no limits—just nonstop performance.
- THINK FAST, CREATE FASTER: With random read/write speeds of up to 1,850K/2,600K IOPS¹, the 9100 PRO SSD fuels seamless AI content creation, swift loads, and smooth gameplay. Work, play, and create at lightning speed.
- SPEED, WHENEVER YOU NEED: From laptops, desktop PCs, to PS5 with added heatsink devices², experience blazing PCIe 5.0 speeds and up to 8TB of storage. Perfect for video editing, gaming, and creative tasks, with the compatibility to match your device.
- STAY COOL, RUN FAST: Push limits, not temperatures. A 5nm controller boosts power efficiency up to 49% over the 990 PRO SSD³, while advanced thermal control keeps performance smooth and reliable.
Inference has different data flows
Inference is often described as a forward pass, but serving a request requires several movements before and during that pass. NVIDIA’s Inference Reference Architecture: Data Flow Diagrams maps storage, cache, network and topology as parts of the serving path.
Model loading and recovery
Weights may be read from local or shared storage when a service starts, scales out or recovers after failure. A large model can make cold-start time a storage and network problem even if steady-state token generation is compute-bound. Replicas that restart together can create a burst that overwhelms shared storage.
Request and response traffic
Tokenization, request routing, batching and response streaming traverse CPU and network paths. For small requests, network and software overhead can dominate arithmetic. For multi-accelerator serving, the placement of workers and the links between them determine whether model or activation traffic crosses a slow boundary.
KV cache and other state
Autoregressive inference keeps key-value (KV) attention data for the conversation or sequence. Longer context, larger batches and more concurrent sessions increase the cache footprint and the traffic needed to read, write, evict or move it. A cache miss that requires fetching state from a slower memory tier can raise latency even when the model weights remain resident.
Throughput versus tail latency
Batching improves utilization by moving more work per transfer, but waiting to form a batch increases request latency. Conversely, serving each request immediately may leave transfer and communication overhead exposed. The correct setting depends on the service objective and the measured distribution of request sizes and arrival times.
Recommended Free Tools
Rank #4
- HUGE SPEED BOOST: Get random read/write speeds that are 40%/55% faster than 980 PRO; Experience up to 1400K/1550K IOPS, while sequential read/write speeds up to 7,450/6,900 MB/s reach near the max performance of PCIe 4.0*
- BREAKTHROUGH POWER EFFICIENCY: Use less power and get more performance; Enjoy up to 50% improved performance per watt over 980 PRO, plus optimal power efficiency with max PCIe 4.0 performance**
- SMART THERMAL CONTROL: Samsung's own nickel-coated controller delivers effective thermal control; With its slim size, 990 PRO is a perfect fit for desktops and laptops that meet the PCI-SIG D8 standard***
- THE CHAMPION MAKER: Up to 65% improvement in random performance enables faster loads for an ultimate gaming experience on PS5 and DirectStorage PC games****
- SAMSUNG MAGICIAN SOFTWARE: Get the most out of your SSD with Samsung Magician's advanced yet intuitive optimization tools; Monitor drive health, protect valuable data, and receive important updates for your 990 PRO
How to locate the limiting path
Diagnose one workload phase at a time: training input, training checkpoint, model loading, steady-state inference or recovery. Keep batch size, sequence length, concurrency and placement recorded because changing any of them can move the bottleneck.
- Define the symptom. Record accelerator utilization, step time or tokens per second, input wait time, checkpoint duration, startup time and tail latency. Note whether the problem appears only at cold start, at larger batches or under concurrent jobs.
- Trace the stages. Timestamp storage reads, preprocessing completion, host-to-device copies, kernel execution, synchronization and network transfers. Look for idle gaps between stages rather than relying on a single utilization percentage.
- Check sustained rates and queues. Compare observed storage, memory, interconnect and network throughput with their available capacity during the same interval. A short burst at peak speed does not prove that the path sustains that rate; queue depth and jitter matter for latency.
- Test locality and contention. Compare local staging with the normal shared or remote source, and run with competing jobs removed if possible. A large change points to the storage or network path, while no change shifts attention to preprocessing, synchronization or compute.
- Count copies and conversions. Identify every movement between drive, host memory, device memory and peer devices. Include format conversion, decompression and serialization. A nominally fast link can be negated by an extra copy or by many small transfers.
- Change one variable. Try a larger transfer size, a different cache policy, fewer or more preprocessing workers, overlapped I/O, or a topology-aware placement one at a time. Keep the same workload and compare both average and worst-case results.
Interventions matched to the measured constraint
When storage bandwidth or latency is limiting
- Stage the hot dataset or model shards on local NVMe when capacity, form factor, thermal limits and endurance permit.
- Use a cache that keeps frequently reused data close to the accelerator, while monitoring eviction and warm-up behavior.
- Prefer larger, contiguous reads where the format allows them; thousands of tiny files amplify metadata and latency costs.
- Consider a supported direct-storage path only after verifying GPU, drive, PCIe topology, drivers and application compatibility.
A faster drive cannot fix a CPU-bound decoder, a saturated network or a synchronization barrier. It may also shift pressure to the host-to-device link once reads become faster.
When host-to-device copying is limiting
- Batch small tensors to reduce per-transfer latency.
- Use asynchronous transfers and double buffering so one batch moves while another computes, provided the framework can overlap them.
- Reduce unnecessary format conversions and intermediate copies.
- Evaluate direct I/O on supported hardware, but verify CPU utilization and end-to-end step time rather than copy time alone.
When preprocessing is limiting
- Profile tokenization, decoding and augmentation separately from file reads.
- Cache deterministic preprocessing results or store data in a representation that reduces repeated conversion.
- Set worker counts to the point where the accelerator stays fed without exhausting CPU cores or memory bandwidth.
- Schedule preprocessing on nodes with adequate CPU capacity instead of assuming every accelerator node has spare host resources.
When the interconnect or topology is limiting
- Place communicating accelerators, storage devices and network interfaces on the intended PCIe switch or high-bandwidth fabric path.
- Avoid configurations that force traffic through an oversubscribed root complex or a remote NUMA domain.
- For distributed jobs, keep ranks close to their data and peers when the platform supports topology-aware placement.
- Check that software sees the same devices and links the hardware design assumes; a compatible component can still be connected through a slower path.
When synchronization or network traffic is limiting
- Measure collective communication and barrier wait time separately from kernels.
- Reduce unnecessary synchronization and choose a batch or parallelism strategy that does not create frequent small exchanges.
- Control checkpoint timing so large writes do not coincide with peak input or serving traffic.
- For inference, place request routing, cache services and model replicas to avoid avoidable cross-zone or cross-host transfers.
What published numbers do—and do not—prove
NVIDIA’s technical blog on GPUDirect Storage reports bounded examples from a described local-NVMe setup. The figures are useful for understanding possible effects, not guarantees for every server or workload.
| Reported observation | Scope and interpretation |
|---|---|
| 13.3 GB/s I/O bandwidth | A local-NVMe DMA example in NVIDIA’s setup; it is not a universal storage baseline. |
| About 10% performance improvement | Reported relative to the CPU-to-GPU memory-transfer rate in that example; the gain depends on the stated configuration and workload. |
| 3.8× lower end-to-end latency | Reported for explicit direct-transfer examples; it should not be generalized to all reads, models or serving systems. |
No representative cross-workload statistic establishes a universal percentage of AI performance lost to data movement. Results vary with model size, sequence length, batch size, cache hit rate, storage location, parallelism and contention. Vendor documentation is valuable for mechanism and configuration guidance, but it is not an independent cross-platform benchmark.
Requirements and trade-offs before changing hardware
Direct storage, larger caches and faster links add compatibility checks and operational complexity. Confirm the server’s drive form factor and interface, available capacity, cooling, firmware, driver versions, filesystem behavior and support in the framework or runtime. A local cache also needs an invalidation and consistency policy when datasets or model versions change.
Best Value
- SPEED UP PROJECTS. Launch creator applications fast with uncompromising PCIe 4.0 read speeds up to 7,100MB/s,[2] (1TB and 2TB[1] models) and write speeds up to 6,700MB/s[2] (1TB[1]-4TB[1] models).
- CREATE AND STORE MORE. Make more room for your 4K videos and high-resolution images with capacities from 500GB[1] up to 4TB[1] on M.2 2280 built with our trusted 8th generation SANDISK BiCS QLC 3D CBA NAND.
- IT GOES WHERE YOU GO. With an all-new power efficient design, your drive delivers high performance with low power, giving you more time to be productive while on the go.
- UNCOMPROMISED RELIABILITY. With up to 1,200 TBW[3] (4TB[1] model) endurance rating, your drive is designed for creators.
- KEEP YOUR DRIVE UPDATED. Monitor your SSD’s performance and check for updates with the downloadable SANDISK Dashboard application.[5]
Capacity is not the same as useful bandwidth. A large drive may be unable to sustain the required rate under concurrent reads, while a smaller local cache may deliver better latency for the hot working set. Likewise, a high-bandwidth interconnect helps only if traffic reaches it rather than taking a slower detour through host memory or an oversubscribed switch.
Account for failure behavior. If a cache is lost, model reloads can create a startup storm; if a direct path is unsupported after a driver update, the system may fall back to host-mediated copies. Keep a measured baseline and a rollback path so an optimization can be removed when its assumptions no longer hold.
A practical decision rule
Start with the phase that users experience as slow and follow its critical path to the waiting point. If storage queues and read latency rise while CPUs and accelerators wait, improve locality or storage. If preprocessing consumes the available CPU time, optimize that stage instead. If copies dominate, reduce them or overlap them. If peers wait at synchronization, inspect communication and topology. If the problem appears only with long contexts or high concurrency, examine KV-cache traffic and network behavior.
Free tools Windows power users keep installed
One-click scans. No signup required.
The fastest accelerator is valuable only when the rest of the system can continuously supply it and retire its results. Treat data movement as a first-class part of model performance, measure it under the workload that matters, and choose the smallest intervention that removes the measured constraint.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
