There is no single storage bandwidth or latency number that fits every AI system. The right target is the performance needed to keep accelerators supplied with data, complete checkpoint saves and restores within your recovery window, and meet the latency or throughput objective of the serving workload.
Size storage from measured workload behavior—bytes read per step, cache hit rate, checkpoint size and interval, concurrency, metadata activity, and network limits—not from a drive specification or a headline sequential benchmark.
As an Amazon Associate I earn from qualifying purchases.
Start with the workload, not the storage device
AI storage requirements change with model type, dataset modality, cache residency, number of workers, concurrent jobs, and the way data moves through the network. A training run may read the same examples repeatedly, while an inference service may load a model once and then be dominated by request latency—or may repeatedly fetch features, embeddings, or multimodal assets.
The Tool Desk
Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →Outbyte Driver Updater FREEScan for outdated or missing drivers - takes under a minuteDriver Scan →NVIDIA states the principle clearly in its DGX B300 SuperPOD storage architecture: "DGX SuperPOD is designed to support all workloads, but the storage performance required to maximize training performance can vary depending on the type of model and dataset." Its published figures are reference-architecture targets, not universal minimums. A different GPU count, filesystem, topology, model, or data pipeline can require substantially different values.
#1 Best Overall
- MEET THE NEXT GEN: Consider this a cheat code; Our Samsung 990 PRO Gen4 SSD helps you reach near max performance with lightning-fast speeds; Whether you’re a hardcore gamer or a tech guru, you’ll get power efficiency built for the final boss
- REACH THE NEXT LEVEL: Gen4 steps up with faster transfer speeds and high-performance bandwidth; With a more than 55% improvement in random performance compared to 980 PRO, it’s here for heavy computing and faster loading
- THE FASTEST SSD FROM THE WORLD'S FLASH MEMORY BRAND: The speed you need for any occasion; With read and write speeds up to 7450/6900 MB/s you’ll reach near max performance of PCIe 4.0 powering through for any use
- PLAY WITHOUT LIMITS: Give yourself some space with storage capacities from 1TB to 4TB; Sync all your saves and reign supreme in gaming, video editing, data analysis and more
- IT’S A POWER MOVE: Save the power for your performance; Get power efficiency all while experiencing up to 50% improved performance per watt over the 980 PRO; It makes every move more effective with less consumption
The four questions that determine a target
- Can the data stay in a useful cache? If workers read from accelerator or host memory, local NVMe, or a warm shared-filesystem cache, remote storage is less often on the critical path. A cache miss, first uncached epoch, or dataset larger than cache changes the calculation.
- How many bytes must arrive per unit of compute? Measure bytes per sample or training step, worker count, and target step rate. Include preprocessing, shuffling, decompression, and metadata operations.
- How quickly must state be saved and restored? Checkpoint size, save interval, number of simultaneous writers, and acceptable recovery time create a separate read/write budget.
- What does serving promise? Online inference is usually constrained by time to first token, time per output token, tail latency, concurrency, and request throughput. Offline inference can use larger batches and optimize total completion time instead.
Why training reads are often the dominant concern
Deep-learning training generally iterates over data. The same dataset may be consumed over many steps or epochs, so sustained read delivery and cache effectiveness matter more than a one-time sequential transfer test. Recommender workloads can sometimes train in a fraction of an epoch, while existing-model inference can also be I/O intensive when every request pulls features or media.
Cache-resident (standard) workloads
NVIDIA describes a standard class in which multiple concurrent large-language-model or fine-tuning jobs use periodic checkpoints and most data fits an effective local compute-memory cache. Compute tends to dominate elapsed time, so storage still has to avoid starving workers and absorb checkpoint traffic, but extreme read bandwidth is less likely to improve every step.
Cache-missing (enhanced) workloads
Its enhanced class covers concurrent multimodal jobs whose datasets exceed local cache. Image, video, audio, and other large records make storage I/O part of end-to-end training time. In this case, cache misses, parallel reads, metadata operations, and network placement can determine whether accelerators remain busy.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
A shared-filesystem RAM cache can be much faster than repeatedly retrieving the same objects remotely. Treat the first uncached pass and later warm-cache passes as different test cases; reporting only the warm result can hide a startup bottleneck.
Rank #2
- Ideal for high speed, low power storage
- Gen 4x4 NVMe PCle performance
- Up to 6,000MB/s read, 4,000MB/s write
- Includes Acronis cloning software
- 5-year limited warranty
NVIDIA DGX B300 reference targets (and how to interpret them)
The following values come from NVIDIA’s current DGX B300 SuperPOD reference architecture. “SU” is that architecture’s system-unit term. The figures are aggregate targets for the stated configuration, not independent benchmark results and not requirements for every AI deployment.
| Workload class | One SU aggregate read | One SU aggregate write | Eight-SU aggregate read | Eight-SU aggregate write | Architecture context |
|---|---|---|---|---|---|
| Standard | 80 GB/s | 40 GB/s | 640 GB/s | 320 GB/s | Concurrent LLM or fine-tuning jobs, periodic checkpoints, generally cache-resident and compute-dominated |
| Enhanced | 250 GB/s | 124 GB/s | 2,000 GB/s | 992 GB/s | Concurrent multimodal jobs whose data exceeds local cache and for which I/O affects training time |
Do not multiply these numbers onto an unrelated cluster and call the result a minimum. Scaling depends on the number of clients, filesystem architecture, network fabric, data layout, compression, cache hit rate, and whether reads and writes contend. Use the table as an example of how a vendor expresses aggregate goals after characterizing a specific system.
Checkpoint performance needs its own budget
Checkpointing protects against hardware failures and lets a run resume. A save can pause or throttle computation; a restore delays the next run. A storage design that feeds training quickly but writes checkpoints slowly can still miss an availability or recovery objective.
Recommended Free Tools
First-order calculation
Start with:
required bandwidth = checkpoint size ÷ acceptable save (or restore) time
Rank #3
- SPEED UP PROJECTS. Launch creator applications fast with uncompromising PCIe 4.0 read speeds up to 7,100MB/s,[2] (1TB and 2TB[1] models) and write speeds up to 6,700MB/s[2] (1TB[1]-4TB[1] models).
- CREATE AND STORE MORE. Make more room for your 4K videos and high-resolution images with capacities from 500GB[1] up to 4TB[1] on M.2 2280 built with our trusted 8th generation SANDISK BiCS QLC 3D CBA NAND.
- IT GOES WHERE YOU GO. With an all-new power efficient design, your drive delivers high performance with low power, giving you more time to be productive while on the go.
- UNCOMPROMISED RELIABILITY. With up to 1,200 TBW[3] (4TB[1] model) endurance rating, your drive is designed for creators.
- KEEP YOUR DRIVE UPDATED. Monitor your SSD’s performance and check for updates with the downloadable SANDISK Dashboard application.[5]
For example, a 2 TB checkpoint that must be written in 120 seconds implies about 17.1 GB/s of sustained application-level throughput before accounting for replication, serialization, metadata, protocol overhead, parallel writers, and competing jobs. This is a planning calculation, not a performance guarantee.
Concurrency changes the result
If eight jobs checkpoint simultaneously, each job’s nominal rate can be reduced by shared network and filesystem limits. Measure synchronized saves, staggered saves, and restores; they produce different contention patterns. Include model loading, optimizer state, shard counts, and small-file metadata when calculating the actual checkpoint set.
Cloud-specific patterns
Google Cloud’s TPU VM guidance positions Managed Lustre for synchronous checkpoints and high-speed reinforcement-learning weight propagation where sub-1-millisecond latency is relevant. It positions Cloud Storage Rapid Bucket for high-throughput asynchronous and multi-tiered checkpointing. Those recommendations describe services in that environment, not guarantees for other platforms.
AWS guidance for large-scale checkpoint workflows discusses S3 Express One Zone and FSx for Lustre, including hierarchical distribution to reduce repeated external reads. The design lesson is portable: stage or fan out state close to workers when repeatedly reading a checkpoint, but validate durability, failure handling, and consistency for your application.
Rank #4
- HUGE SPEED BOOST: Get random read/write speeds that are 40%/55% faster than 980 PRO; Experience up to 1400K/1550K IOPS, while sequential read/write speeds up to 7,450/6,900 MB/s reach near the max performance of PCIe 4.0*
- BREAKTHROUGH POWER EFFICIENCY: Use less power and get more performance; Enjoy up to 50% improved performance per watt over 980 PRO, plus optimal power efficiency with max PCIe 4.0 performance**
- SMART THERMAL CONTROL: Samsung's own nickel-coated controller delivers effective thermal control; With its slim size, 990 PRO is a perfect fit for desktops and laptops that meet the PCI-SIG D8 standard***
- THE CHAMPION MAKER: Up to 65% improvement in random performance enables faster loads for an ultimate gaming experience on PS5 and DirectStorage PC games****
- SAMSUNG MAGICIAN SOFTWARE: Get the most out of your SSD with Samsung Magician's advanced yet intuitive optimization tools; Monitor drive health, protect valuable data, and receive important updates for your 990 PRO
Inference has distinct online and offline profiles
Online serving
Define a latency and concurrency contract before choosing storage. Record time to first token (TTFT), time per generated token, p95 or p99 tail latency, requests per second, concurrent sessions, and model-load time. A service may have adequate average throughput yet violate its tail-latency objective when a cache miss, model reload, or checkpoint-like update occurs.
Published inference benchmarks are meaningful only with their model, hardware, scenario, quality or accuracy target, batch policy, and latency constraint attached. A throughput figure from an offline batch cannot be compared directly with an interactive server result.
Offline inference
Batch inference can trade latency for throughput by increasing batch size and prefetching inputs. State the completion deadline, total data volume, acceptable retry behavior, and whether outputs are streamed or written in large files. Sequential throughput may matter more than microsecond-level latency, but small-file and metadata rates can still dominate pipelines containing millions of records.
Do these 3 things before closing this tab:
1Clear out junk files and repair common Windows errors2Scan for outdated or missing drivers - takes under a minute3Repair Windows errors before they cause bigger problemsCompare storage options on the dimensions that affect AI
Evaluate every candidate under the workload you will actually run. Aggregate bandwidth alone can conceal a per-node bottleneck, high tail latency, or metadata collapse.
Best Value
- This product has been replaced by our latest generation. Please search for the SANDISK Optimus GX 7100 NVMe SSD
- HIGH-OCTANE GAMING. Experience speeds up to 7,250MB/s read and 6,900MB/s write (1-2TB models), with up to 35% faster performance than previous generation.
- PURPOSE-BUILT. Designed for serious on-the-go gamers, with a PCIe Gen4 interface and SANDISK’s next generation TLC 3D NAND.
- MORE TIME TO CLEAR THAT CHECKPOINT. Built with laptops and handheld gaming devices in mind, with up to 100% more power efficiency over the previous generation.
- DO MORE WITH DASHBOARD. Ensure your drive is optimized for prime performance with the downloadable WD_BLACK Dashboard (Windows only).
| Option or role | Where it helps | Questions to verify | Important limitation |
|---|---|---|---|
| Local NVMe SSD cache or staging | Fast local reuse of hot training shards, temporary preprocessing, and model staging | Per-node read/write rate, capacity for the working set, endurance, cache warming time, eviction policy, and recovery after node replacement | Not shared storage; every node needs a consistent staging and refresh strategy |
| Shared parallel filesystem | High-concurrency access to common datasets, checkpoints, and model shards across many workers | Aggregate and per-client bandwidth, latency and tail latency, IOPS, metadata rate, scaling with simultaneous clients, and network path | Contention, namespace hot spots, and network oversubscription can reduce performance under bursts |
| Object storage | Durable datasets, asynchronous checkpoints, archives, and multi-tier workflows | Request-rate limits, object-size behavior, consistency semantics, transfer latency, lifecycle policy, and egress or access costs | Small random reads and synchronous save/restore deadlines may require a cache or filesystem layer |
| Hybrid cache plus shared or object tiers | Hot data stays near compute while durable copies remain centrally managed | Warm-up and invalidation behavior, prefetch bandwidth, failure recovery, and checkpoint promotion rules | More components to operate and more opportunities for stale or incomplete data |
For each option, test sustained and peak reads and writes per node and cluster, latency distribution, small-random I/O, metadata concurrency, simultaneous-client scaling, cache behavior, checkpoint save and restore under contention, and operational or network limits. Google Cloud’s guidance similarly emphasizes I/O, throughput, scale, latency, metadata concurrency, network bandwidth, and placement rather than a single drive metric.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.A practical sizing procedure
- Describe the workload. Record model family, dataset size and modality, file or object count, sample and shard sizes, compression, preprocessing, epoch or step behavior, and number of workers and jobs.
- Measure data demand. Capture bytes read per step or sample, target step rate, read amplification, cache hit rate, and the split between memory, local NVMe, and remote storage. Measure both cold and warm cache conditions.
- Set checkpoint objectives. Write down checkpoint size, interval, number of concurrent writers, required save time, required restore time, durability level, and whether optimizer and metadata files are included. Apply the bandwidth calculation, then add overhead and contention margin.
- Define serving objectives. For online systems, specify TTFT, per-token latency, tail percentile, concurrency, model-load time, and request throughput. For offline systems, specify batch size, total records, output volume, and completion deadline.
- Map the network path. Check host-to-storage links, oversubscription, placement, routing, and whether training, checkpoint, and control traffic share links. A fast array cannot deliver its rated bandwidth through a saturated uplink.
- Load-test representative behavior. Reproduce shard sizes, randomization, metadata calls, synchronized checkpoints, model loads, and the intended number of clients. Measure accelerator utilization and data-wait time, not storage throughput alone.
- Size for bursts and growth. Include simultaneous jobs, cache misses after restart, rolling model updates, recovery storms, and expected dataset or cluster growth. Leave headroom for the slowest node and tail latency, not just the average.
How to tell when storage is the bottleneck
- Accelerators show regular idle gaps while workers wait for input: inspect read latency, queue depth, cache misses, and per-node throughput before adding GPUs.
- Throughput is high in a single-client test but collapses with jobs: test aggregate fairness, metadata hot spots, network oversubscription, and shared queue limits.
- Training is fast after the first epoch but slow after every restart: measure cache warm-up and staging time; persistent local cache or prefetching may help.
- Checkpoint saves stall all jobs: benchmark synchronized writers, separate checkpoint traffic where possible, and consider asynchronous or hierarchical distribution.
- Average inference latency looks fine but p99 fails: correlate tail latency with cache misses, model loads, garbage collection, metadata calls, and network retransmits.
- Small files consume disproportionate time: package records into larger shards where the framework permits, and benchmark metadata operations independently from bulk bandwidth.
Common sizing mistakes
- Choosing from sequential bandwidth alone. AI pipelines mix large transfers, random reads, metadata, synchronization, and bursts.
- Quoting aggregate bandwidth without per-node numbers. One slow client can starve a worker even when the cluster total looks impressive.
- Ignoring cache state. Publish cold-start, steady-state, and post-failure recovery results separately.
- Treating a vendor target as a universal minimum. The DGX B300 figures apply to NVIDIA’s reference architecture; Google Cloud’s sub-1-ms recommendation applies to particular TPU VM workflows; AWS examples apply to its service patterns.
- Forgetting writes and restores. A read-optimized design can miss recovery objectives or delay deployment when models and optimizer state must be reloaded.
- Testing storage without measuring compute starvation. The useful outcome is shorter step time or better serving SLO compliance, not a larger benchmark number in isolation.
What a defensible requirement statement looks like
Instead of saying “we need very fast storage,” write a requirement that preserves its conditions: “For 16 concurrent multimodal training jobs, with a cold-cache dataset larger than local memory, the system must sustain at least the measured aggregate read rate while keeping p99 input wait below the training budget; it must write each checkpoint within the stated recovery window under synchronized saves; and online serving must meet the defined TTFT, token-latency, concurrency, and throughput targets.” Attach per-node limits, cache state, network topology, test pattern, and growth assumptions.
This format lets you compare local staging, a shared filesystem, object storage, or a hybrid design without pretending that one number describes every workload.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
