Halloween DealsAmazon USHalloween week deals: useful picks before month-endAmazon US: current deals, useful picks and tech finds.Check DealsPC HealthRecommendedCrashes, freezes, slowdowns? Check your PC nowSpot repairable issues before they interrupt work.Check PCHalloween DealsAmazon USOctober picks are moving - compare before checkoutAmazon US: current deals, useful picks and tech finds.See Picks×
FREEDOM251

Freedom251 › Blog › AI infrastructure

How to Benchmark SSDs for AI Workloads Beyond Advertised Speeds

By Freedom251 Team · · 10 min read

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Advertised sequential read and write speed is only one device-level number. A useful AI-storage benchmark measures workload-shaped throughput, IOPS and latency, then checks whether the complete data path keeps the target job supplied. Test the SSD in isolation with controlled synthetic workloads and in an end-to-end training, checkpoint, serving or retrieval workload, while documenting cache state and every important system setting.

Start with the workload, not the SSD label

There is no universal “AI SSD” score. Different jobs create different access patterns, and a result that is excellent for one pattern can be irrelevant to another. Before opening a benchmark tool, write down what the storage is expected to do.

As an Amazon Associate I earn from qualifying purchases.

Training and fine-tuning

Training data loaders commonly issue many reads at once, often with randomized sample selection. Record sample and batch sizes, the number of data-loader workers, preprocessing performed on the host, and whether data is read from a local filesystem, a RAID or volume, or shared storage. A benchmark should reproduce the degree of randomization and concurrency that the training job actually uses.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Checkpoint save and restore

Checkpointing is a sustained-write test followed by a sustained-read restore test. Measure the two directions separately. Include the required synchronization or durability operation, because a fast stream into a cache is not equivalent to a checkpoint that is safely committed. Checkpoint operations can be synchronous; if training pauses while the checkpoint is written, report that interruption rather than hiding it behind an average bandwidth number.

#1 Best Overall
Sale
Kingston NV3 1TB M.2 2280 NVMe SSD | PCIe 4.0 Gen 4x4 | Up to 6000 MB/s | SNV3S/1000G
  • Ideal for high speed, low power storage
  • Gen 4x4 NVMe PCle performance
  • Up to 6,000MB/s read, 4,000MB/s write
  • Includes Acronis cloning software
  • 5-year limited warranty

Inference, model loading and KV-cache access

Serving may read model files during startup and repeatedly access a key-value cache during operation. A model-load test is mostly a large-transfer exercise, while KV-cache activity can involve smaller, latency-sensitive reads and writes. If KV-cache storage is part of the design, benchmark it directly and report tail read latency together with throughput.

Vector retrieval

Vector search has its own query, index and metadata behavior. Sequential bandwidth from a drive test cannot predict query performance. Use a vector-database workload with the intended index, vector dimensions, query concurrency and durability settings.

Describe the storage path

Identify whether the target is a bare local SSD, a filesystem on local drives, a RAID or logical volume, or networked/shared storage. Note the dataset size, file-size distribution, number of concurrent jobs, read/write balance, and whether the data can fit in system memory or a local cache. Those facts determine whether you are measuring the SSD, a host bottleneck or a cache.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Build a reproducible device-level baseline

A synthetic baseline isolates the storage layer. It does not replace an AI workload test, but it helps explain one when the results diverge. A tool such as fio can sweep transfer size, randomness, queue depth and concurrency under controlled conditions.

Record the platform before testing

  • SSD model, usable capacity, firmware version and temperature.
  • Interface type, PCIe generation, negotiated link width and whether the drive is behind a switch, adapter or shared backplane.
  • CPU model and count, system memory, operating-system release and kernel.
  • Raw-device or filesystem mode, filesystem and mount options.
  • Benchmark version, complete job configuration and I/O engine.
  • Test-file size, run time, block size, read/write mix, queue depth and job or thread count.
  • Direct-I/O setting, cache-clearing method and drive temperature or throttling observations.

Without these details, two “7,000 MB/s” results may describe different paths, cache states or workloads and should not be ranked together.

Use a test file larger than available memory

If the file fits in client RAM, a later run can be served by the page cache instead of the SSD. Size the test data so that the storage path, rather than memory capacity, is exercised. MLPerf Storage’s described training test uses a dataset at least five times the aggregate client DRAM and processes a minimum number of batches per accelerator to reduce the chance that a run is merely timing a client-memory cache. That five-times figure is a benchmark rule for that test, not a universal SSD-sizing formula.

Rank #2
Sale
Samsung SSD 990 PRO 2TB, PCIe 4.0 M.2 2280, Up to 7,450 MB/s | NVMe Internal Solid State Hard Drive for High End Computing, Gaming, and Heavy Duty Workstations, (MZ-V9P2T0B/AM)
  • MEET THE NEXT GEN: Consider this a cheat code; Our Samsung 990 PRO Gen4 SSD helps you reach near max performance with lightning-fast speeds; Whether you’re a hardcore gamer or a tech guru, you’ll get power efficiency built for the final boss
  • REACH THE NEXT LEVEL: Gen4 steps up with faster transfer speeds and high-performance bandwidth; With a more than 55% improvement in random performance compared to 980 PRO, it’s here for heavy computing and faster loading
  • THE FASTEST SSD FROM THE WORLD'S FLASH MEMORY BRAND: The speed you need for any occasion; With read and write speeds up to 7450/6900 MB/s you’ll reach near max performance of PCIe 4.0 powering through for any use
  • PLAY WITHOUT LIMITS: Give yourself some space with storage capacities from 1TB to 4TB; Sync all your saves and reign supreme in gaming, video editing, data analysis and more
  • IT’S A POWER MOVE: Save the power for your performance; Get power efficiency all while experiencing up to 50% improved performance per watt over the 980 PRO; It makes every move more effective with less consumption

Measure sequential bandwidth

Run large contiguous reads and writes to establish the ceiling for model transfers, bulk dataset staging and checkpoint streams. An illustrative read command is:

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
fio --name=seq-read --filename=/path/testfile --size=200G --rw=read --bs=1M --iodepth=32 --numjobs=4 --direct=1 --runtime=300 --time_based --group_reporting

Repeat with --rw=write only on a disposable test area. The values above are a reproducible example, not a claim about the correct settings for every system. Choose a file size and run time that expose sustained behavior, and state those choices in the report.

Measure small and random I/O

Random reads and writes at small block sizes reveal IOPS and latency behavior that a sequential test misses. Example:

fio --name=rand-read --filename=/path/testfile --size=200G --rw=randread --bs=16k --iodepth=32 --numjobs=8 --direct=1 --runtime=300 --time_based --group_reporting

Repeat with random writes and with block sizes that match the application. Small, random operations are common in production systems, and IOPS is therefore a necessary part of an AI-storage benchmark, not an optional extra.

Test workload-relevant streaming and mixed traffic

Training samples may be larger than metadata operations but less regular than a pure sequential stream. Add a larger random-read case that approximates sample or shard size. If input reads overlap checkpoint or output writes, run a mixed workload with the application’s approximate read/write ratio. Report both aggregate throughput and the separate read and write results when the tool provides them.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Sweep queue depth and concurrency

Do not publish one queue depth as the drive’s capability. Run a low-concurrency condition, then increase job count and queue depth in steps. Low concurrency may fail to exercise a fast SSD; very high concurrency can increase tail latency and alter application behavior. A sweep might include queue depths of 1, 4, 16 and 32 with one, four and eight jobs, but the exact matrix should reflect the target application and be stated explicitly.

Rank #3
Sale
Samsung SSD 9100 PRO 2TB, PCIe 5.0x4 M.2 2280, Up to 14,700MB/s | NVMe Internal Solid State Hard Drive best for AI Computing, Gaming, and Heavy Duty Workstations (MZ VAP2T0B/AM)
  • BREAKTHROUGH PCIe 5.0 PERFORMANCE: Supercharge your workflow and gaming with PCIe 5.0, boasting up to 14,700/13,400 MB/s¹ sequential read/write speeds. Tackle massive files and power up your gaming with Gen5—twice as fast as the 990 PRO SSD.
  • EVERY TASK, TURBOCHARGED: Speed past productivity limits. With random read/write speeds up to 1,850K/2,600K IOPS¹, enjoy fast game loads, seamless AI apps, and efficient multitasking. Virtually no lag, no limits—just nonstop performance.
  • THINK FAST, CREATE FASTER: With random read/write speeds of up to 1,850K/2,600K IOPS¹, the 9100 PRO SSD fuels seamless AI content creation, swift loads, and smooth gameplay. Work, play, and create at lightning speed.
  • SPEED, WHENEVER YOU NEED: From laptops, desktop PCs, to PS5 with added heatsink devices², experience blazing PCIe 5.0 speeds and up to 8TB of storage. Perfect for video editing, gaming, and creative tasks, with the compatibility to match your device.
  • STAY COOL, RUN FAST: Push limits, not temperatures. A 5nm controller boosts power efficiency up to 49% over the 990 PRO SSD³, while advanced thermal control keeps performance smooth and reliable.

Run long enough to expose sustained behavior

Short bursts can measure a controller or host cache rather than the NAND media. Use a time-based run long enough to pass the expected burst phase, then inspect throughput and latency over time. Note any thermal throttling, garbage-collection dip or change after the drive fills. A single average over a short run can conceal the behavior that determines a long training or checkpoint job.

Separate cold-cache and warm-cache results

Cache handling is one of the easiest ways to produce an impressive but misleading number. Publish cold and warm results as different measurements.

Cold-cache procedure

  1. Use a test dataset larger than the memory available to the client processes.
  2. State whether the benchmark uses direct I/O. The current MLPerf Storage documentation points to NVMe testing with O_DIRECT to bypass the page cache.
  3. If direct I/O is not used, document exactly how the page cache was cleared between runs and whether the operating system permitted the filesystem to retain data.
  4. Run enough passes to show whether the first pass differs from later passes, and label the first pass as cold only when the cache procedure supports that claim.

Warm-cache procedure

For a warm test, deliberately load the working set into the intended cache, then measure the application with that cache populated. State the cache size, warm-up operation and eviction policy. Warm results answer a different question: how fast the system serves data after caching, not how fast the SSD supplies data from media.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Add an end-to-end AI workload test

After the baseline, run the real framework or a recognized storage workload. The goal is to determine whether the complete path supplies data at the rate the job needs and whether accelerators remain productive.

Training data-loading test

  1. Use the production dataset or a representative copy with the same file and sample-size distribution.
  2. Match shuffling, random access, batch size, worker count, preprocessing and prefetch settings.
  3. Sweep data-loader concurrency, recording storage throughput, batch-delivery time and accelerator utilization.
  4. Repeat with cold and warm conditions when the deployment can encounter either state.
  5. Report stalls or idle periods rather than converting them into a single storage-bandwidth figure.

MLPerf Storage is designed to measure how fast storage systems supply training data while a model is being trained. Its workload-specific rules and submission details should be read before comparing a result with another submission.

Checkpoint save and restore test

  1. Start from a stable training state and save a checkpoint of the production size.
  2. Measure application-visible pause time, sustained write rate, synchronization time and the final completion time.
  3. Restore the checkpoint into a clean or controlled cache state and measure time to resume useful training.
  4. Repeat at the expected concurrency if several jobs checkpoint to the same storage system.

Report save and restore independently. A drive can have strong write bandwidth but poor restore latency, or vice versa.

Rank #4
Sale
Samsung SSD 990 PRO 1TB, PCIe 4.0 M.2 2280, Up to 7,450 MB/s | NVMe Internal Solid State Hard Drive for High End Computing, Gaming, and Heavy Duty Workstations (MZ-V9P1T0B/AM)
  • HUGE SPEED BOOST: Get random read/write speeds that are 40%/55% faster than 980 PRO; Experience up to 1400K/1550K IOPS, while sequential read/write speeds up to 7,450/6,900 MB/s reach near the max performance of PCIe 4.0*
  • BREAKTHROUGH POWER EFFICIENCY: Use less power and get more performance; Enjoy up to 50% improved performance per watt over 980 PRO, plus optimal power efficiency with max PCIe 4.0 performance**
  • SMART THERMAL CONTROL: Samsung's own nickel-coated controller delivers effective thermal control; With its slim size, 990 PRO is a perfect fit for desktops and laptops that meet the PCI-SIG D8 standard***
  • THE CHAMPION MAKER: Up to 65% improvement in random performance enables faster loads for an ultimate gaming experience on PS5 and DirectStorage PC games****
  • SAMSUNG MAGICIAN SOFTWARE: Get the most out of your SSD with Samsung Magician's advanced yet intuitive optimization tools; Monitor drive health, protect valuable data, and receive important updates for your 990 PRO

Serving and KV-cache test

Load the model, then exercise the serving path at representative request concurrency and sequence lengths. If the KV cache is stored on the SSD, include its read/write pattern and measure p95 and p99 read latency as well as throughput. A high average rate with unacceptable tail latency can still produce timeouts or uneven response times.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Vector-database test

Populate the intended index, warm it according to the deployment policy, and run representative queries at several concurrency levels. Record query throughput, p95 and p99 latency, index-build or compaction behavior, and the storage mode. Do not infer these values from a sequential SSD result.

Metrics that explain an AI result

Metric What it tells you Where it matters
Sequential throughput Rate for large contiguous transfers Model loading, dataset staging and checkpoint streams
Random IOPS Number of small operations completed per second Random sample access, metadata and cache activity
Average latency Typical operation completion time Baseline responsiveness under a stated queue depth
Tail latency (p95/p99) Slow operations under concurrency Serving, retrieval and any workload sensitive to stalls
Sustained write rate Performance after burst resources are exhausted Long checkpoints, ingestion and index building
End-to-end throughput Rate delivered through the complete software and hardware path Training and serving decisions
Accelerator utilization Whether GPUs or other accelerators are being fed continuously Detecting a storage or data-loader bottleneck

Always attach queue depth, job count, block size, read/write mix and cache state to a metric. “Latency” without those conditions is not a portable result.

Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Support on Ko-Fi

How to compare SSDs fairly

Compare only runs with equivalent workload, platform path, test size, cache state and software settings. Useful comparison axes are:

  • Sequential bandwidth for large transfers.
  • Small and random IOPS plus latency for sample and metadata-heavy work.
  • Tail latency at the concurrency the application uses.
  • Sustained write behavior and restore performance for checkpoint workflows.
  • End-to-end throughput and accelerator utilization under the target AI job.
  • Capacity, interface compatibility, cooling and the constraints of the host or shared-storage architecture.

A PCIe NVMe SSD is a sensible physical category for local dataset staging, but an exact model cannot be selected from advertised speed alone. Interface compatibility, capacity, cooling and whether the target is a local device or a larger storage system all change the decision.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Why a fast SSD benchmark can still produce slow AI loading

The test measured cache

Check file size, direct-I/O use, cache-clearing steps and whether repeated passes were served from memory. Re-run with a larger dataset and a documented cold-cache procedure.

Best Value
Sandisk Optimus 5100 1TB NVMe SSD, PCIe 4.0, M.2 2280
  • SPEED UP PROJECTS. Launch creator applications fast with uncompromising PCIe 4.0 read speeds up to 7,100MB/s,[2] (1TB and 2TB[1] models) and write speeds up to 6,700MB/s[2] (1TB[1]-4TB[1] models).
  • CREATE AND STORE MORE. Make more room for your 4K videos and high-resolution images with capacities from 500GB[1] up to 4TB[1] on M.2 2280 built with our trusted 8th generation SANDISK BiCS QLC 3D CBA NAND.
  • IT GOES WHERE YOU GO. With an all-new power efficient design, your drive delivers high performance with low power, giving you more time to be productive while on the go.
  • UNCOMPROMISED RELIABILITY. With up to 1,200 TBW[3] (4TB[1] model) endurance rating, your drive is designed for creators.
  • KEEP YOUR DRIVE UPDATED. Monitor your SSD’s performance and check for updates with the downloadable SANDISK Dashboard application.[5]

The CPU or data loader is saturated

Profile decoding, decompression, tokenization, augmentation and worker queues. If the storage queue is empty while CPU workers are busy, replacing the SSD will not fix the bottleneck.

Filesystem and metadata work dominates

Millions of small files can make directory and metadata operations more important than bulk bandwidth. Test the real file layout, consider sharded formats where appropriate, and measure random I/O and metadata behavior separately.

The PCIe or network path is limiting throughput

Verify negotiated link width and generation, NUMA placement, switch topology and shared-storage network utilization. A local-drive result cannot be used as the expected rate for a networked path.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

The workload is not storage-bound

Model and dataset characteristics determine demand. Cache-friendly language-model cases and large uncached image or video cases can require very different storage rates. Compare accelerator idle time and data-loader wait time before assigning blame to the SSD.

Thermal or sustained-write behavior changed the result

Log temperature and throughput over the complete run. A short benchmark may capture only the burst phase, while a checkpoint or index build runs long enough to trigger throttling or background media management.

Use published AI-storage figures in context

MLPerf Storage v3.0 includes distinct training, KV-cache and vector-database workloads. Its results guidance emphasizes that comparisons are meaningful within the same workload, not across different workloads. Review the complete submission and configuration before drawing a ranking.

NVIDIA’s DGX SuperPOD H200 storage architecture gives example read and write targets for named workload classes at particular node and system configurations. Those are architecture-specific system figures, not universal pass/fail targets for one SSD. Do not convert a system-level number directly into an individual-drive specification.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

No single published figure establishes that an SSD is “fast enough for AI.” The correct target comes from the workload, system path, concurrency, cache policy and required accelerator utilization.

Publishable benchmark checklist

  • Workload purpose: training, fine-tuning, checkpointing, serving, KV-cache or vector retrieval.
  • Storage topology: local device, filesystem, RAID or volume, or shared storage.
  • Dataset size, file and sample distribution, and whether it exceeds client memory.
  • SSD model, capacity, firmware, interface and link state.
  • CPU, memory, operating system, kernel, filesystem and mount configuration.
  • Benchmark and framework versions, exact configuration and I/O engine.
  • Block size, read/write mix, queue depth, job count, test size and runtime.
  • Cold versus warm state, direct-I/O setting and cache-clearing procedure.
  • Throughput, IOPS, average latency and tail latency with their test conditions.
  • Sustained behavior, temperature and any throttling or burst-to-steady transition.
  • End-to-end throughput, accelerator utilization and application-visible stall time.
  • Comparison limited to equivalent workloads and configurations.

A practical decision rule

Choose the SSD or storage system that meets the end-to-end requirement with acceptable tail latency and sustained behavior, then verify that it fits the interface, capacity and cooling envelope. Use synthetic results to diagnose the storage layer; use the actual AI workload to decide whether the system is fast enough. If a drive wins only on advertised sequential bandwidth but loses on the workload’s IOPS, tail latency or sustained phase, it is not the better choice for that job.

Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.

AI infrastructurefioMLPerf StorageNVMeSSDsstorage benchmarking

Leave a Reply

Your email address will not be published. Required fields are marked *

What’s actually slowing this PC down?

Pick the symptom - the matching free tool is one click away.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Recommended PC Tool
Recommended PC Tool
Windows Errors? Fix Them Before They SpreadFree repair scan
Outdated Drivers Are Slowing You DownFree scan - exact matches

Two free Windows tools

One Free Minute Could Fix That PC

Before you go - each of these free tools takes about a minute and tackles what quietly slows a Windows PC down.

Special offer. View Outbyte info, uninstall instructions, EULA, and Privacy Policy.