Choose storage for large language model (LLM) inference only after separating persistent model files from the memory an inference engine needs while serving requests. Model weights and active key-value (KV) cache principally consume GPU memory; host RAM can be an offload tier in supported runtimes, and persistent storage holds checkpoint files or may serve as a slower cache tier in some configurations. An SSD alone does not replace GPU memory or guarantee faster token generation.
First, distinguish storage from inference memory
The word “storage” can refer to several different resources in an inference system. They hold different data, have different roles, and are not interchangeable.
As an Amazon Associate I earn from qualifying purchases.
| Tier | Typical role in inference | What to check |
|---|---|---|
| GPU memory (VRAM) | Holds model weights and active inference state, including KV cache; other allocations may include activations and runtime buffers. | Whether the deployed model and intended workload fit with enough room for runtime allocations. |
| Host memory (system RAM) | May act as a primary offload tier in a runtime that supports it. | How the engine uses host memory, how much headroom remains for the operating system and other processes, and the cost of moving data between tiers. |
| Persistent storage | Holds checkpoint files used to load model weights. In some supported configurations, it can also hold secondary cache data. | Capacity, read/write behavior under the engine’s access pattern, and explicit support for the intended offload feature. |
Checkpoint storage affects where model files live and can affect loading. It does not by itself determine how much active model state fits in VRAM or how quickly each token is generated.
Do these 3 things before closing this tab:
1Clear out junk files and repair common Windows errors2Fix the driver behind crashes, sound loss and screen glitches3Repair Windows errors before they cause bigger problemsEstimate weight memory, then leave room for runtime state
A useful first-pass estimate for weight memory per GPU is:
#1 Best Overall
- MEET THE NEXT GEN: Consider this a cheat code; Our Samsung 990 PRO Gen4 SSD helps you reach near max performance with lightning-fast speeds; Whether you’re a hardcore gamer or a tech guru, you’ll get power efficiency built for the final boss
- REACH THE NEXT LEVEL: Gen4 steps up with faster transfer speeds and high-performance bandwidth; With a more than 55% improvement in random performance compared to 980 PRO, it’s here for heavy computing and faster loading
- THE FASTEST SSD FROM THE WORLD'S FLASH MEMORY BRAND: The speed you need for any occasion; With read and write speeds up to 7450/6900 MB/s you’ll reach near max performance of PCIe 4.0 powering through for any use
- PLAY WITHOUT LIMITS: Give yourself some space with storage capacities from 1TB to 4TB; Sync all your saves and reign supreme in gaming, video editing, data analysis and more
- IT’S A POWER MOVE: Save the power for your performance; Get power efficiency all while experiencing up to 50% improved performance per watt over the 980 PRO; It makes every move more effective with less consumption
parameter count × bytes per parameter ÷ tensor-parallel degree
The estimate depends on the model’s stored precision or quantization and how weights are distributed across GPUs. NVIDIA NIM’s guidance gives these approximate bytes-per-parameter values:
| Weight format | Approximate bytes per parameter |
|---|---|
| BF16 or FP16 | 2 bytes |
| FP8 | 1 byte |
| INT4 or NVFP4 | 0.5 bytes |
These are weight-sizing heuristics, not total VRAM requirements. NVIDIA’s current NIM guidance estimates Llama 3.1 8B in BF16 at 16 GB of weights on one GPU, and Llama 3.3 70B in BF16 at 35 GB per GPU across four GPUs using tensor parallelism. Those documentation examples are not guarantees that a particular card or serving configuration will fit the model: KV cache, activations, communication buffers, CUDA graphs, runtime overhead, and other allocations also need space.
Rank #2
- Ideal for high speed, low power storage
- Gen 4x4 NVMe PCle performance
- Up to 6,000MB/s read, 4,000MB/s write
- Includes Acronis cloning software
- 5-year limited warranty
TensorRT-LLM documentation describes engine-specific memory allocations, including activations and I/O tensors. Actual requirements depend on the model, engine, runtime version, and request settings. Treat the formula as a screening calculation, then confirm with the exact deployment’s configuration and startup behavior.
Size for KV cache, context length, and concurrency
KV cache stores attention state from earlier tokens so decoding can reuse it instead of recomputing that state. Its memory demand grows with sequence length and batch size. A model that fits when serving short prompts or few simultaneous requests may run out of VRAM under longer contexts or greater concurrency.
NVIDIA Developer’s 2023 inference example estimates roughly 14 GB for Llama 2 7B weights at 16-bit precision and about 2 GB of KV cache for batch size one with a 4096-token sequence. These are illustrative figures for that model and workload, not universal sizing rules.
Rank #3
- SPEED UP PROJECTS. Launch creator applications fast with uncompromising PCIe 4.0 read speeds up to 7,100MB/s,[2] (1TB and 2TB[1] models) and write speeds up to 6,700MB/s[2] (1TB[1]-4TB[1] models).
- CREATE AND STORE MORE. Make more room for your 4K videos and high-resolution images with capacities from 500GB[1] up to 4TB[1] on M.2 2280 built with our trusted 8th generation SANDISK BiCS QLC 3D CBA NAND.
- IT GOES WHERE YOU GO. With an all-new power efficient design, your drive delivers high performance with low power, giving you more time to be productive while on the go.
- UNCOMPROMISED RELIABILITY. With up to 1,200 TBW[3] (4TB[1] model) endurance rating, your drive is designed for creators.
- KEEP YOUR DRIVE UPDATED. Monitor your SSD’s performance and check for updates with the downloadable SANDISK Dashboard application.[5]
Account for the actual serving workload, not only the model’s parameter count:
- Maximum and typical prompt-plus-generation sequence lengths.
- Number of active requests or batch size the service must support.
- Model precision, quantization, and parallelism across devices.
- Runtime allocations beyond weights and KV cache, including activations and buffers.
- Headroom for variation in request mix and for the deployed engine’s own memory use.
Cache allocation behavior is also engine-specific. TensorRT-LLM documents paged KV-cache allocation based on configuration; when explicit limits are absent, its documented behavior uses remaining free GPU memory. Check the documentation and startup logs for the exact runtime version rather than assuming another engine’s defaults apply.
Understand what offloading can—and cannot—solve
Offloading can extend the memory hierarchy, but it introduces software and data-transfer constraints. In its KV Offloading Usage Guide, vLLM describes a CPU-only offloading tier and a tiered configuration with CPU primary memory and optional secondary tiers. Completed KV blocks may be placed in larger, slower tiers and promoted back to GPU when needed. In that tiered path, transfers between GPU and secondary tiers stage through the CPU: vLLM states, “Only the CPU primary tier has direct GPU access.”
Rank #4
- HUGE SPEED BOOST: Get random read/write speeds that are 40%/55% faster than 980 PRO; Experience up to 1400K/1550K IOPS, while sequential read/write speeds up to 7,450/6,900 MB/s reach near the max performance of PCIe 4.0*
- BREAKTHROUGH POWER EFFICIENCY: Use less power and get more performance; Enjoy up to 50% improved performance per watt over 980 PRO, plus optimal power efficiency with max PCIe 4.0 performance**
- SMART THERMAL CONTROL: Samsung's own nickel-coated controller delivers effective thermal control; With its slim size, 990 PRO is a perfect fit for desktops and laptops that meet the PCI-SIG D8 standard***
- THE CHAMPION MAKER: Up to 65% improvement in random performance enables faster loads for an ultimate gaming experience on PS5 and DirectStorage PC games****
- SAMSUNG MAGICIAN SOFTWARE: Get the most out of your SSD with Samsung Magician's advanced yet intuitive optimization tools; Monitor drive health, protect valuable data, and receive important updates for your 990 PRO
The guide notes support for CUDA, ROCm, and XPU, but feature availability and configuration are version-sensitive. Confirm that the precise serving-engine version and hardware support the tier and settings you plan to use. Do not assume that adding disk capacity creates an effective offload path.
Capacity is only one part of a useful offload tier
vLLM advises leaving headroom in host memory and, for its single-tier setup, making the CPU tier large enough to be useful relative to aggregate GPU cache capacity. It also recommends tuning filesystem read and write threads to the storage’s sustainable concurrency. The guide notes that reads can be latency-sensitive on the prefill path when cache-hit rates are high.
In practice, usefulness depends on whether requests reuse cached data, the size of each tier, access patterns, I/O parallelism, and the transfer path. A large secondary tier with poor reuse or unsuitable I/O behavior may not help the workload. Measure the actual service path before relying on offload for capacity or performance.
Best Value
- This product has been replaced by our latest generation. Please search for the SANDISK Optimus GX 7100 NVMe SSD
- HIGH-OCTANE GAMING. Experience speeds up to 7,250MB/s read and 6,900MB/s write (1-2TB models), with up to 35% faster performance than previous generation.
- PURPOSE-BUILT. Designed for serious on-the-go gamers, with a PCIe Gen4 interface and SANDISK’s next generation TLC 3D NAND.
- MORE TIME TO CLEAR THAT CHECKPOINT. Built with laptops and handheld gaming devices in mind, with up to 100% more power efficiency over the previous generation.
- DO MORE WITH DASHBOARD. Ensure your drive is optimized for prime performance with the downloadable WD_BLACK Dashboard (Windows only).
Compare the whole system against the service target
There is no universal SSD, RAM, or GPU specification that follows from model size alone. Establish the workload and service target, then evaluate the system as a whole:
| Decision area | Questions to answer |
|---|---|
| Model fit | What is the parameter count and weight format? How are weights split through tensor or pipeline parallelism, and what is the per-device estimate? |
| Active memory | What VRAM is needed for KV cache at target context lengths and concurrency, plus activations, runtime buffers, adapters, and headroom? |
| Memory tier | Does the serving engine support the planned GPU, CPU, and secondary-tier path? Which tier holds each kind of state? |
| Performance | What are prefill and decode latency and throughput at target concurrency? What storage latency and I/O concurrency does the offload path require, and how does data move between tiers? |
| Operations | How often will cached data be reused? What filesystem thread settings, model-loading behavior, version compatibility, and capacity controls are needed? |
| Economics | What is the total system cost for the required workload target? Compare cost against measured capacity and performance; model size alone cannot establish the best-value configuration. |
When an SSD matters
Persistent local storage is relevant for keeping checkpoint files and, where the serving engine explicitly supports it, a secondary cache tier. Choose capacity and I/O characteristics to match those jobs and the engine’s documented configuration. The cited NVIDIA and vLLM documentation does not establish a general rule that a particular SSD interface or product improves token-generation speed, so do not treat SSD selection as a substitute for sizing VRAM, host memory, or the inference workload.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
