Yes—but an SSD does not become extra VRAM. An inference system can keep some model weights, expert blocks, activations, or the key-value (KV) cache on an NVMe SSD and stream the data through system RAM into GPU memory when needed. The CPU and GPU still execute the neural-network operations. This arrangement can make a model larger than available VRAM runnable, but SSD latency is far higher than GPU-memory latency, so speed depends on the model architecture, access pattern, caching, concurrency, and how well data transfers overlap with computation.
The practical target is a large, fast PCIe 5.0 NVMe SSD used as the backing tier in a VRAM–RAM–SSD hierarchy. Sparse mixture-of-experts (MoE) models benefit most because each token activates only a subset of experts. Dense models usually move much more data and can become impractically slow.
As an Amazon Associate I earn from qualifying purchases.
What “running from an SSD” actually means
A GPU cannot execute instructions directly from an ordinary consumer SSD. The drive stores bytes; the CPU and GPU perform matrix multiplication, attention, normalization, and other model operations. During inference, software decides which data is needed, reads it from NVMe, places it in host RAM or pinned buffers, and copies it to VRAM. The GPU then computes on that data.
Crashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minuteWindows Errors? Fix Them Before They Spread
Repair common Windows errors and clear accumulated junk for a smoother, more stable PC - no reinstall needed.Free scan · no reinstallThis is therefore offloaded inference, not a replacement for GPU memory or GPU compute. The SSD supplies capacity, while VRAM remains the fastest working space. If the next layer, expert, or cache block is not already resident, the inference pipeline must wait for a storage read unless it has prefetched the data early enough.
#1 Best Overall
- Read Before You Buy — No Video Output: These adapters support charging and USB 2.0 data transfer, but cannot transmit video signals. Except for standard USB webcams (which use USB data only), they are not compatible with HDMI/DisplayPort cables, video-capable USB-C hubs, or docking stations with video output.
- Convert USB-A Ports to USB-C: Designed to connect USB-C earphones, cables, flash drives, card readers, and other USB-C accessories to standard USB-A ports. Plug-and-play with no drivers or software required.
- Aluminum Alloy Housing: Built with a sturdy aluminum alloy shell that aids in heat dissipation and protects against daily wear and scratches. Designed to maintain a stable and secure connection.
- Compact & Travel-Friendly: The ultra-compact design allows the adapter to stay plugged into your device without blocking adjacent ports or adding bulk, reducing wear and tear on your original USB ports.
- 12-Month Warranty: Backed by a 12-month manufacturer warranty for peace of mind. Designed to meet strict quality control standards for reliable everyday performance.
How the storage-backed pipeline works
- Partition the model state. The runtime keeps frequently reused tensors in VRAM, less-active data in RAM, and the complete backing set on SSD.
- Predict the next requirement. For a dense model this may be the next layer’s weights; for an MoE model it is usually the experts selected by the router. For long-context generation it may be a KV-cache block.
- Issue reads early and concurrently. Large batches of outstanding requests hide NVMe latency better than one small random read at a time.
- Transfer through host memory. Data is read into RAM, often in pinned buffers, and copied asynchronously to VRAM while the GPU works on the previous operation.
- Compute and evict. The GPU executes the operation, then the runtime reuses or evicts the buffer according to its cache policy. Modified cache data can be written back to SSD when required.
Naive memory mapping or operating-system paging can turn this pipeline into serialized page faults and fragmented reads. Systems designed for offload use explicit, asynchronous I/O and layouts that match the unit of computation.
Which models benefit most?
| Model type | What must be fetched | Why it matters | Practical outlook |
|---|---|---|---|
| Dense transformer | Most layers’ weights are used for every token | Continual movement of a large fraction of the model can dominate generation time | Possible when latency is acceptable, but usually much slower than a model that fits in VRAM |
| Sparse MoE | Only the routed experts for each token, plus shared layers | Unused experts can remain on SSD; active experts are cached in RAM or VRAM | Strongest fit for SSD backing because the working set per token is smaller than total capacity |
| Long-context serving | KV-cache blocks grow with sequence length and concurrent sessions | Cache capacity, not just model weights, can exceed GPU memory | SSD or a shared context tier can extend capacity when reuse and prefetching are effective |
Architecture alone does not guarantee a good result. A sparse model with random, tiny expert reads can still stall, while a dense model with highly reusable layers and aggressive caching may be serviceable at low throughput.
What can be offloaded?
Model weights
Weight offload stores some or all parameter tensors outside VRAM. A layer-by-layer schedule can retain a few hot layers on the GPU and stream the rest. Quantization reduces the bytes transferred, but scales, metadata, and dequantization buffers still consume memory. Weight offload is easiest to reason about because weights are read-only during inference.
Recommended Free Tools
Rank #2
- 5-in-1 USB-C Hub: Experience comprehensive connectivity featuring a Power Delivery input, two USB-A 2.0 ports, a USB-A 3.0 port, and an HDMI port. (Note: The USB-C power delivery input port is only for connecting an external wall charger to power your laptop and cannot power peripheral devices.)
- 90W Pass-Through Charging: Achieve optimal charging with 90W pass-through power to your laptop, supported by a total input of 100W, with the hub reserving 10W for operational efficiency. (Note: Wall charger not included.)
- Quick Data Transfers: Accelerate your productivity with rapid data transfers using a high-speed 5Gbps USB 3.0 port and two 480Mbps USB 2.0 ports.
- 4K HDMI Display: Enhance your visual experience with a hub capable of delivering 4K resolution at 30Hz in both mirror and extend modes. Please note that this hub is compatible with MacBook (macOS 12 and newer), Windows 10 and 11, ChromeOS, and laptops equipped with DP Alt Mode and Power Delivery. Note: This device is not compatible with Linux.
- What You Get: Anker USB-C Hub (5-in-1, 4K HDMI), welcome guide, 18-month warranty, and our friendly customer service.
MoE expert blocks
Expert-granularity storage is particularly effective. The router identifies the experts for a token, and the runtime fetches those blocks while leaving inactive experts on SSD. SSD-LLaMA describes this arrangement as an SSD-backed complete expert pool, with RAM and VRAM acting as caches and CPU–GPU hybrid execution.
KV cache
The KV cache records attention keys and values for previous tokens. Moving older or less frequently used blocks to SSD can support longer contexts or more simultaneous sessions. IBM Research’s CHEOPS evaluation measured average KV-cache reads of 2.0 GiB/s and writes of 11.0 MiB/s in its tested traces, showing that reads and writes can have very different demands. Cache offload also requires careful bookkeeping: a block must be returned before attention needs it, or generation pauses.
Activations and temporary tensors
Activations are usually short-lived and less attractive to offload because they must be moved back quickly and may be produced and consumed repeatedly. They can be spilled when memory pressure is severe, but the extra transfers often cost more than reducing batch size or context length.
Rank #3
- Sleek 7-in-1 USB-C Hub: Features an HDMI port, two USB-A 3.0 ports, and a USB-C data port, each providing 5Gbps transfer speeds. It also includes a USB-C PD input port for charging up to 100W and dual SD and TF card slots, all in a compact design.
- Flawless 4K@60Hz Video with HDMI: Delivers exceptional clarity and smoothness with its 4K@60Hz HDMI port, making it ideal for high-definition presentations and entertainment. (Note: Only the HDMI port supports video projection; the USB-C port is for data transfer only.)
- Double Up on Efficiency: The two USB-A 3.0 ports and a USB-C port support a fast 5Gbps data rate, significantly boosting your transfer speeds and improving productivity.
- Fast and Reliable 85W Charging: Offers high-capacity, speedy charging for laptops up to 85W, so you spend less time tethered to an outlet and more time being productive.
- What You Get: Anker USB-C Hub (7-in-1), welcome guide, 18-month warranty, and our friendly customer service.
What published results show
The strongest public evidence comes from systems research rather than ordinary desktop software.
| Result | Conditions and qualification | What it demonstrates |
|---|---|---|
| 128 KiB-dominated model-offload reads | IBM Research CHEOPS traces, 30 March 2025; tested with DeepSpeed and FlexGen | Model offload can generate many medium-sized reads, and the tested workloads did not saturate the NVMe SSD |
| 2.0 GiB/s KV-cache reads; 11.0 MiB/s writes | IBM Research CHEOPS, 2025 averages for its evaluated traces | Read-heavy cache traffic and write traffic have different profiles; one bandwidth number cannot describe the whole workload |
| More than 1 token/s | SSD-LLaMA, 2026; trillion-parameter MoE model, one RTX 5090, no more than 32 GB RAM | An SSD-backed expert pool can make an extremely large sparse model executable on a single GPU in a research system |
| Up to 14.8 GB/s sequential bandwidth | PCIe 5.0 SSD capability cited by SSD-LLaMA authors in 2026; peak sequential figure | High-capacity consumer NVMe can supply substantial throughput, but peak sequential bandwidth is not an inference guarantee |
The more-than-one-token-per-second result is specific to the SSD-LLaMA design, model, GPU, RAM limit, and software implementation. It should not be generalized to arbitrary dense models, different context lengths, or a single consumer drive using ordinary paging.
Hardware requirements for a workable system
- NVMe interface: A PCIe 5.0 SSD is the relevant high-end consumer category when the platform supports it. A PCIe 4.0 drive may work, but has less peak bandwidth headroom.
- Capacity: Choose enough space for the complete model, quantization files, tokenizer, runtime overhead, and multiple versions. An 8 TB drive is a practical scale for keeping several large checkpoints or a very large expert pool, not a promise of a particular model size.
- Sustained performance: Look beyond the advertised sequential maximum. Inference may issue concurrent reads of many moderate or small blocks, so queue-depth behavior and steady-state throughput matter.
- Thermals: PCIe 5.0 drives can throttle when sustained transfers heat the controller. Use the motherboard heatsink or a properly designed cooler and monitor temperatures during long runs.
- RAM: System memory is the intermediate cache. More RAM reduces repeat SSD reads and gives the runtime room for pinned transfer buffers; SSD capacity alone cannot compensate for a very small RAM pool.
- GPU transfer path: A platform and runtime that support asynchronous host-to-device copies are important. Direct-I/O or GPUDirect Storage-style paths can reduce copies in supported configurations, but support depends on the operating system, driver, filesystem, and application.
Software design choices that determine speed
Filesystem paging versus direct I/O
Standard memory mapping is convenient, but a page fault may fetch one small, scattered block and block the thread that requested it. Explicit direct I/O, large aligned requests, and a queue of asynchronous operations give the runtime control over ordering and concurrency. Storage Performance Development Kit (SPDK) and GPUDirect Storage are examples of lower-overhead approaches; neither automatically optimizes a model without an application designed to use it.
Rank #4
- Dual Converters, Infinite Potential：Includes 2× USB C male to USB A female adapters and 2× USB A male to USB C female adapters. Perfect for a wide range of uses—tablets with Bluetooth keyboards, expand USB ports on macbook, and more. Two different converters for all your daily needs
- Next-Level 10Gbps & 3A Charging: No more slow 480Mbps, this usb to usb c adapter has a transfer speed of up to 10Gbps, allowing you to do more transferring in less time. This usb adapter fits both USB A and USB C charger, supporting up to 3A fast charging
- Upgraded Exquisite Craftsmanship: With an aluminum alloy housing and metal connector, the usbc to usb adapter is extremely durable and sturdy. Rigorously tested to withstand more than 10,000 times of plugging and unplugging, ensuring long-lasting performance
- Broad Compatible: The usb c to usb adapter widely supports all USB C/ USB A devices like laptops, tablets, cellphones, car chargers, and phone chargers. Such as compatible with MacBook Pro/Air 2023/2022, Thunderbolt 4/3 Devices,Apple MagSafe Watch 9/8/7/SE/Ultra, iPad Pro 2022/2021, Samsung Galaxy S23/S20/S10, and iPhone 17/16/15 Pro. Plug and play
- Please Note: To reach 10Gbps speed, keep the cable under 3.3 ft. For USB A Male to USB C adapters, try flipping the USB C connector. USB C Male to USB A adapters support bidirectional 10Gbps transfer within 3.3 ft
Layout and granularity
Store tensors in units that match computation. For MoE, that generally means contiguous expert blocks rather than thousands of tiny fragments. For KV cache, fixed-size blocks simplify allocation, eviction, and prefetching. Compression and quantization lower traffic, but the runtime must account for decompression time and temporary buffers.
Reuse-aware caching
A useful policy keeps hot shared layers and frequently selected experts in VRAM, a larger working set in RAM, and cold data on SSD. Least-recently-used alone may be insufficient when the router’s selection pattern changes; frequency, recency, and predicted reuse can be combined. Cache statistics should be recorded per layer or expert so you can identify thrashing.
How to evaluate an SSD-backed local model
- Define the target. Record model architecture, parameter format, quantization, context length, batch size, and the minimum acceptable tokens-per-second or time-to-first-token.
- Measure the memory gap. Check peak VRAM use, RAM use, and the total on-disk footprint with the exact runtime. Leave headroom for the operating system, CUDA allocations, and tokenizer or server processes.
- Start with a fitting baseline. Run a smaller or more aggressively quantized version entirely in VRAM. This establishes GPU-only latency for comparison.
- Enable one offload target. Test weight, expert, or KV-cache offload separately before combining them. Otherwise you cannot tell which tier causes stalls.
- Use asynchronous prefetching. Configure enough concurrent reads to keep the SSD busy while the GPU computes, but avoid queue depths that exhaust RAM or cause thermal throttling.
- Inspect traces. Look for read sizes, queue depth, cache hit rate, GPU idle periods, and PCIe link utilization. A low SSD utilization number with a stalled GPU often indicates serialized software or tiny requests, not a slow NAND device.
- Test realistic sessions. Repeat measurements at the intended context length and number of concurrent users. KV-cache pressure and expert locality change as sessions grow.
- Record sustained results. Report tokens per second and latency after the drive reaches steady temperature, not only the first few requests served from an empty or unusually favorable cache.
Common bottlenecks and remedies
| Symptom | Likely cause | What to change |
|---|---|---|
| GPU utilization repeatedly drops to zero | Data arrives after computation requests it | Prefetch earlier, increase concurrent reads, and overlap copies with kernel execution |
| SSD shows low utilization but generation is slow | Fragmented page faults or a single synchronous reader | Use contiguous tensor layout, aligned direct I/O, and multiple outstanding requests |
| Performance collapses as context grows | KV cache exceeds RAM/VRAM and is repeatedly fetched | Use block-aware KV caching, reduce context or concurrency, or add a faster cache tier |
| Throughput declines during long runs | SSD thermal throttling or cache thrashing | Improve cooling, monitor temperature, and tune eviction and prefetch policies |
| MoE model is still slow despite sparse routing | Experts are scattered or router choices have little locality | Pack each expert contiguously, retain popular experts in RAM/VRAM, and batch requests when latency goals allow |
| System runs out of RAM | Too many pinned buffers or an oversized host cache | Cap cache size, limit in-flight reads, and leave operating-system headroom |
Computational Storage is a different category
NVM Express defines Computational Storage as NVMe devices executing programs on data resident in the NVM subsystem. Its Computational Programs and Subsystem Local Memory command sets are intended to reduce host data movement by processing data near where it is stored. This is distinct from an ordinary consumer SSD, which primarily stores and transfers bytes; buying a faster drive does not make it an LLM accelerator.
Best Value
- 5-in-1 Connectivity: Equipped with a 4K HDMI port, a 5 Gbps USB-C data port, two 5 Gbps USB-A ports, and a USB C 100W PD-IN port. Note: The USB C 100W PD-IN port supports only charging and does not support data transfer devices such as headphones or speakers.
- Powerful Pass-Through Charging: Supports up to 85W pass-through charging so you can power up your laptop while you use the hub. Note: Pass-through charging requires a charger (not included). Note: To achieve full power for iPad, we recommend using a 45W wall charger.
- Transfer Files in Seconds: Move files to and from your laptop at speeds of up to 5 Gbps via the USB-C and USB-A data ports. Note: The USB C 5Gbps Data port does not support video output.
- HD Display: Connect to the HDMI port to stream or mirror content to an external monitor in resolutions of up to 4K@30Hz. Note: The USB-C ports do not support video output.
- What You Get: Anker 332 USB-C Hub (5-in-1), welcome guide, our worry-free 18-month warranty, and friendly customer service.
Specialized research such as AiF explores in-flash processing and dynamic parameter loading for models larger than host memory. These architectures are experimental, require specialized hardware and software, and should not be confused with a retail NVMe feature.
Where enterprise context tiers fit
NVIDIA CMX uses BlueField-4 infrastructure to manage NVMe SSDs as a shared pod-level context tier for ephemeral KV cache. It is aimed at long-context, multi-turn, and agentic inference, where sessions can reuse context across GPU resources. CMX extends effective capacity for that serving scenario; it is not marketed as a consumer SSD that executes arbitrary LLM layers or replaces GPUs.
When an SSD-backed model is a sensible choice
- Choose this approach when the model is valuable enough to run locally, exceeds available VRAM, and you can accept lower throughput or higher latency.
- Prefer sparse MoE architectures when you can choose the model, because selective expert access reduces bytes moved per token.
- Use it for experimentation, batch jobs, or low-concurrency services before targeting interactive, high-concurrency production.
- Do not expect an SSD to make a model compute-bound at GPU speed. If every token requires fetching most of a dense model, adding SSD capacity may only produce a slower version of the same workload.
- Consider splitting the model across multiple GPUs, reducing quantization size, shortening context, or using a smaller model before investing in a more complex storage pipeline.
The accurate mental model is a three-tier memory system: VRAM holds the hottest tensors, RAM caches the active working set, and SSD stores the capacity tier. With expert-aware layout, concurrent direct reads, asynchronous transfers, compression, and reuse-aware caching, that hierarchy can run models larger than GPU memory. Without those software techniques, an SSD is merely a large, relatively slow file from which the GPU must wait for data.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
