There is no single RAM or VRAM number that runs every local AI model. Start with the exact model file you will load—especially its quantization—then add memory for the context window, runtime overhead, your operating system and any parallel requests. System RAM and GPU VRAM are separate pools on most PCs, although runtimes such as llama.cpp and Ollama can split a model between them.
As a rough starting point, Intel guidance says system RAM should be at least as large as the model file, while GPU VRAM should hold the file plus room for other GPU work. That is planning guidance, not a universal minimum: a model file is only the weight storage, and actual inference use changes with context length, cache settings, concurrency and placement.
As an Amazon Associate I earn from qualifying purchases.
What RAM and VRAM do during local inference
System RAM
System memory holds CPU-side inference work, the runtime and operating-system processes. If some model layers are kept off the GPU, those layers also use system RAM. A model can therefore run without a large graphics card, but the CPU has to process more of it and performance depends heavily on the processor, memory bandwidth and runtime.
Free tools Windows power users keep installed
One-click scans. No signup required.
GPU VRAM
VRAM is the memory available to the graphics processor for model weights, activations, the key/value (K/V) cache and other GPU tasks. A card’s advertised capacity is not entirely available to one model: the display driver, desktop, runtime and other applications consume some of it. Intel’s guidance therefore recommends padding beyond the model-file size rather than treating the capacity as a precise limit.
#1 Best Overall
- Requires overclocking/BIOS adjustments. Maximum speed and performance depends on system components, including motherboard and CPU.
- G.SKILL Trident Z5 Neo RGB Series DDR5 U-DIMM Memory Kit, Model: F5-6000J3636F32GX2-TZ5NR
- Non-ECC, DDR5 U-DIMM, 288-pin, for Desktop PC & Gaming
- Includes JEDEC default profile, and AMD EXPO & Intel XMP 3.0 memory overclock profile
- Do not mix memory kits. Memory kits are sold in matched kits that are designed to run together as a set. Mixing memory kits will result in stability issues or system failure.
Why the model-file size is only the starting point
Quantization stores weights with fewer bits and can reduce a download dramatically, but different formats have different size, speed and quality characteristics. The same parameter count can produce substantially different files. Context length adds K/V-cache memory, and serving several requests can multiply that cache. Runtime data structures and temporary buffers also need room. An SSD can store a large file, but it does not increase either RAM or VRAM while the model is running.
Published model sizes: use the exact quantized file
The following Llama 3.1 figures illustrate why parameter count alone is not a sizing method. “Original” is the larger, unquantized file reported in ggml-org’s version-tagged quantization documentation (2026). Q4_K_M is one quantized format from that documentation. Ollama’s library listing reports rounded download sizes for its variants (accessed 2026); those values are not a promise that the complete runtime will fit in exactly that amount of memory.
| Model variant | Original file | Q4_K_M file | Ollama listing | What the number means |
|---|---|---|---|---|
| Llama 3.1 8B | 32.1 GB | 4.9 GB (4.58 GiB in the quantization table) | 4.9 GB | Weight-file/download size; runtime headroom is additional. |
| Llama 3.1 70B | 280.9 GB | 43.1 GB | 43 GB | A single desktop GPU normally cannot provide this much free VRAM, so split or CPU placement may be needed. |
| Llama 3.1 405B | 1,625.1 GB | 249.1 GB | 243 GB | Large-server-scale storage and memory territory even after quantization. |
GB and GiB are different units: the 8B Q4_K_M entry is 4.58 GiB in the quantization table, while other listings use decimal GB. Keep the unit attached to every figure when comparing files. These examples are named model-file sizes, not universal hardware recommendations or benchmark results.
Do these 3 things before closing this tab:
1Repair Windows errors before they cause bigger problems2Scan for outdated or missing drivers - takes under a minute3Clear out junk files and repair common Windows errorsA practical RAM and VRAM sizing procedure
- Choose the exact model and format. Record the family, parameter variant, quantization (for example, Q4_K_M) and the file size shown by the repository or runtime. Do not substitute a parameter-count rule for the actual file.
- Decide where inference will run. Plan for CPU/system RAM, GPU/VRAM or a split. A GPU number matters only if the runtime can use that device and enough of its memory remains free.
- Add headroom. Start with at least the model-file size in system RAM, following Intel’s broad guidance, then reserve additional room for the operating system, runtime buffers, context cache and other applications. For VRAM, reserve space beyond the file for GPU tasks and the cache rather than filling the card to its label capacity.
- Set the context you actually need. A short chat and a long document prompt do not have the same memory demand. Longer context windows require more cache; use the runtime’s default only if it fits your workload.
- Account for concurrency. Interactive single-user use and a service handling several requests have different requirements. Parallel requests can make cache use grow roughly with the number of requests times the context length.
- Load and verify placement. Check the runtime’s status after loading instead of inferring placement from the hardware specification. This catches cases where a model silently fell back to CPU or was split across CPU and GPU.
- Check storage separately. Keep enough SSD capacity for the downloaded file, alternate quantizations and temporary files. Storage capacity affects whether you can keep the model, not whether it fits in inference memory.
If the estimate leaves almost no free memory, choose a smaller quantization, shorter context, fewer simultaneous requests or a smaller model before troubleshooting crashes. The right trade-off depends on whether your priority is response speed, answer quality, model size or the number of users.
Context length and parallel requests can change the answer
Weights are not the only significant allocation. Every token in the active context contributes to the K/V cache, and that allocation persists while the request is being processed. A runtime may therefore load a model successfully with a short prompt and fail when you increase the context window.
Ollama’s defaults and controls
Ollama’s FAQ documents a default context window of 4,096 tokens. It also states that parallel request processing increases the context size by the number of parallel requests. In practical terms, memory pressure follows the product of OLLAMA_NUM_PARALLEL and OLLAMA_CONTEXT_LENGTH for a given model, before accounting for the weights and other allocations. Raising either setting can require more RAM or VRAM.
Ollama also documents Flash Attention and quantized K/V-cache settings as ways to reduce cache memory. These are Ollama-specific controls, and reduced-precision caches involve precision and quality trade-offs that can vary by model and task. Treat them as workload settings, not as a substitute for choosing adequate hardware.
Quick wins for a faster PC:
Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Clear out junk files and repair common Windows errorsFree Scan →Scan for outdated or missing drivers - takes under a minuteDriver Scan →Planning a long-context workload
- Measure the longest prompt and generated response you expect, not just the average chat.
- Leave more headroom when the application accepts documents, code repositories or multiple conversation histories.
- For a service, multiply the cache expectation by the maximum simultaneous requests rather than the number of registered users.
- If memory rises only when context grows, reduce context or concurrency before replacing the model file.
CPU-only, GPU-only and hybrid placement
GPU-only loading
When the weights and runtime allocations fit in free VRAM, the runtime can place the model entirely on the GPU. This generally avoids transferring layers between CPU and GPU, but “fits” means the entire working set fits, not merely the downloaded file.
Rank #2
- Boosts System Performance: 64GB DDR5 RAM laptop memory kit (2x32GB) that operates at 4800MHz to improve multitasking and system responsiveness for smoother performance
- Accelerated gaming performance: Every millisecond gained in fast-paced gameplay counts—power through heavy workloads and benefit from versatile downclocking and higher frame rates
- Optimized DDR5 compatibility: Best for 12th Gen Intel Core and AMD Ryzen 7000 Series processors — Intel XMP 3.0 and AMD EXPO also supported on the same RAM module
- Trusted Micron Quality: Backed by 42 years of memory expertise, this DDR5 RAM is rigorously tested at both component and module levels, ensuring top performance and reliability
- ECC type=non-ECC, Form Factor=SODIMM, Pin count=262-pin, PC speed=PC5-38400, Voltage=1.1V, Rank and Configuration=2Rx8
CPU-only loading
A model can run from system RAM when there is insufficient usable VRAM or no compatible GPU path. Insufficient RAM can force swapping, which Intel warns may make inference extremely slow or cause a crash. The model file, context and runtime still need to fit in memory at the same time.
Split CPU/GPU loading
llama.cpp supports CPU/GPU hybrid inference for models larger than available VRAM. Ollama’s status output similarly distinguishes models loaded entirely on the GPU, entirely in system memory or partially across both. Hybrid placement makes a model larger than your VRAM runnable, but speed and responsiveness depend on how much work remains on the CPU and how often data moves between memory pools. It is a capacity option, not a guarantee of a particular tokens-per-second rate.
How to check Ollama’s actual placement
After loading a model, run ollama ps. Ollama’s FAQ says the Processor column shows which memory the model was loaded into. Use that report to distinguish “100% GPU,” system-memory placement and a CPU/GPU split; the graphics card’s specification alone cannot tell you what happened.
Worked planning examples
An 8B quantized model
The Llama 3.1 8B Q4_K_M example has a 4.9 GB decimal file (4.58 GiB in the quantization table). That figure is a floor for the weights, not a 4.9 GB RAM or VRAM recommendation. A machine must still provide space for the operating system, runtime, context cache and other programs. If a GPU has only a small amount of free VRAM after display use, the runtime may place some or all layers in system RAM even though the file appears smaller than the card’s advertised capacity.
A 70B quantized model
The Q4_K_M file is 43.1 GB in the cited quantization documentation. A single card with less free VRAM cannot hold the entire working set, but hybrid CPU/GPU execution can still make the model usable if system RAM and the runtime support the split. Expect a different performance profile from an all-GPU load, and budget extra memory for context and concurrent requests.
A 405B quantized model
The Q4_K_M example is 249.1 GB, with an Ollama listing of 243 GB. This is beyond the memory capacity of typical consumer systems once headroom is included. Running it requires a platform with sufficient combined memory, a compatible runtime and substantial storage; the file-size example alone cannot establish a practical configuration.
When to upgrade RAM, VRAM or storage
Upgrade system RAM when
- The model is intended to run on the CPU or with CPU offload and the current machine cannot keep the file, context and applications resident.
- The operating system begins swapping during inference or the runtime reports an allocation failure.
- You need more context or parallel requests and GPU memory is not the limiting pool.
Upgrade GPU VRAM when
- The model would otherwise fit on the GPU, but free VRAM is consumed by the display, runtime or other applications.
- You want more layers on the GPU to reduce CPU offload, or you need larger context and concurrency on the GPU.
- The GPU is physically compatible with the system and supported by the chosen runtime; VRAM capacity alone does not guarantee software compatibility.
Buy storage when
- You need room for multiple model families, quantizations, updates and backups.
- Downloads or model switching are constrained by disk capacity or throughput.
An SSD improves model storage and load times, but it cannot substitute for RAM or VRAM during generation. Before buying memory, verify motherboard-supported RAM type and capacity, available slots, GPU power and physical clearance, operating-system support and runtime compatibility. No single amount is universally suitable across platforms.
Troubleshooting memory failures
| Symptom | Likely cause | First change to try |
|---|---|---|
| Model will not load | The file plus runtime allocations exceed the available RAM/VRAM. | Use a smaller quantization or model, close other applications and confirm actual free memory. |
| It loads but is extremely slow | Swapping, CPU fallback or a large CPU/GPU split. | Check placement, reduce context and stop memory-heavy background tasks. |
| Short prompts work; long prompts fail | K/V-cache growth from the larger context. | Lower context length or enable a documented cache-saving option, accepting its trade-offs. |
| One request works; several fail | Parallel cache allocations multiply with concurrency. | Reduce parallel requests or reserve more memory. |
| GPU appears underused | The runtime loaded the model in system RAM or split it. | Inspect the runtime status rather than relying on the GPU’s capacity label. |
The decision rule
For any local model, use this order: exact quantized file size, intended CPU/GPU placement, context length, concurrent requests, then headroom for the operating system and runtime. Treat the published file-size examples as evidence of scale, not as promises that a model will run in exactly that memory. If the complete working set does not fit, reduce the model or workload, add compatible RAM or VRAM, or use a supported hybrid configuration and accept the performance trade-off.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
