Do these 3 things before closing this tab:
1Fix the driver behind crashes, sound loss and screen glitches2Repair Windows errors before they cause bigger problems3Scan for outdated or missing drivers - takes under a minuteSystem RAM and GPU VRAM are not interchangeable on a computer with a discrete graphics card. VRAM is the fast memory directly attached to the GPU and is the main capacity limit for keeping a model fully on the GPU. System RAM serves the operating system, CPU inference and, when the software supports it, model layers that are offloaded from the GPU. More RAM can make an otherwise too-large model load or run, but it does not make CPU work or CPU-to-GPU transfers as fast as GPU-resident inference.
Size the whole serving workload—not just the downloaded model file. Account for weights, context and KV cache, engine overhead, the operating system, other applications and any concurrent requests. The right upgrade depends on whether your priority is fitting a model, generating tokens quickly, serving several users or running a machine with unified memory.
As an Amazon Associate I earn from qualifying purchases.
RAM and VRAM perform different jobs
On a conventional desktop or laptop with a discrete graphics card, the processor and graphics card have separate memory pools. The CPU accesses system RAM; the GPU accesses its own VRAM. NVIDIA’s PAIR guidance describes these as separate resources, so adding their capacities together does not create one large pool of equally fast memory.
Crashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minuteWindows Errors? Fix Them Before They Spread
Repair common Windows errors and clear accumulated junk for a smoother, more stable PC - no reinstall needed.Free scan · no reinstall| Resource | Primary user | What it holds during local inference | What increasing it usually changes |
|---|---|---|---|
| System RAM | CPU and operating system | CPU-hosted model weights or layers, application data, operating-system services and buffers | Whether CPU inference or a mixed placement can fit; how much headroom remains for other software |
| GPU VRAM | Discrete GPU | GPU-resident weights or layers, activations, KV cache and runtime allocations | Whether the model can stay fully on the GPU and how much GPU-side context or concurrency is practical |
| Unified memory | CPU and GPU share one physical memory pool | Both processor types draw from the same capacity, subject to platform and engine support | A different capacity calculation, but not freedom from context, runtime or operating-system overhead |
A large RAM number therefore cannot be treated as a replacement for a small VRAM number. RAM can rescue capacity by hosting work on the CPU, while VRAM is what enables the GPU to process that work without repeatedly crossing the system bus.
#1 Best Overall
- 【High-Performance APU】The MS-S1 MAX features an AMD Ryzen AI Max+ 395 APU, integrating a Zen 5 architecture CPU (up to 5.1GHz, 16C/32T, 64M L3 Cache), an RDNA 3.5 GPU, and an NPU (50 TOPS). The total system output is 126 TOPS. It provides powerful parallel computing capabilities for demanding AI workflows. It is ideal for running local LLMs, multimodal models, and computationally intensive tasks
- 【128GB UMA Memory】Equipped with up to 128GB of LPDDR5x-8000MT/s unified memory, it enables the CPU and GPU to access a shared, high-bandwidth memory pool with extremely low latency. Ideal for large-scale AI inference, 3D workloads, and complex timelines in video editing. It eliminates traditional VRAM bottlenecks, ensuring smoother data transfer during high-intensity computations. The UMA design maximizes performance stability under high loads
- 【Flexible Expansion】The MS-S1 MAX features USB4 V2 (up to 80Gbps), dual 10GbE LAN, HDMI 2.1 (up to 8K60), a full-length PCIe x16 expansion slot, and dual M.2 slots supporting up to 16TB RAID 0/1. Wi-Fi 7 provides stronger signal coverage and a more stable wireless experience. The slide-out design facilitates upgrades and maintenance. It easily adapts to personal, studio, or rack-mount enterprise environments
- 【High-Efficiency Cooling System】Utilizing an aerospace-grade aluminum alloy chassis, copper base plate, six heat pipes, dual turbine fans, and advanced PCM thermal conductive material, it maintains stable cooling performance even under continuous load. This system supports 130W continuous power and 160W peak power operation, with a built-in 320W power supply. It boasts multiple global certifications including CCC, FCC, UL, CE, and UKCA, ensuring stable and reliable operation in various environments
- 【Cluster Design】Two MS-S1 MAX units can be configured as a dual-unit cluster to run a large 235B Q4 model locally, achieving an output speed of 10.87 tok/s. Supporting 2U rack deployment, multiple MS-S1 MAX units can be cascaded into a distributed cluster to create a high-efficiency AI computing center. A cluster of four MS-S1 MAX units successfully ran a DeepSeek-R1 671B Q4 large model. A reserved cluster power-on interface allows for unified start-up and shutdown
What a local model actually needs in memory
The file you download is only one component of the live footprint. NVIDIA states, “A model’s download size is not its total memory requirement.” A serving process needs room for several allocations at once:
- Model weights: the parameters in the selected precision or quantization and the buffers needed to load them.
- Context and KV cache: data retained so the model can attend to the prompt and generated conversation. Longer context consumes more memory.
- Engine runtime overhead: workspace, graph or kernel allocations, tokenizer state and other implementation-specific buffers.
- Parallel requests: each active request can add context and cache allocations. The total rises with concurrency.
- System headroom: the operating system, desktop, drivers, monitoring tools and other applications must continue to fit.
Parameter count alone cannot provide a universal RAM or VRAM answer. Precision or quantization, context length, runtime, batch or parallel settings, model architecture and hardware all affect the result. Treat any capacity estimate that omits those variables as incomplete.
Four ways a runtime can place the model
CPU-only inference
In CPU-only mode, system RAM holds the model and the CPU performs the computation. This can run without a discrete GPU, provided the model and the rest of the workload fit in RAM. It is generally a capacity option rather than a speed advantage when a suitable GPU is available; generation rate depends on the processor, memory bandwidth, model and runtime.
Free tools Windows power users keep installed
One-click scans. No signup required.
Fully GPU-resident inference
When the relevant weights and runtime allocations fit in one GPU’s VRAM, a runtime can keep the model on that card. This avoids CPU-to-GPU transfers for the model’s normal execution path and is usually the simplest route to predictable GPU performance. Free VRAM—not the card’s advertised total alone—determines whether it fits alongside the operating system’s graphics allocations and the inference engine.
Mixed CPU/GPU placement
A compatible engine can place some layers in VRAM and leave the remainder in system RAM. LocalAI documents this mode for its llama.cpp backend with GGUF models: users can assign GPU layers and inspect startup logs for the number of offloaded layers and CPU/GPU buffer sizes. This can make a model run when VRAM alone is insufficient.
The trade-off is movement and slower work. CPU layers execute on the processor, and data may travel between RAM and VRAM. LocalAI warns that these transfers and CPU execution can reduce generation speed compared with a model that fits entirely on the GPU. The amount of slowdown is workload- and hardware-dependent; there is no universal percentage.
Multi-GPU distribution
Some runtimes can divide a model across multiple GPUs. Ollama evaluates available VRAM and can place a model on one GPU when it fits or spread it across available GPUs when it does not fit on one card. Distribution is an engine feature, not an automatic property of every model format. Check support for your operating system, GPU architectures, interconnect and runtime before assuming that two cards behave like one larger card.
The Tool Desk
Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →How to decide whether RAM or VRAM is your bottleneck
- Define the workload. Write down the model format, precision or quantization, target context length, expected number of simultaneous requests and whether you need an API server or a single interactive session.
- Measure free memory, not installed memory. Record available VRAM while the desktop and drivers are running, and available system RAM with your usual applications open.
- Estimate the complete footprint. Include weights, context/KV cache, runtime overhead and headroom for the operating system and other programs. Do not use the download size as the estimate.
- Choose a placement target. If the complete GPU-side footprint fits, prefer full GPU placement for a performance-sensitive workload. If it does not, decide whether CPU-only or mixed placement is acceptable.
- Check engine capabilities. Confirm that the runtime supports your model format, GPU architecture, layer offload, multi-GPU operation and desired context or concurrency settings.
- Validate with logs and an actual request. Look for reported offloaded layers, CPU and GPU buffer sizes, allocation failures and achieved generation speed. A model that loads is not automatically fast enough for your use.
What happens when VRAM is full?
Full VRAM does not always mean “cannot run.” If the runtime supports mixed placement, it can reduce the number of GPU layers and keep the rest in RAM. The model then consumes capacity from both pools, and the CPU handles the layers left outside the GPU.
Rank #2
- [3352 AI TOPS, 5th Gen Tensor Cores, AI Content Creation] Built for AI-assisted photo and video workflows including upscaling, denoise, background removal, masking, and generative AI creation for faster creator productivity.
- [32GB GDDR7 VRAM, Local LLM Inference, Larger Models] Run local LLM inference and on-device AI tools with massive VRAM headroom for larger models, longer context, and heavier multitasking across AI and creator apps.
- [28 Gbps, 512-bit, 21760 CUDA Cores] High-throughput next-gen memory and core resources for demanding creator projects, complex timelines, large assets, and GPU-accelerated ML experimentation and inference pipelines.
- [Quad-Fan Force, Vapor Chamber, Phase-Change Thermal Pad] Designed for sustained performance under heavy loads with quad-fan cooling, a patented vapor chamber, and a phase-change GPU thermal pad to help lower temps and reduce hotspots.
- [DP 2.1b x3, HDMI 2.1b x2, Bundle GPU Holder] Multi-display ready with up to 4 displays and up to 7680 x 4320 max digital resolution, plus an included GPU Holder to help reduce GPU sag and improve long-term build stability.
Three outcomes are possible:
- It fits fully on the GPU: the preferred arrangement when low latency or high token throughput matters.
- It fits with partial offload: the model runs, but CPU work and transfers may make generation substantially slower.
- It does not fit in total available memory: loading fails, the operating system starts swapping, or the process is terminated. Adding RAM helps only when system memory is the limiting pool; it cannot create additional VRAM for a GPU-only placement.
For LocalAI’s llama.cpp backend, adjust the GPU-layer setting to match available VRAM and inspect the startup log. The log is more informative than the nominal layer count because it reveals the actual CPU/GPU buffers allocated for that model and configuration.
Context length and concurrency can change the answer
Two machines with the same model and GPU can need different memory amounts because they use different context and concurrency settings. A longer context retains more tokens in the KV cache. Multiple active requests create multiple context allocations.
Ollama documents a concrete example: a 2K context configured for four parallel requests produces an 8K total context allocation. Its FAQ expresses the relationship as required RAM scaling with OLLAMA_NUM_PARALLEL multiplied by OLLAMA_CONTEXT_LENGTH. Those names and behaviors are Ollama-specific; other runtimes may allocate memory differently.
When memory is tight, reduce one variable at a time and observe the result:
- Lower the configured context length if your prompts do not need a long history.
- Reduce the number of parallel requests when serving a single user or a small queue.
- Use supported KV-cache quantization or Flash Attention options where the runtime and model support them; Ollama documents these as memory-saving choices in supported configurations.
- Leave enough headroom that the operating system does not reclaim memory or swap while the model is generating.
Is unified memory the same as VRAM?
No. Unified-memory systems change the physical arrangement: the CPU and GPU share a common memory pool rather than using separate system RAM and discrete VRAM. NVIDIA identifies Apple silicon Macs and NVIDIA Spark systems as examples of this approach.
Unified memory can let the GPU address a larger shared capacity than a small discrete card, but the capacity is still finite and shared with the operating system and applications. It also does not guarantee discrete-GPU-level bandwidth, latency or software support. Confirm that the specific inference engine supports the platform, model format and acceleration path, then budget for weights, context, runtime allocations and concurrent requests just as you would on a discrete system.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Capacity is not performance
“Can it load?” and “Is it fast enough?” are separate questions. RAM may allow a model to start through CPU execution or partial offload, while VRAM capacity determines whether the GPU can keep the working set local. The processor, memory bandwidth, GPU architecture, interconnect, kernel implementation, context and request pattern all influence speed.
Ollama’s scheduling article illustrates why a single benchmark must not be generalized. For a Gemma 3 12B model at 128K context on one RTX 4090, Ollama reported 52.02 versus 85.54 tokens per second, 19.9 versus 21.4 GiB of VRAM and 48/49 versus 49/49 layers on GPU for its old and new scheduling examples. Those are vendor-reported results for that exact model, context, card and scheduling configuration—not a RAM-versus-VRAM conversion or a prediction for another workload.
Rank #3
- System Compatibility Note: This 2-slot card measures 271 x 112 x 39 mm and requires a single 12V-2x6-pin power connector. Please verify chassis and PSU compatibility before purchase.
- Dedicated Support: Please contact us directly through Amazon for any product questions or assistance you may require.
- Professional Intel Arc Pro B70 GPU: Built on the Intel Xe2-HPG architecture, it features 32 Xe cores and 256 XMX engines, designed to accelerate AI, rendering, and complex visualization workloads.
- Massive 32GB GDDR6 VRAM: Equipped with 32GB of high-speed GDDR6 memory on a 256-bit bus, running at 19 Gbps, which allows for handling large AI models and complex datasets locally.
- High-Performance Engine Clock: Delivers an engine clock of 2540 MHz, providing the compute power needed for demanding professional applications and AI inference.
Ollama also reported in a June 5, 2026 article that Ollama 0.30 was “up to 20% faster” on NVIDIA hardware, using Gemma 4 26B on an RTX 5090 with Q4_K_M quantization as the example. “Up to” describes a configuration-specific vendor claim, not a guaranteed improvement for every model or system.
Upgrade decisions: which memory should you buy?
Prioritize VRAM when GPU throughput is the goal
Choose a GPU with enough usable VRAM for the complete GPU-resident footprint when your priority is interactive latency, high token throughput or predictable performance. Verify the model format, quantization, context and runtime before buying; a card’s VRAM number alone does not prove that a particular model will fit.
Prioritize system RAM for CPU or mixed workloads
More system RAM is relevant when you run CPU-only inference, use layer offload, host several applications or need room for large contexts and multiple requests. It is not a substitute for discrete VRAM and should not be marketed as a way to obtain the same speed as full GPU placement.
Consider unified-memory hardware for a different balance
A unified-memory machine may offer a large shared capacity without a separate VRAM ceiling. Evaluate its memory bandwidth, supported engines, GPU acceleration, upgradeability and total cost for the exact workload. Shared capacity still has to cover the model, context, runtime and the rest of the system.
Check platform and upgrade constraints first
- Confirm motherboard, laptop and operating-system compatibility for additional system RAM.
- Check the GPU’s architecture, driver support and available free VRAM under normal desktop load.
- Verify that the runtime supports your model format and the intended CPU/GPU or multi-GPU placement.
- Account for power, cooling, physical clearance and the number of expansion slots when selecting a discrete GPU.
- Set a performance target before spending: acceptable first-token latency, tokens per second, concurrent users and context length.
Troubleshooting common memory failures
The model reports an out-of-memory error immediately
Confirm whether the error names VRAM or system RAM. If VRAM is exhausted, lower GPU layers, reduce context or use a smaller quantization if your quality requirements allow it. If system RAM is exhausted, close applications, reduce concurrency or choose a smaller workload. A larger pool helps only when it is the pool named by the error.
The model loads but generation is unexpectedly slow
Inspect the startup log for CPU-resident layers and transfer buffers. A partial offload can explain a large performance drop even when the model technically runs. Compare with a smaller model or lower context that fits entirely on the GPU, and measure under the same prompt and concurrency rather than relying on another system’s benchmark.
Memory usage grows when several users connect
Check the runtime’s parallel-request and context settings. In Ollama, the documented scaling with OLLAMA_NUM_PARALLEL and OLLAMA_CONTEXT_LENGTH means a setting suitable for one request can exceed memory when multiplied across several. Reduce concurrency or context, or add capacity to the limiting memory pool.
Quick wins for a faster PC:
Scan for outdated or missing drivers - takes under a minuteDriver Scan →Clear out junk files and repair common Windows errorsFree Scan →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →A second GPU does not solve the problem
Verify that the engine supports multi-GPU distribution for the selected model format and that both cards are visible to the runtime. If the software cannot distribute layers, the model still has to fit on one card or use CPU offload. Even when distribution works, inter-GPU communication and uneven free memory can affect performance.
A practical checklist before you deploy
- Have you identified whether the machine uses discrete or unified memory?
- Did you budget weights, context/KV cache, engine overhead, the operating system and other applications?
- Are your context length and concurrent-request assumptions explicit?
- Does the runtime support the model format, GPU architecture and desired placement?
- Is the model fully GPU-resident, partially offloaded or CPU-only, and is that performance acceptable?
- Did you test the actual deployment settings and read the runtime’s allocation and offload logs?
- Are your memory figures tied to a specific precision, quantization, date, hardware and software version?
Bottom line
For discrete-GPU local AI, VRAM is the decisive resource for full-speed GPU inference; system RAM is the safety net for CPU execution, partial offload and the rest of the computer. Neither pool can be converted into the other by simply adding capacities. Plan for the complete live footprint, keep context and concurrency under control, and choose hardware only after confirming the runtime’s placement options. Unified memory changes the architecture, but it does not remove the need for careful capacity and performance testing.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
