The Tool Desk
Outbyte Driver Updater FREEScan for outdated or missing drivers - takes under a minuteDriver Scan →Outbyte PC Repair FREEClear out junk files and repair common Windows errorsFree Scan →You can run some AI models locally with a CPU and system RAM; a discrete GPU is not required. The right setup depends on the model, its quantization, context length, runtime support and the speed you expect. For GPU inference, allow memory for more than model weights: runtime buffers and the context cache need room too.
Start with the model and workload, not a universal RAM minimum
There is no single RAM or VRAM figure that guarantees every model will run. First choose the specific model and runtime, then check the model file size and quantization. Budget additional memory for runtime buffers, the context’s key/value (KV) cache, the operating system and other applications. Longer context and multiple simultaneous requests can increase memory use.
| # | Preview | Product | Price | |
|---|---|---|---|---|
| 1 |
|
GMKtec EVO-X2 AI Mini PC Ryzen Al Max+ 395 Superchip 128GB LPDDR5X 2TB SSD | $3,649.99 | Buy on Amazon |
As an Amazon Associate I earn from qualifying purchases.
For Ollama, documented context defaults vary by available VRAM: 4K tokens below 24 GiB, 32K from 24 GiB to under 48 GiB, and 256K at 48 GiB or more. These are Ollama defaults, not universal hardware requirements or a guarantee that every model supports those context lengths. See Ollama’s FAQ and context-length guide for the runtime’s current behavior.
What each hardware path can do
| Hardware path | What it offers | What to check |
|---|---|---|
| CPU-only computer | Runs compatible models without a discrete graphics card. The process uses system memory. | Capacity and speed depend on the CPU, available RAM, model and runtime. There is no supported universal speed figure. |
| Desktop with a discrete GPU | A supported GPU backend can accelerate inference, and VRAM holds the portion of the model placed on the GPU. | Check VRAM against weights plus runtime and context memory, and verify the runtime supports the GPU and model format. |
| Apple Silicon | Can use llama.cpp’s Apple Silicon path, including ARM/Accelerate and Metal support. | Unified memory is shared by CPU and GPU; it is not all dedicated VRAM. Account for system use and the workload. |
| CPU/GPU hybrid | Can offload part of a model to the GPU while other parts use system memory, allowing some models too large for full GPU residency to run. | Offloading trades performance against capacity; results depend on configuration and workload. |
| Intel GPU, NPU or other accelerator | llama.cpp documents Intel SYCL and OpenVINO support for Intel CPUs, GPUs and NPUs, as well as Vulkan and other backends. | Confirm the exact runtime, device, driver, model format and feature support; support for one backend does not imply support across all runtimes. |
llama.cpp lists CUDA for NVIDIA, HIP for AMD, Metal for Apple Silicon, SYCL for Intel GPUs and Vulkan among its backends. Its project documentation also describes CPU/GPU hybrid inference and quantization options from 1.5-bit through 8-bit. These are available paths, not a promise that every model performs equally across devices. Check the llama.cpp documentation and the chosen runtime’s compatibility notes.
#1 Best Overall
- EVOLUTION RYZEN AI MAX+ 395 MINI PC - GMKtec EVO-X2 is the next evolution in AI mini PC Ryzen Strix Halo series. Thanks to AMD Simultaneous Multithreading (SMT) the core-count is effectively doubled, to 32 threads. Ryzen AI Max+ 395 has 64 MB of L3 cache and can boost up to 5.1 GHz, depending on the workload. The Ryzen AI Max+ 395 is currently rated as the "most powerful x86 APU" on the market for AI computing.
- AI NPU with XDNA 2 ARCHITECTURE - Powered by 16 “Zen 5” CPU cores, 50+ peak AI TOPS XDNA 2 NPU and a truly massive integrated GPU driven by 40 AMD RDNA 3.5 CUs, the Ryzen AI MAX+ 395 is a transformative upgrade and delivers a significant performance boost over the competition. The Ryzen AI Max+ 395 excels in consumer AI workloads like the llama.cpp-powered application: LM Studio. Shaping up to be the must-have app for client LLM workloads, LM Studio allows users to locally run the latest language model without any technical knowledge required and unleash their creativity and productivity.
- AMD RADEON 8090S iGPU GAMING PC - The AMD Radeon RX 8060S offers all 40 CUs with up to 2.9 GHz graphics clock and uses the new RDNA 3.5 architecture. The powerful iGPU is positioned between an RTX 4060 and 4070 laptop GPU and therefore enables gaming in FHD at maximum details in most demanding games. The 8060S can also utilize the full 128GB pool, which is perfect for running LLMs such as Deepseek 70B Q8, which runs comfortably on this machine.
- EIGHT CHANNEL LPDDR5X - LPDDR5X is a new ground breaking memory small form factor installed on-board. With blazing speeds up to to 8000MT/s, it runs 1.5x faster than the DDR5 SODIMMs; 90% better performance over DDR5 SODIMMs in video conferencing and photo editing; 30% better performance in productivity apps; 12% better performance in digital content workloads.
- QUAD SCREEN 8K DISPLAY SUPPORT - EVO-X2 AI Mini PC support 4-screen 4K/8K output via HDMI 2.1 (8K@60Hz), DisplayPort 1.4 (4K@60Hz), and dual USB 4 40Gbps Transfer speed (supporting PD3.0/DP1.4/DATA). Ideal for gaming, video editing, and multitasking, it provides expansive and crisp multi-display support.
Why model file size is not the whole memory budget
Weights are only part of inference memory. The llama.cpp gpt-oss guide gives configuration-specific estimates that include model data, compute buffers and KV cache:
| Model | Context | Model data | Compute buffers | KV cache | Estimated total |
|---|---|---|---|---|---|
| gpt-oss 20B | 8,192 tokens | 12.0 GB | 2.7 GB | 0.2 GB | 14.9 GB |
| gpt-oss 20B | 131,072 tokens | 12.0 GB | 2.7 GB | Not stated separately in the guide | 17.9 GB |
| gpt-oss 120B | 8,192 tokens | 61.0 GB | 2.7 GB | 0.3 GB | 64.0 GB |
| gpt-oss 120B | 131,072 tokens | 61.0 GB | 2.7 GB | Not stated separately in the guide | 68.5 GB |
These estimates are for the guide’s stated configurations, not guaranteed minimums; the guide notes that command-line settings can change memory use. Its totals show how a longer context can raise the budget even when the model weights stay the same. The guide also explains that full GPU residency is not essential: CPU offload can run a model that does not fit entirely in VRAM, with a performance trade-off. See the llama.cpp gpt-oss guide.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.How quantization changes the trade-off
Quantization stores model weights at lower precision, reducing their memory footprint and potentially making a model fit on less capable hardware. It does not eliminate memory for buffers or context, and a smaller quantized model may produce different output quality. The right balance depends on the model and task; compare the exact model’s available quantizations and test whether the output is suitable for your use. llama.cpp documents multiple quantization widths, while Hugging Face’s Transformers optimization guide describes inference memory as broader than the model label alone.
Free tools Windows power users keep installed
One-click scans. No signup required.
Choosing hardware before you buy
- Choose a model and context length. Look up the model’s downloadable weight size, quantization choices, supported context and runtime requirements.
- Choose where the model will run. For CPU inference, compare the workload with available system RAM. For GPU inference, compare available VRAM with the complete inference budget, not just the file size. For Apple Silicon, treat unified memory as shared system capacity.
- Allow headroom. Leave memory for the operating system, runtime buffers, context cache and other applications. If you plan parallel requests or a longer context, account for their added demand.
- Verify the software path. Check the exact runtime’s backend, driver, device and model-format support before purchasing. A hardware family being listed by one project does not establish compatibility with every model or runtime.
- Decide whether offload is acceptable. CPU/GPU hybrid execution can extend capacity when a model does not fit in VRAM, but do not assume it will match the speed of a model fully resident on the GPU.
A graphics card with more VRAM is useful when the goal is to keep more of a model on the GPU, but it is not the only route to local inference. System RAM helps CPU and hybrid workloads; it does not become discrete GPU VRAM. An SSD can store downloaded model files, but it does not add inference compute.
Ollama’s GPU documentation includes an NVIDIA GeForce RTX 4090 configuration example. That establishes it as one documented example, not as a universal recommendation or the best choice for every workload. Consult Ollama’s GPU documentation for current device support and configuration details.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
