What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
There is no single memory requirement for local AI development: it depends on what you plan to do, which model and precision you use, how much context it handles, and what your runtime supports. For GPU inference, start with model-weight memory, then account for the KV cache and runtime overhead. Fine-tuning can need substantially more memory than inference. System RAM is a separate capacity that matters most for CPU execution, model loading, and CPU offload.
Start with the workload, not a single RAM number
“Local AI development” can mean running a model for inference, adapting it with LoRA or Q-LoRA, or fully fine-tuning it. Those workloads have different memory demands. GPU VRAM is the main constraint when weights and inference state are placed on the GPU; system RAM supports CPU-side work and can hold model components when a runtime offloads them from the GPU.
As an Amazon Associate I earn from qualifying purchases.
- Inference: Estimate weights, then add memory for the KV cache, runtime allocations, and other GPU use.
- Fine-tuning: Use an estimate for the specific training method. An inference estimate is not a sound proxy for training memory.
- CPU inference or offload: Size system RAM for the model and runtime configuration. Host RAM can help make a model loadable, but it is not interchangeable with GPU VRAM.
NVIDIA’s general guidance is to choose hardware based on the operating system, available GPU or unified memory, model size, and workflow. The runtime and backend matter too: they determine which hardware is supported and how memory is allocated.
Estimate VRAM for inference
First estimate the model weights
As a quick weight-only rule, Hugging Face’s Transformers documentation, version 4.42.0, estimates roughly 4 × X GB to load a model with X billion parameters in float32, or 2 × X GB in bfloat16/float16. For example, the rule gives about 16 GB of weights for an 8-billion-parameter model in float32 and about 8 GB in bfloat16/float16. These figures estimate weights, not a complete safe VRAM capacity for running the model.
#1 Best Overall
- AMD Ryzen 7 9850X3D 4.7GHz (5.6GHz Turbo Boost) CPU Processor | 2TB NVMe M.2 SSD – Up to 30x Faster Than Traditional HDD | 360mm AIO Liquid CPU Cooler with ARGB Fans, say goodbye to outdated and inefficient air coolers.
- NVIDIA GeForce RTX 5080 16GB GDDR7 Graphics Card (Brand may vary) | 32GB DDR5 RAM 6000 RGB Gaming Memory with Heat Spreader | Windows 11 Home 64-bit
- WI-FI 5 802.11ac | No Bloatware | Graphic output options include 1 x HDMI, and 1 x Display Port Promised, Additional Ports may vary | USB Ports Including 2.0, 3.0, and 3.2 Gen1 Ports | HD Audio & Mic | Free Gaming Keyboard & Mouse
- High-spec AIO liquid coolers used, delivering unmatched cooling performance for a perfect operational experience and unparalleled cooling performance. With hardware unrestricted by temperature limits, you can unleash its full potential. Whether gaming, creating, or working, you'll never suffer from thermal throttling again. | Showcase Your PC with the Stunning King 95 Case - Black | 1 Year Warranty on Parts and Labor | Free Technical Support | Assembled in the USA
- This powerful gaming PC is capable of running all your favorite games such as Elden Ring, Baldur's Gate 3, Cyberpunk 2077, Hogwarts Legacy, Black Myth: Wukong, Helldivers 2, Diablo IV, Starfield, Valorant, Counter-Strike 2, Forza Horizon 5, Resident Evil 4, Alan Wake 2, Warhammer 40,000: Space Marine 2, God of War Ragnarök, Overwatch 2, Dragon's Dogma 2, Marvel's Spider-Man, more at Ultra settings, detailed 4K Ultra HD resolution, and smooth 60+ FPS gameplay.
The estimate assumes the stated precision and parameter count. Check the actual checkpoint and the precision your runtime will use rather than relying on a model’s name alone.
Then account for context and runtime memory
During inference, a model’s KV cache stores keys and values for tokens in context. It grows with context length and can become a substantial part of memory use. Hugging Face’s Llama 3.1 guide gives these FP16 KV-cache estimates:
Rank #2
- AMD Ryzen 7 9850X3D 4.7GHz (5.6GHz Turbo Boost) CPU Processor | 2TB Gen4 NVMe M.2 SSD – Up to 30x Faster Than Traditional HDD | 360mm AIO Liquid CPU Cooler with ARGB Fans, say goodbye to outdated and inefficient air coolers.
- AMD Radeon RX 9070 XT 16GB GDDR6 Graphics Card (Brand may vary) | 32GB DDR5 RAM 5600 Gaming Memory with Heat Spreader | Windows 11 Home
- High-spec AIO liquid coolers used, delivering unmatched cooling performance for a perfect operational experience and unparalleled cooling performance. With hardware unrestricted by temperature limits, you can unleash its full potential. Whether gaming, creating, or working, you'll never suffer from thermal throttling again. | Skytech Azure Gaming Case with Tempered Glass, Black | 1 Year Warranty on Parts and Labor | Free Technical Support | Assembled in the USA
- This powerful gaming PC is capable of running all your favorite games such as Elden Ring Nightreign, Baldur's Gate 3, Cyberpunk 2077, Hogwarts Legacy, Helldivers 2, Diablo IV, Starfield, Valorant, Counter-Strike 2, Forza Horizon 5, Resident Evil 9, Alan Wake 2, Warhammer 40,000: Space Marine 2, God of War Ragnarök, Overwatch 2, Dragon's Dogma 2, Marvel's Spider-Man, Clair Obscur: Expedition 33,, more at Ultra settings, detailed 4K Ultra HD resolution, and smooth 60+ FPS gameplay.
| Model | 1k-token context | 16k-token context | 128k-token context |
|---|---|---|---|
| Llama 3.1 8B | 0.125 GB | 1.95 GB | 15.62 GB |
| Llama 3.1 70B | 0.313 GB | 4.88 GB | 39.06 GB |
These are estimates for those Llama 3.1 models, not a universal cache formula. Concurrent sequences and implementation details can also affect the workload. Hugging Face notes that its checkpoint-loading estimates omit framework-reserved space for items such as kernels or CUDA graphs, so leave room beyond the listed weights and cache.
The Tool Desk
Outbyte Driver Updater FREEScan for outdated or missing drivers - takes under a minuteDriver Scan →Outbyte PC Repair FREEClear out junk files and repair common Windows errorsFree Scan →Quantization can reduce weight memory
For Llama 3.1, Hugging Face estimates the following GPU memory just to load the checkpoint:
Rank #3
- 【System】AMD Ryzen 7 9800X3D CPU Processor 8 Cores 16 Threads 4.7 GHz CPU (max up to 5.2 GHz) , AMD B850 Chipset Motherboard, Windows 11 Home Prebuilt Gaming PC
- 【Graphics & Memory】 RTX 5080 16 GB GDDR7, 256 bit Graphics Card Gaming PC, 32GB DDR5 6000Mhz RGB Memory, 2TB NVMe Gen4 SSD
- 【Cooler & Power】STORMCRAFT Phantom Gaming Computer Case, 360mm AIO Liquid Cooling PC, 7x ARGB Color Adjustable System Fans, 850W Gold Certified Power Supply, Case Size 17" x 9.25" x 17"
- WARRANTY: 2 Year Parts and 3 Year Labor, 1 Year Shipping, FREE Lifetime Technical Support , Assembled in California, USA
- 【Game Without Limits】This powerful Gaming PC use AI rendering to deliver a massive performance, which is capable of running all your favorite games whether you’re a optinal gamer of Black Myth WuKong, World of Warcraft, Call of Duty Warzone, Valorant, League of Legends, Apex Legends, Roblox, Overwatch, Elden Ring, Rocket League and Diablo IV etc
| Model | FP16 | FP8 | INT4 |
|---|---|---|---|
| Llama 3.1 8B | 16 GB | 8 GB | 4 GB |
| Llama 3.1 70B | 140 GB | 70 GB | 35 GB |
Those figures are checkpoint-only estimates from Hugging Face’s Llama 3.1 guide; they do not include KV-cache and runtime memory. Quantization reduces the space used by weights, but its effect on output accuracy or speed depends on the model, quantization method, runtime, and task. If quality matters, compare a specific quantized version on the task you intend to run.
Fine-tuning has a different memory budget
Hugging Face’s Llama 3.1 guide estimates the following memory for three training approaches. The figures are estimates, not guarantees, and are specific to the models and methods named.
Rank #4
- Intel Core i5 14400F 2.5GHz (4.7GHz Turbo Boost) CPU Processor | 1TB NVMe M.2 SSD – Up to 30x Faster Than Traditional HDD | High-Performance Air Cooler
- NVIDIA GeForce RTX 5060 8GB GDDR7 Graphics Card (Brand may vary) | 16GB DDR5 RAM 6000 Gaming Memory with Heat Spreader | Windows 11 Home 64-bit
- 802.11 AC | No Bloatware | Graphic output options include 1 x HDMI, and 1 x Display Port Promised, Additional Ports may vary | USB Ports Including 2.0, 3.0, and 3.2 Gen1 Ports | HD Audio & Mic | Free Gaming Keyboard & Mouse
- High-Performance Air Cooler: Maximum Airflow & ARGB Fans | Skytech Archangel 5 Gaming Case with Tempered Glass, White | 1 Year Warranty on Parts and Labor | Free Technical Support | Assembled in the USA
- This powerful gaming PC is capable of running all your favorite games such as Call of Duty, Fortnite, Escape from Tarkov, Grand Theft Auto V, Valorant, World of Warcraft, League of Legends, Apex Legends, PLAYERUNKNOWN’s Battlegrounds, Overwatch 2, Counter-Strike 2, Battlefield V, Minecraft, ELDEN RING Shadow of the Erdtree, Rocket League, Baldur’s Gate 3, Dota 2, HELLDIVERS 2, Monster Hunter, Terraria, Rainbow Six Siege, Black Myth Wukong, Marvel Rivals, Stellar Blade, more at Ultra settings, detailed 1080p Full HD resolution, and smooth 60+ FPS gameplay.
| Model | Full fine-tuning | LoRA | Q-LoRA |
|---|---|---|---|
| Llama 3.1 8B | 60 GB | 16 GB | 6 GB |
| Llama 3.1 70B | 500 GB | 160 GB | 48 GB |
The differences show why an inference budget cannot answer whether a training job will fit. Before choosing hardware, identify whether the intended job is full fine-tuning, LoRA, or Q-LoRA, then check the requirements for the particular model, training setup, and software.
PC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11Outdated Drivers Are Slowing You Down
One free scan finds every outdated or missing driver and matches the right update for your exact hardware.Free scan · exact hardware matchSystem RAM is separate from VRAM
There is no universal system-RAM minimum established by the cited documentation. The amount you need depends on whether inference runs on the CPU, whether layers are offloaded from the GPU, the model file and context, and what else is running.
Best Value
- System: Core i9 Unlocked OC CPU | Premium Chipset | 64GB Ram (Twice the high end average of 32GB in other systems) | 5TB Storage Total: 1TB M.2 NVMe up to 7000MB/s speeds SSD + 4TB 7200RPM HDD (Ultra Fast Storage), Extra M.2 NVME and HDD Port for additional Storage | Windows 11 PRO preinstalled for Advanced security and device control.
- Graphics: NVIDIA GeForce RTX 5070 OC 12GB | Factory overclocked for higher and more consistent frame rates | Real-time ray tracing for realistic lighting and reflections | DLSS 4.0 support for smoother performance at higher resolutions | Improved efficiency and lower power draw | Stronger support for multi-monitor setups with 1x HDMI and 3x DisplayPort | Better stability for long gaming sessions and GPU-accelerated tasks | VR and AI Deeplearning Ready
- Cooling & Design: 360mm Liquid Cooling | Intelligently controlled Fan Speeds for whisper quiet performance | ARGB Lighting (Software Control for thousands of options) | Dragon Front Panel | Total of 11 Fans (3 on GPU, 1 on Power supply, 8 on Overall temperature control)
- Connectivity: 1 x USB-C 3.2 | 8 x USB 3 |1 x LAN / Ethernet up to 2.5GB/s | WiFi up to 2.4GB/s | Bluetooth Enabled | Game and VR Ready | 850W 80+ GOLD Power Supply With x6 Extra SATA Connectors
- Build Quality & Support: Premium components chosen for long-term reliability | Thorough quality testing before shipment | 3-year parts warranty and 5-year labor warranty | Access to specialists with over 20 years of experience for hardware, software, and performance support | Quiet and dependable operation for everyday and extended use || As of August 17, 2026, all firmware and software components are fully updated before shipment. Fast, free 10 minute firmware update assistance is now available through our support team (Note: Firmware only needs to be updated once every 2-3 years)
For example, llama.cpp documents memory-mapped model loading, an option to lock model pages in RAM, and offloading to devices. Its documentation warns that a model larger than available RAM can fail to load when memory mapping is disabled. That makes the exact runtime configuration important: check how it loads the model and what it places on the CPU versus GPU. Adding system RAM does not increase GPU VRAM or guarantee a particular speed.
Will an 8 GB GPU run a local model?
It depends on the model, quantization, context, and runtime. In Hugging Face’s Llama 3.1 checkpoint estimates, 8B INT4 weights use 4 GB, while 8B FP8 weights use 8 GB just to load the checkpoint. Neither number by itself establishes that a complete inference workload will fit in an 8 GB GPU: the KV cache and runtime allocations require additional room. A shorter context uses less cache than a long one, but the actual fit must be checked for the chosen software and workload.
For broader hardware context, NVIDIA lists GeForce RTX cards in a 6–32 GB VRAM category range and RTX PRO cards in a 16–96 GB range. These are category ranges, not recommendations for a particular model or task; verify the exact GPU and runtime compatibility.
A practical way to size a local AI setup
- Name the workload and model. Decide whether you need inference, LoRA, Q-LoRA, or full fine-tuning, and identify the actual checkpoint and parameter count.
- Set the precision or quantization. Estimate weight memory at the precision you will use. As a general Transformers rule, weights take roughly 2 × X GB for X billion parameters in bfloat16/float16 or 4 × X GB in float32.
- Choose the context and concurrency you need. Include the KV cache for that workload; do not size from weights alone.
- Check the runtime and hardware path. Confirm support for the GPU, operating system, backend, quantization, and any CPU offload you plan to use.
- Leave headroom. Account for runtime allocations, the operating system, development tools, other applications, batches, and longer prompts than usual.
- If it does not fit, change a real constraint. Consider a smaller model, a quantized checkpoint, multiple GPUs, or CPU offload. Offload depends on host RAM and runtime support; moving work to system memory may allow a model to load but does not provide the same capacity or performance as GPU memory.
When comparing upgrades, compare the whole workload rather than a headline VRAM number: model and precision, context and concurrent sequences, training method if applicable, available GPU or unified memory, host RAM needs, runtime compatibility, and acceptable throughput.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
