There is no single RAM or VRAM number that guarantees a local LLM will run. Start with the model file’s weight size at your chosen quantization, then add memory for the KV cache, runtime buffers, your context length, simultaneous requests, and whatever the operating system and other applications already consume. Decide whether the workload will live in GPU VRAM, system RAM, or a split between CPU and GPU.
As a practical starting point, a quantized 7–8B model often fits an 8GB graphics card at a modest context, while a 70B model commonly needs roughly 42–43GB just for Q4_K_M weights and therefore belongs in a 48GB-class GPU or a slower hybrid setup. Those are planning estimates, not minimum specifications. The exact model file and runner determine the result.
As an Amazon Associate I earn from qualifying purchases.
What “memory” means when an LLM runs locally
A local model uses more memory than the size shown for its download. Four separate demands matter:
- Weights: the model parameters stored in the model file. This is the baseline number most people see when downloading a model.
- KV cache: temporary memory that stores attention keys and values for the active conversation. It grows as the context gets longer and as more requests are processed at once.
- Runtime and compute buffers: working space used by the inference engine for intermediate calculations, loading, and scheduling.
- Available headroom: memory left for the operating system, desktop, drivers, browser, and other GPU workloads.
A model can have a 4.9GB file and still fail on a card advertised with 6GB because the card does not have 6GB free for the model alone. Conversely, a model that does not fit entirely in VRAM may still run with CPU or hybrid execution, provided the system has enough RAM and the runner supports that placement.
#1 Best Overall
- Read Before You Buy — No Video Output: These adapters support charging and USB 2.0 data transfer, but cannot transmit video signals. Except for standard USB webcams (which use USB data only), they are not compatible with HDMI/DisplayPort cables, video-capable USB-C hubs, or docking stations with video output.
- Convert USB-A Ports to USB-C: Designed to connect USB-C earphones, cables, flash drives, card readers, and other USB-C accessories to standard USB-A ports. Plug-and-play with no drivers or software required.
- Aluminum Alloy Housing: Built with a sturdy aluminum alloy shell that aids in heat dissipation and protects against daily wear and scratches. Designed to maintain a stable and secure connection.
- Compact & Travel-Friendly: The ultra-compact design allows the adapter to stay plugged into your device without blocking adjacent ports or adding bulk, reducing wear and tear on your original USB ports.
- 12-Month Warranty: Backed by a 12-month manufacturer warranty for peace of mind. Designed to meet strict quality control standards for reliable everyday performance.
The Ollama FAQ and the llama.cpp README both document placement and runtime behavior, but neither treats a model’s parameter count as a complete memory specification.
Model weights: the first sizing number
Quantization changes how many bits are used to store weights. Lower-bit formats make the file substantially smaller, usually with a quality and speed trade-off that depends on the model and runtime. The llama.cpp documentation lists 1.5-, 2-, 3-, 4-, 5-, 6-, and 8-bit integer quantization options; the label alone does not tell you the final file size, because formats also include metadata and use different schemes for different layers.
The quantization documentation displays these Llama 3.1 examples:
| Model | Original weight size | Q4_K_M weight size | What the figure represents |
|---|---|---|---|
| Llama 3.1 8B | 32.1GB | 4.9GB | Model-file examples shown in the llama.cpp quantization README |
| Llama 3.1 70B | 280.9GB | 43.1GB | Model-file examples shown in the same documentation |
These values are documentation examples, not universal VRAM guarantees. A different quantization, a different model revision, or a different file format can produce a different result. Check the exact file you intend to load rather than estimating from “8B” or “70B” alone.
Approximate memory tiers for common model sizes
The following estimates come from OllamaLab’s guide published 1 August 2026 and updated 18 August 2026. The guide describes Q4_K_M weight sizes and illustrative VRAM tiers, then advises budgeting another 1–3GB for KV cache, compute buffers, and desktop use. They are estimates, not independently tested minimums.
Rank #2
- 5-in-1 USB-C Hub: Experience comprehensive connectivity featuring a Power Delivery input, two USB-A 2.0 ports, a USB-A 3.0 port, and an HDMI port. (Note: The USB-C power delivery input port is only for connecting an external wall charger to power your laptop and cannot power peripheral devices.)
- 90W Pass-Through Charging: Achieve optimal charging with 90W pass-through power to your laptop, supported by a total input of 100W, with the hub reserving 10W for operational efficiency. (Note: Wall charger not included.)
- Quick Data Transfers: Accelerate your productivity with rapid data transfers using a high-speed 5Gbps USB 3.0 port and two 480Mbps USB 2.0 ports.
- 4K HDMI Display: Enhance your visual experience with a hub capable of delivering 4K resolution at 30Hz in both mirror and extend modes. Please note that this hub is compatible with MacBook (macOS 12 and newer), Windows 10 and 11, ChromeOS, and laptops equipped with DP Alt Mode and Power Delivery. Note: This device is not compatible with Linux.
- What You Get: Anker USB-C Hub (5-in-1, 4K HDMI), welcome guide, 18-month warranty, and our friendly customer service.
| Model size | Approximate Q4_K_M weights | Illustrative VRAM tier in the guide | Interpretation |
|---|---|---|---|
| 7–8B | About 4.8GB | 8GB | Can suit a modest context; remaining headroom may be tight. |
| 13–14B | About 8.4GB | 12GB is a stretch for short context; 16GB is more comfortable | Context, desktop use, and runtime overhead determine whether the card remains usable. |
| 27–32B | About 19GB | 24GB | Described as a comfortable Q4_K_M choice in that guide. |
| 70B | About 42GB | 48GB | A moderate-context tier; larger contexts or multiple requests need more room. |
Source: OllamaLab’s August 2026 sizing guide. The guide’s figures should not be read as promises that every model in a parameter band will run at the stated context on every machine.
VRAM, system RAM, and hybrid execution
Full GPU placement
When all weights and working data fit in VRAM, the runner can keep the computation on the GPU. This generally avoids moving tensors across the system bus, but the model still needs room for the KV cache and runtime buffers. A card’s advertised capacity is not the same as capacity available to the model after the display, driver, desktop compositor, and other applications have taken their share.
Recommended Free Tools
CPU placement
CPU inference uses system RAM instead of requiring the complete model in VRAM. This makes larger models possible on a machine without a high-memory graphics card, but speed depends heavily on the processor and memory bandwidth. Storage capacity is also relevant for keeping model files, although storage does not substitute for RAM while a model is actively generating.
Hybrid placement
llama.cpp supports splitting a model between CPU and GPU, and the Ollama FAQ describes distributing a model across available GPUs. Hybrid execution can make a model runnable when it exceeds one GPU’s VRAM, but it should not be presented as equivalent in speed to keeping the entire working set on one GPU. More data transfers and less GPU residency can reduce tokens per second.
For a split setup, budget both pools independently: enough system RAM for the CPU-resident portion and enough free VRAM for the GPU-resident portion, plus each pool’s operating overhead. A machine can fail even when total RAM plus VRAM appears large if the runner cannot place the layers in the available portions.
Rank #3
- Sleek 7-in-1 USB-C Hub: Features an HDMI port, two USB-A 3.0 ports, and a USB-C data port, each providing 5Gbps transfer speeds. It also includes a USB-C PD input port for charging up to 100W and dual SD and TF card slots, all in a compact design.
- Flawless 4K@60Hz Video with HDMI: Delivers exceptional clarity and smoothness with its 4K@60Hz HDMI port, making it ideal for high-definition presentations and entertainment. (Note: Only the HDMI port supports video projection; the USB-C port is for data transfer only.)
- Double Up on Efficiency: The two USB-A 3.0 ports and a USB-C port support a fast 5Gbps data rate, significantly boosting your transfer speeds and improving productivity.
- Fast and Reliable 85W Charging: Offers high-capacity, speedy charging for laptops up to 85W, so you spend less time tethered to an outlet and more time being productive.
- What You Get: Anker USB-C Hub (7-in-1), welcome guide, 18-month warranty, and our friendly customer service.
Why context length changes the requirement
The KV cache grows with the number of tokens the model must retain. A short chat and a long document-processing prompt therefore have different memory footprints even when they use the same weights. Increasing the context window can turn a model that loads successfully into one that runs out of memory during generation.
The Tool Desk
Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →Ollama states that Flash Attention, a feature of most modern models, “can significantly reduce memory usage as the context size grows.” The statement appears in the project’s FAQ; whether it applies depends on model and runtime support. The same documentation also describes quantized KV-cache options, which trade memory use against numerical precision.
Do not size a system from the model’s advertised maximum context alone. Choose the context you will actually keep active, then test a prompt near that length. If you need a very long context, leave substantially more free memory than the weight file suggests.
Concurrency and multiple loaded models
One request at a time has a different footprint from a server handling several conversations. Ollama documents that required RAM scales with the parallel-request count multiplied by the context length. Two simultaneous requests at the same context therefore require more cache capacity than one; increasing both parallelism and context compounds the demand.
Loading more than one model also consumes memory for each model’s weights and runtime state. A practical server plan should specify:
Rank #4
- Dual Converters, Infinite Potential：Includes 2× USB C male to USB A female adapters and 2× USB A male to USB C female adapters. Perfect for a wide range of uses—tablets with Bluetooth keyboards, expand USB ports on macbook, and more. Two different converters for all your daily needs
- Next-Level 10Gbps & 3A Charging: No more slow 480Mbps, this usb to usb c adapter has a transfer speed of up to 10Gbps, allowing you to do more transferring in less time. This usb adapter fits both USB A and USB C charger, supporting up to 3A fast charging
- Upgraded Exquisite Craftsmanship: With an aluminum alloy housing and metal connector, the usbc to usb adapter is extremely durable and sturdy. Rigorously tested to withstand more than 10,000 times of plugging and unplugging, ensuring long-lasting performance
- Broad Compatible: The usb c to usb adapter widely supports all USB C/ USB A devices like laptops, tablets, cellphones, car chargers, and phone chargers. Such as compatible with MacBook Pro/Air 2023/2022, Thunderbolt 4/3 Devices,Apple MagSafe Watch 9/8/7/SE/Ultra, iPad Pro 2022/2021, Samsung Galaxy S23/S20/S10, and iPhone 17/16/15 Pro. Plug and play
- Please Note: To reach 10Gbps speed, keep the cable under 3.3 ft. For USB A Male to USB C adapters, try flipping the USB C connector. USB C Male to USB A adapters support bidirectional 10Gbps transfer within 3.3 ft
- the number of models kept loaded simultaneously;
- the maximum context per request;
- the maximum number of concurrent requests;
- which layers, if any, may spill to CPU memory; and
- how much memory must remain available for the operating system and monitoring tools.
If a model is unloaded between requests, peak memory may be lower than in an always-warm multi-model service, but reload time becomes part of the user experience.
A step-by-step way to size a local LLM
- Choose the exact model file. Record its parameter count, quantization format, and file size. Use the file’s published size rather than a generic “7B” or “70B” label.
- Decide where it will run. Select full GPU, CPU, or hybrid placement. For GPU placement, start with free VRAM, not the card’s nominal capacity. For CPU placement, start with available system RAM.
- Set the real context length. Use the maximum context your application needs, not merely the model’s advertised limit. Long-context workloads require more KV-cache memory.
- Set concurrency. Multiply the planning burden when several requests can be active at once. A personal chat with one request needs a different budget from a local API serving multiple users.
- Add runtime and desktop headroom. OllamaLab’s guide uses an illustrative 1–3GB allowance beyond weights for cache, buffers, and desktop use, but the margin is not guaranteed for every model or runner.
- Check for other consumers. Close or account for games, browsers, image-generation jobs, virtual machines, and other GPU processes. Their allocations reduce memory available to the LLM.
- Validate the exact configuration. Load the intended model, use the intended context, and exercise the intended concurrency while watching VRAM and RAM. A successful load at a short prompt does not prove that the production context will fit.
Choosing hardware for a workload
There is no universally best graphics card for local LLMs. Compare the category of hardware against your workload:
| Decision factor | Why it matters |
|---|---|
| Free VRAM or unified memory | Determines how much of the model and cache can remain on the GPU after the operating system and other workloads. |
| System RAM | Enables CPU inference, hybrid placement, and model loading when VRAM is insufficient. |
| Memory bandwidth | Influences generation speed after the model fits; the cited sources do not provide a universal card ranking. |
| Model and quantization | Sets the weight baseline and quality trade-off. |
| Context and concurrency | Sets KV-cache demand and can dominate the extra memory requirement. |
| Price and tolerance for hybrid speed | Determines whether a high-VRAM graphics card or a less expensive CPU/hybrid arrangement is appropriate for your use. |
A high-VRAM graphics card with 24GB can be a sensible category to investigate for 13–14B and 27–32B Q4_K_M workloads, but exact fit still depends on context, cache settings, and runtime overhead. Treat 24GB as an available-memory target, not a guarantee. If your model exceeds one card’s capacity, multi-GPU or hybrid execution may work, subject to the runner’s layer-placement behavior.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Common out-of-memory failures and fixes
The model will not load
First compare the exact file size with free VRAM or RAM. If the file is larger than the chosen pool, switch to a smaller quantization, use CPU or hybrid placement, close competing applications, or choose a smaller model. A nominally sufficient card can still fail when only a small fraction of its capacity is free.
It loads, then fails on a long prompt
This usually indicates KV-cache growth rather than a weight-loading problem. Reduce the active context, lower concurrency, enable a supported Flash Attention or quantized KV-cache mode, or move part of the workload to system RAM if the runner permits it.
Best Value
- 5-in-1 Connectivity: Equipped with a 4K HDMI port, a 5 Gbps USB-C data port, two 5 Gbps USB-A ports, and a USB C 100W PD-IN port. Note: The USB C 100W PD-IN port supports only charging and does not support data transfer devices such as headphones or speakers.
- Powerful Pass-Through Charging: Supports up to 85W pass-through charging so you can power up your laptop while you use the hub. Note: Pass-through charging requires a charger (not included). Note: To achieve full power for iPad, we recommend using a 45W wall charger.
- Transfer Files in Seconds: Move files to and from your laptop at speeds of up to 5 Gbps via the USB-C and USB-A data ports. Note: The USB C 5Gbps Data port does not support video output.
- HD Display: Connect to the HDMI port to stream or mirror content to an external monitor in resolutions of up to 4K@30Hz. Note: The USB-C ports do not support video output.
- What You Get: Anker 332 USB-C Hub (5-in-1), welcome guide, our worry-free 18-month warranty, and friendly customer service.
It works for one user but not several
Parallel requests multiply cache demand. Lower the maximum number of simultaneous requests, cap each request’s context, or add memory. Keep separate limits for interactive chat and batch jobs so a large batch cannot consume all available cache.
It runs but is unexpectedly slow
Check whether the runner placed layers on the CPU or split them across devices. Hybrid execution can be functional while being much slower than full GPU placement. Also check memory pressure and bandwidth; fitting the weights is necessary, but it does not determine generation speed by itself.
How to interpret a “fits in 8GB” claim
Such a claim normally means the model’s weights are close to that capacity under a particular quantization and context. It does not promise room for a long context, multiple conversations, a desktop session, or another GPU workload. Treat every fit claim as a configuration-specific starting point and reproduce it with the same model file, runner, context, and concurrency.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
The safest rule is simple: size for the peak working set, not the download. Weights establish the floor; KV cache, runtime buffers, placement, and headroom establish whether the system remains usable.
Frequently Asked Questions
Does unified memory count as VRAM or system RAM?
For sizing, count only the portion the runner can actually reserve for the model while the operating system and other applications remain responsive. A shared pool is not equivalent to having that entire capacity dedicated to the LLM.
Can I run a model larger than my graphics card’s VRAM?
Often, yes, with CPU or hybrid execution when the runner supports it and sufficient system RAM is available. Expect different performance from full-GPU placement, and verify how the runner distributes layers.
Why did a model fit yesterday but fail today?
Available memory may have changed because of a longer context, more concurrent requests, another loaded model, or a desktop application using the GPU. Measure free memory and repeat the test with the intended workload.
Windows Errors? Fix Them Before They Spread
Repair common Windows errors and clear accumulated junk for a smoother, more stable PC - no reinstall needed.Free scan · no reinstallOutdated Drivers Are Slowing You Down
One free scan finds every outdated or missing driver and matches the right update for your exact hardware.Free scan · exact hardware matchQuick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
