Free tools Windows power users keep installed
One-click scans. No signup required.
Yes—as the GPU in a hybrid setup, a single RTX 4090 can participate in running a 100B+ mixture-of-experts (MoE) model with llama.cpp. That does not mean the whole model fits in the card’s VRAM: MoE weights can be kept in CPU memory, while the GPU holds only the layers that fit. The full weight set still has to be accessible, and host memory, context, runtime buffers, model quantization, and the exact llama.cpp build all affect whether a particular configuration loads and how fast it runs.
What “100B+” and “active parameters” mean for memory
An MoE model may have a large total parameter count but route each token through only a subset of its experts. A January 2026 Hugging Face community guide describes DeepSeek V3 as having 671 billion total parameters, with 37 billion active during each forward pass. The active count helps describe computation; it does not mean the remaining weights can be discarded. The model’s full weight set must remain available in memory or through the model’s loading path.
| # | Preview | Product | Price | |
|---|---|---|---|---|
| 1 |
|
VIPERA NVIDIA GeForce RTX 4090 Founders Edition Graphic Card | $4,425.00 | Buy on Amazon |
| 2 |
|
ASUS TUF Gaming GeForce RTX™ 5080 16GB GDDR7 OC Edition Graphics Card | $1,829.97 | Buy on Amazon |
As an Amazon Associate I earn from qualifying purchases.
Quantization can reduce the weight-file footprint, but the resulting file is not the whole runtime memory budget. For one concrete example, bartowski’s mutable DeepSeek-V3-0324-GGUF repository listed its Q4_K_M file at 404.43 GB on October 7, 2026. That is a third-party listing for a particular model artifact and quant—not a universal size for DeepSeek variants, a quality guarantee, or an official RAM recommendation. Runtime buffers, the operating system, context length, and key-value (KV) cache add requirements beyond the file itself.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
How llama.cpp decides what stays on the CPU or goes to the GPU
llama.cpp’s server README documents controls for placing MoE weights in CPU memory and a separate control for the maximum number of layers stored in VRAM. These are placement controls; they do not amount to a documented rule that the server automatically identifies the most frequently used individual experts and keeps those experts on the GPU. The README is mutable, so check the version you are running and use its startup logs to confirm actual placement and memory use.
#1 Best Overall
- 16,384 NVIDIA CUDA Cores
- Supports 4K 120Hz HDR, 8K 60Hz HDR and variable refresh rate as indicated in HDMI 2.1A
- New streaming multiprocessors: up to 2x power and power efficiency
- Fourth generation tensor cores: up to 2x AI power
- Third-generation RT cores: up to 2x ray tracing performance
| Option | Documented scope | What it does not mean |
|---|---|---|
--cpu-moe or -cmoe |
Keeps all MoE weights in CPU memory. | It does not move only selected, frequently routed experts to the CPU. |
--n-cpu-moe N or -ncmoe N |
Keeps MoE weights from the first N layers in CPU memory. |
N counts layers, not individual experts. |
--n-cpu-ffn N or -ncffn N |
Keeps dense feed-forward network (FFN) weights from the first N layers in CPU memory. |
This is a separate control from MoE expert weights. |
--n-gpu-layers N or -ngl N |
Sets the maximum number of layers stored in VRAM. The README documents a count, auto, or all. |
It does not specify that the entire model must fit on the GPU. |
--fit and related fit options |
Can automatically adjust unset arguments to fit available device memory, using documented margin and minimum-context settings. | Automatic fitting does not remove the need to budget for host memory or verify the resulting configuration. |
For the literal question of which experts land on each device: the documented MoE controls describe CPU placement across all MoE weights or weights in the first N layers, not a per-expert GPU-selection policy. The layer limit is a separate VRAM constraint. Do not infer a more precise expert-by-expert allocation algorithm from these option descriptions; confirm the actual placement in logs for your model and build.
Plan a hybrid setup around the whole runtime
A 24 GB RTX 4090 can be the GPU component of a hybrid run, but the card’s capacity alone cannot establish that a model will load. Before downloading a large quant or tuning placement, work through the whole memory and software configuration.
Rank #2
- Powered by the NVIDIA Blackwell architecture and DLSS 4. System Requirements: Minimum 850W PSU with 16-pin 12V-2x6 (12VHPWR) connector required. Verify before purchasing.
- Military-grade components deliver rock-solid power and longer lifespan for ultimate durability. Compatibility: 348mm (13.7") length, 3.6 slots, 4.3 lbs. Confirm case clearance and slot spacing. GPU bracket included.
- Protective PCB coating helps protect against short circuits caused by moisture, dust, or debris
- 3.6-slot design with massive fin array optimized for airflow from three Axial-tech fans
- Phase-change GPU thermal pad helps ensure optimal thermal performance and longevity, outlasting traditional thermal paste for graphics cards under heavy loads
- Choose the exact model artifact and quant. Check the repository’s current file listing and the specific file size rather than estimating from total parameter count. Quantization affects the footprint and may affect model quality; do not assume a smaller file is quality-neutral.
- Budget host memory for more than the weights. The complete weight set must be accessible, with additional capacity for the operating system and runtime needs. A model file’s size alone is not a RAM minimum. The available sources establish no universal RAM recommendation for 100B+ MoE inference.
- Reserve VRAM for more than model layers. VRAM also has to accommodate the KV cache and compute/runtime buffers. The amount left for weights depends in part on context length and the configuration, so a layer count that works at one context may not work at another.
- Set CPU MoE placement and the GPU layer limit deliberately. Use
--cpu-moefor all MoE weights in CPU memory, or--n-cpu-moe Nfor MoE weights in the first N layers. Set--n-gpu-layersto a count,auto, orallas appropriate for the current build and available VRAM. Treat--n-cpu-ffnas a separate dense-FFN setting, not an MoE setting. - Check fit behavior and startup logs. llama.cpp documents fit options that can adjust unset arguments to available device memory. Inspect the current server README for option syntax and defaults, then inspect startup logs to see what your particular build actually placed and how much memory it reports using.
- Record the conditions when testing. Keep the model repository and exact file, quant, llama.cpp revision and build backend, driver/runtime, context, batch and ubatch settings, and hardware together with the results. Measure prompt processing and token generation separately; neither alone describes the other.
What to expect from performance—and what not to assume
Hybrid offloading makes a model that exceeds VRAM potentially usable by drawing on host memory, but it does not turn host-resident weights into GPU-resident weights. Performance depends on the model, quant, placement, context and KV-cache needs, CPU/RAM subsystem, and exact software build. The reviewed sources do not establish a universal RAM requirement, context recommendation, or speed for 100B+ inference on one RTX 4090.
Do these 3 things before closing this tab:
1Clear out junk files and repair common Windows errors2Fix the driver behind crashes, sound loss and screen glitches3Repair Windows errors before they cause bigger problemsA September 2, 2026 llama.cpp community discussion proposed a persistent expert-slot pool under the --moe-expert-cache name and described a companion branch. Its author reported tests on one 24 GB RTX 4090 using Qwen3.8-Flash-Next at 8K context with greedy generation, but those measurements are branch-specific and depend on that run’s quantization and conditions. The discussion is an RFC, not evidence that the feature is available in upstream llama.cpp or that its results generalize to other models. It also notes that cache benefit depends on routing skew and working-set size; choosing a slot count equal to top-k may be a poor setting. Treat the feature as experimental unless your exact build and branch verify otherwise, and do not use those reported results as a general speed promise.
Quick Recap
When this approach is—and is not—a fit
It may be worth trying when
- You have enough host memory to make the selected quant and runtime accessible, while preserving room for the operating system and other allocations.
- You accept that some weights remain in CPU memory and are evaluating a hybrid configuration rather than expecting the model to fit wholly in VRAM.
- You can test a specific model, context, build, and placement on your own system and check the resulting logs and throughput.
It is not established by the parameter count alone when
- A model’s active parameters per token appear close to what the GPU could hold: active count describes routing and computation, not the storage requirement for all weights.
- A quantized file seems to fit a nominal memory number: file size omits additional runtime needs and does not by itself prove the model will load.
- A community benchmark suggests a particular speed: a result from one branch and setup does not establish upstream support or comparable performance elsewhere.
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
