Outdated Drivers Are Slowing You Down
One free scan finds every outdated or missing driver and matches the right update for your exact hardware.Free scan · exact hardware matchPC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11A “Max” GPU-layer setting is a requested ceiling, not proof that every model layer landed in VRAM. To see where the model actually runs, check the loader’s startup report for its offloaded-layer count. A report of 54/65 confirms partial GPU placement for that load; by itself, it does not explain why.
What does “Max” mean when GPU offload stops at 54 of 65?
In llama.cpp, the GPU-layer option sets the maximum number of layers to store in VRAM. Its documented values include a number, auto, and all. The requested value and the result are different things: the startup/load report is the evidence of how many layers the runtime actually placed on the GPU. llama.cpp server README
As an Amazon Associate I earn from qualifying purchases.
So “54 of 65” means that the runtime reported offloading 54 of the model’s 65 layers in that load. The other layers remain on the CPU in a hybrid placement. That count does not establish whether memory capacity, fitting behavior, split configuration, or another setting caused the outcome.
How do I check whether my LLM is running on GPU or CPU?
- Identify the loader and version. Note whether the model was loaded by llama.cpp, llama-cpp-python, or another application. A “Max” control in one interface should not be assumed to have the same defaults or reporting convention as a similarly named control elsewhere.
- Find the startup or model-load report. Record the requested GPU-layer value and the actual loaded/offloaded count. Use the count from the same load rather than the UI setting alone.
- Record the conditions for that load. Note the GPU model, available VRAM at load time, model file and quantization, context size, batch settings, and any other GPU workloads running concurrently. Without those details, a partial count cannot be attributed to a particular cause.
- Check the runtime’s own configuration. For llama.cpp, inspect the GPU-layer option and whether automatic fitting is enabled. If multiple GPUs are involved, check the split mode and tensor-split settings. For llama-cpp-python, inspect
n_gpu_layers, split configuration, context and batch settings, and verbose output. - Change one setting at a time and reload. Compare the next startup report against the original. This makes it possible to see whether that specific change affected placement.
Which controls should I inspect?
| Runtime or setting | What it controls | What to verify |
|---|---|---|
llama.cpp GPU-layer option: -ngl, --gpu-layers, or --n-gpu-layers |
Maximum number of layers to store in VRAM; documented values include a number, auto, and all. |
Compare the requested value with the actual offloaded count in the load report. llama.cpp server README |
llama.cpp automatic fitting: --fit, --fit-target, and --fit-ctx |
--fit adjusts unset arguments to fit device memory; --fit-target sets a per-device memory margin; --fit-ctx sets a minimum context size used by fitting. |
Check whether fitting is enabled and which relevant arguments are set. Do not infer that fitting caused a particular layer count without comparing the configuration and load report. llama.cpp server README |
| llama.cpp multi-GPU split settings | Split mode and tensor split configure how work is distributed across multiple GPUs. | If more than one GPU is used, inspect the split configuration alongside the per-load report. llama.cpp server README |
llama-cpp-python: n_gpu_layers |
Number of layers to put on the GPU, with the rest on the CPU; -1 requests all layers. |
Check the configured value and verbose model-load output; also review split, context, and batch settings. llama-cpp-python API reference |
Why can’t the 54/65 count identify the cause?
The count describes placement, not the reason for it. The title does not specify the runtime and version, GPU and available VRAM, context and batch settings, other GPU use, or the startup log. Those details are needed to diagnose a specific load. In particular, do not treat a partial count alone as proof that the GPU is undersized or that a particular fitting or split option is responsible.
#1 Best Overall
- Axial-tech fans now feature a smaller fan hub that facilitates longer blades and a barrier ring that increases downward air pressure
- 2.5-slot design allows for greater build compatibility while maintaining cooling performance
- 0dB technology lets you enjoy light gaming in relative silence
- Dual BIOS switch lets you toggle between Quiet and Performance BIOS profiles
- Dual ball fan bearings last up to twice as long as sleeve bearing designs
How should I describe a partially offloaded model?
Use the load report’s actual count. If it reports 54/65, describe that load as partially offloaded or hybrid: 54 layers were reported offloaded and some work remains on the CPU. Avoid saying simply that the model “runs on the GPU” if the report shows layers were not offloaded.
Quick Recap
Best Value
- Axial-tech fans now feature a smaller fan hub that facilitates longer blades and a barrier ring that increases downward air pressure
- Phase-change GPU thermal pad helps ensure optimal heat transfer, lowering GPU temperatures for enhanced performance and reliability
- 2.5-slot design allows for greater build compatibility while maintaining cooling performance
- Dual-ball fan bearings last up to twice as long as standard conventional sleeve bearings designs
- 0dB technology lets you enjoy light gaming in relative silence
Rank #4
- Powered by Radeon RX 9070 XT
- WINDFORCE Cooling System
- Hawk Fan
- Server-grade Thermal Conductive Gel
- RGB Lighting
Rank #3
- Powered by the NVIDIA Blackwell architecture and DLSS 4
- Powered by GeForce RTX 5060
- Integrated with 8GB GDDR7 128bit memory interface
- PCIe 5.0
- WINDFORCE cooling system
Rank #2
- Powered by the NVIDIA Blackwell architecture and DLSS 4
- Powered by GeForce RTX 5070 Ti
- Integrated with 16GB GDDR7 256bit memory interface
- PCIe 5.0
- WINDFORCE cooling system
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
