Quick wins for a faster PC:
Clear out junk files and repair common Windows errorsFree Scan →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Fix Raspberry Pi 5 local inference by matching the model to available memory, keeping the SoC cool, using an adequate power supply, and checking the serving runtime’s configuration. First identify whether generation is always slow, slows after sustained work, fails during loading, loses the model, or reboots the board; each pattern has a different remedy. A Pi 5 can run useful local models, but there is no universal tokens-per-second figure: model size, quantization, context length, runtime, cooling, clock speed and background services all matter.
Start with the failure pattern
Do not begin by changing random flags. Record what happens and when it happens. The timing usually narrows the cause before you touch the model.
As an Amazon Associate I earn from qualifying purchases.
| What you see | Most likely areas to check first |
|---|---|
| Generation is consistently slow from the first token | Model size, quantization, context length, CPU thread settings, runtime choice and realistic Pi 5 performance expectations |
| It starts acceptably, then slows after several minutes | Temperature, fan operation, airflow and thermal throttling; then memory pressure and swapping |
| The model never appears or the server cannot be reached | Model directory and file layout, router mode, autoload settings, loaded-model state and health/models endpoints |
| Loading fails, the desktop freezes or requests are killed | RAM headroom, context size, KV-cache growth, other loaded models and background services |
| The Pi reboots or reports a power warning | USB-C supply rating, cable and connector, and the current drawn by attached USB devices |
Before changing anything, note the exact model file and quantization, context setting, runtime, number of threads, cooling arrangement, Pi 5 RAM capacity and whether the problem appears only under sustained generation. That record lets you compare one change at a time.
Make a safe baseline before tuning
- Stop extra workloads. Close browsers, desktop applications, compilers and containers that are not needed for inference. Unload any other model in the serving runtime.
- Check memory. Use a system monitor or
free -hwhile the model is loaded and while it is generating. Distinguish RAM used by the model from memory still available to the operating system. - Measure temperature during generation. Run
vcgencmd measure_temprepeatedly, not only when the board is idle. Raspberry Pi notes that Linux temperature measurements can be inaccurate and recommends this command for an instantaneous SoC reading. - Record power symptoms. Note under-voltage warnings, USB disconnects and the exact point at which a reboot occurs. A reboot under load is a different problem from low tokens per second on a stable system.
- Change one variable. Keep a short log of model, context, runtime, clock profile, temperature and observed prompt and generation rates. Otherwise an apparent improvement may come from an unrelated change.
Fix slowdowns caused by heat
A Pi 5 may generate normally for a short prompt and then become slower as sustained CPU work raises the SoC temperature. Raspberry Pi’s documentation states that a heatsink or small fan can reduce thermal throttling and improve performance. Official Pi 5 options include the Active Cooler and the Raspberry Pi 5 case fan.
#1 Best Overall
- Official RPi 5 Active Cooler -- This is Official RPi Active Cooler for the latest RPi 5 4GB/8GB Board
- Composition--The RPi 5 Active Cooler is composed of Temperature-controlled Blower Fan and Aluminium Heatsink and comes with Thermal Tapes to accelerate heat dissipation
- Input Voltage--5V DC (supplied via four-pin fan header on RPi 5)
- How to Install-- Connect the 4pin cable to the fan header on RPi 5, and fix the Active Cooler via spring-loaded push pins
- NOTE -- RPi 5 Board is NOT Included
Check cooling while the model is busy
- Start a representative generation request, long enough to reproduce the slowdown.
- Run
vcgencmd measure_tempduring the fast phase and again after the rate falls. - Confirm that the fan is connected to the intended header, spins under load and is not blocked by a case panel or dust.
- Improve ventilation around the board and keep the Pi away from heat sources. If the board is in an enclosed case, test briefly with the cover removed to separate airflow from software causes.
- Repeat the same prompt and context after installing or correctly connecting an Active Cooler or case fan.
A broad Raspberry Pi hardware document describes progressive Arm throttling in the 80°C to 85°C range and says that at 85°C both Arm cores and the GPU are throttled. That passage is not presented as a Pi 5-specific universal trigger, so use the current board’s observed behavior rather than treating 85°C as a guaranteed threshold. The practical test is whether better cooling prevents the frequency drop and restores sustained throughput.
Cooling cannot make an oversized model fit into RAM, remove CPU inference limits or compensate for an inefficient context configuration. If temperature is stable while generation remains slow, continue with model and runtime checks.
Fit the model, quantization and context to RAM
Model weights are only part of the memory requirement. The runtime also needs working space, the operating system needs headroom, and the key-value (KV) cache grows with context length and active sequences. A model can load successfully yet leave so little free memory that the system swaps, becomes unresponsive or kills the process.
Use the 8 GB Qwen3 example as a scale reference, not a specification
A community guide measured an 8 GB Pi 5 running Qwen3-8B Q4_K_M. The model file was reported as 4.68 GiB, and approximately 5.2 GB was reported as used while serving. The guide treated a 4096-token context as a sensible target for that particular test system and warned that larger contexts require more KV-cache memory and can reduce responsiveness or trigger swapping. Those figures describe one model, quantization, runtime and configuration; they do not establish the RAM requirement for every 8B model or every Pi 5.
Rank #2
- This is Official Active Cooler for Raspberry Pi 5
- Combines an Aluminium Heatsink with a Temperature-Controlled Blower Fan to accelerate heat dissipation
- How to Install: Connect the 4pin cable to the fan header on RPi 5, and fix the Active Cooler via spring-loaded push pins
Reduce memory pressure in a controlled order
- Unload other models. A second resident model can consume the headroom needed by the first, even if neither is generating.
- Lower the context. In llama.cpp-based serving, the router guidance specifically recommends lowering
-cwhen loading fails or memory use is excessive. Start with the smallest context that supports your task, then increase it only when memory remains comfortable. - Choose a smaller or more aggressively quantized model. Quantization reduces weight memory, with a corresponding trade-off in quality and sometimes speed. Compare models at the same context rather than assuming a larger parameter count is automatically better.
- Stop background services. Web servers, databases, desktop sessions and build jobs compete for RAM and can cause swapping during a long prompt.
- Watch for swap activity. If free memory collapses and the system begins swapping, lowering context or model size is usually more effective than adding storage. An SSD can provide space for model files, but it does not turn CPU-bound token generation into a faster workload.
If a load fails immediately, first verify that the model file is complete and that the runtime supports its format. If it loads and later freezes, treat memory headroom and KV-cache growth as the primary suspects before blaming temperature.
Repair model discovery and server configuration
A missing model and a slow model are different failures. Runtime-specific settings determine whether a file is visible, whether it is loaded, and whether requests reach the correct server.
llama.cpp router checks
The llama.cpp router guidance calls out several easy-to-miss conditions:
- Confirm the configured models directory and the expected directory and file layout. A file placed one level too deep, or outside the configured directory, may not be discovered.
- Restart the router after adding or renaming model files when the runtime does not rescan automatically.
- Start router mode without
--model,-mor-hfwhen the router is expected to manage multiple models. Supplying one of those model-selection options can bypass the intended router behavior. - If the server uses
--no-models-autoload, a model will not appear ready merely because its file exists. Load it through the router before selecting it for a request. - Check the runtime’s health and models endpoints. A healthy process with an empty model list indicates discovery or loading configuration; an unhealthy endpoint indicates a server or process problem.
Apply these checks to llama.cpp only. Do not copy its flags into another server and assume they have the same meaning.
Rank #3
- Compatible with Raspberry Pi 5 --- This Armor Lite V5 Aluminum Heatsink is only designed for Raspberry Pi 5 4GB/8GB.
- Support PWM Speed Control --- Different from ordinary fans, this cooling fan supports PWM speed regulation, which is perfectly compatible with Raspberry Pi OS.
- Good Heat Dissipation Effect --- With 3510 ultra-quiet cooling fan and thermal pads, it can lower the temperature of Raspberry Pi Board quickly.
- Lightweight and Easy to Install --- With screwdriver and 2pcs screws, it's easy to fix the heatsinks with Raspberry Pi Board.
- Package Includes: 1 x Armor lite V5 for Raspberry Pi 5, 1 x Screw driver, 2 x Screws, 4 x Thermal Pads, 1 x User Manual;
Ollama and llama.cpp: choose by the control you need
Raspberry Pi Foundation project material documents a local setup using Ollama. Separate Pi 5 technical guidance describes llama.cpp when direct control over runtime settings and benchmarking is important. The available material does not provide a controlled head-to-head speed test, so neither should be declared categorically faster.
| Decision factor | Ollama | llama.cpp |
|---|---|---|
| Initial setup | Often simpler for downloading, naming and serving models through its packaged workflow | Requires more explicit attention to binaries, model files and server options |
| Configuration control | Convenient defaults, with fewer low-level knobs exposed in the normal workflow | Fine-grained control over context, router behavior, threads, slots and model loading |
| Diagnostics | Use its own service status, model listing and logs | Router health/models endpoints and explicit model-directory settings make discovery state visible |
| Compatibility | Depends on the model formats and tags supported by the installed release | Depends on the llama.cpp build and the model formats it supports |
| Benchmarking | Measure through the exact API or interface you will use | Can be measured through the serving path and with llama-bench |
Use the runtime that exposes the controls and model compatibility your project needs, then benchmark that exact path. Switching runtimes can help when a configuration is limiting you, but it is not proof that one runtime always delivers higher throughput on every Pi 5.
Rule out power problems before changing performance settings
Unexpected reboots, under-voltage warnings and USB devices disconnecting during inference point to power delivery. Raspberry Pi recommends its 27 W USB-C power supply for Raspberry Pi 5 and documents a recommended 5 A capacity. With a 5 A supply, the board’s maximum downstream USB peripheral allowance is 1.6 A; with a 3 A supply it is 600 mA.
PC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11Crashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minute- Use a supply and cable suitable for the Pi 5 recommendation, and reseat the USB-C connector.
- Temporarily disconnect high-draw USB storage, radio adapters and other peripherals. If reboots stop, add devices back one at a time.
- Check for power warnings in the operating system and correlate them with the generation request.
- Retest with the same model and context after correcting the supply.
A stronger supply is not a general speed upgrade. If the Pi is stable, has no power warnings and simply generates tokens slowly, investigate model size, context, cooling and runtime settings instead.
Rank #4
- KEEP YOUR RASPBERRY PI 5 COOL UNDER HEAVY LOADS - Designed for Raspberry Pi 5 4GB and 8GB models, the iUniker ICE PEAK Active Cooler helps move heat away during demanding workloads such as Home Assistant, Pi-hole, media servers, coding, retro gaming, and light AI projects.
- QUIET PWM FAN WITH ACTIVE AIRFLOW - The 3510 PWM fan adjusts speed based on Raspberry Pi 5 CPU temperature through Raspberry Pi OS. It runs quietly during light tasks and increases airflow when extra cooling is needed.
- ALUMINUM HEATSINK FOR HEAT DISSIPATION - The black aluminum heatsink helps spread heat across the finned surface, while the fan moves airflow through the heatsink to support stable cooling performance during extended use.
- LOW-PROFILE DESIGN FOR EXPANSION PROJECTS - The compact Pi 5 heatsink fan design helps leave more room for many Raspberry Pi 5 HATs, NVMe HATs, GPIO expansion boards, and stacked DIY builds. Please check clearance before use.
- EASY INSTALLATION WITH DIRECT FAN HEADER CONNECTION - We have pre-applied thermal grease on the heatsink. You only need to peel off the release paper and secure two screws to enjoy convenient cooling.
Set realistic speed expectations with a repeatable benchmark
One documented field test used an 8 GB Pi 5 with four Cortex-A76 cores, active cooling, the llama.cpp CPU backend, Qwen3-8B Q4_K_M, four threads and one parallel slot. In web-UI tests at 2.6 and 2.8 GHz, it measured approximately 2.13 to 2.15 generated tokens per second. At a reported 3.0 GHz profile, two llama-bench runs measured 2.30 to 2.45 generated tokens per second. Prompt processing in those runs was approximately 10.56 to 11.18 tokens per second in the web UI and 11.45 to 11.50 tokens per second in the 3.0 GHz benchmark.
These are sourced results for that exact setup, not a typical-speed guarantee. A different model, quantization, prompt length, context, build, thread count, cooling arrangement or Pi variant can produce a different result. Prompt-token rate and generated-token rate are separate measurements; do not substitute one for the other when evaluating responsiveness.
Benchmark your own workload
- Fix the model file and quantization, context length, thread count and parallel-slot setting.
- Warm the system, then run the same prompt several times so startup and model-loading time are not confused with generation speed.
- Record prompt tokens per second and generated tokens per second separately.
- Record temperature at the start and end, clock profile, memory use and whether swapping occurred.
- Repeat after one change—such as a fan, smaller context or different quantization—and keep all other variables fixed.
This method tells you whether a change improved sustained generation, merely shortened model loading, or traded quality for memory. It also gives you a defensible baseline when software updates alter performance.
A recovery sequence for a Pi that is slow, stuck or failing
Use this order when you need a practical decision path rather than a full investigation.
- Stabilize the board. Disconnect unnecessary USB devices, close competing applications and ensure the recommended power supply is firmly connected.
- Measure temperature under load. If the rate falls as temperature rises, fix the fan or airflow and retest before changing the model.
- Unload and reload one model. Confirm that the model file is complete, visible in the configured directory and actually loaded by the runtime.
- Lower context. Reduce
-cin a llama.cpp configuration, or use the equivalent setting in your runtime. Retest with the same prompt. - Check memory and swapping. If available RAM is marginal, move to a smaller model or lower quantization and stop unnecessary services.
- Validate server state. For a llama.cpp router, check health and models endpoints, router mode, autoload behavior and the model-list result. For another runtime, use that runtime’s own status and logs rather than llama.cpp flags.
- Benchmark the stable configuration. Record the exact settings so future changes can be compared rather than guessed.
Prevent the problem from returning
- Keep active cooling and unobstructed airflow for sustained inference.
- Reserve RAM for the operating system instead of configuring the largest context that happens to load once.
- Run one model at a time unless you have measured the memory cost of parallel serving.
- Keep model directories organized and restart or rescan the server after file changes.
- Use the Pi 5 power recommendation when attaching USB peripherals, not only when the board is idle.
- Save benchmark notes with model name, quantization, context, runtime version, threads, slots, clock, cooling and measured rates.
The reliable fix is usually not a single magic setting. A stable Pi 5 inference setup is one in which the model fits with memory headroom, the SoC maintains its clock under sustained load, power remains within the board’s limits, and the serving runtime can see and load the intended model.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
