Windows Errors? Fix Them Before They Spread
Repair common Windows errors and clear accumulated junk for a smoother, more stable PC - no reinstall needed.Free scan · no reinstallCrashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minuteSome links on this page are affiliate links: if you buy through them we may earn a commission, at no extra cost to you.
Yes—an older laptop or desktop can run a local large language model if the model fits its available memory and storage. You do not need a graphics card to experiment: Ollama and llama.cpp can run inference on a CPU. Start with a small model, keep its context length modest, and expect slower responses on a CPU than on suitable GPU hardware.
What does it take to run an LLM on an older computer?
The first constraint is usually available RAM, not the age of the computer by itself. A smaller model can run on modest hardware; a model that exceeds the computer’s practical memory capacity may fail to load, run out of memory, or become frustratingly slow if work shifts onto system memory.
As an Amazon Associate I earn from qualifying purchases.
The Tool Desk
Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →Outbyte Driver Updater FREEScan for outdated or missing drivers - takes under a minuteDriver Scan →Ollama’s Quickstart recommends at least 8 GB of available RAM for 7B models, 16 GB for 13B models, and 32 GB for 33B models. Treat those figures as planning guidance, not a guarantee that every model of that size will run well: quantization, context length, operating-system use, and the individual computer all affect the result.
Check these four things before installing anything:
Popular on Amazon
- Spacious Design: Measuring 21.1" wide and 14.1" deep, our lap desk comfortably fits most laptops up to 15.6". Extra room for accessories ensures convenience.
- Enhanced Functionality: Packed with handy features, including a 5x9" precision tracking mouse pad and a built-in phone slot for seamless work or video calls. Plus, enjoy ergonomic support with the integrated cushioned wrist rest.
- Cool Comfort: Enjoy a stable surface with our lap desk's dual bolster cushion, designed for comfort and airflow, keeping your lap cool during extended use.
- Durable Surface: Work with confidence on our lap desk's solid surface, featuring a sleek black carbon color, ensuring optimal air circulation to prevent your laptop from overheating.
- On-the-Go Convenience: With an integrated handle and lightweight design (2.8 lbs), our lap desk is portable for travel or moving around the house, offering flexibility in any space.
- Available RAM: Look at both installed memory and what remains available when the operating system and your usual applications are running.
- Free storage: Reserve room for the runtime and downloaded model files. Ollama’s Windows documentation says the binary needs at least 4 GB; models can range from gigabytes to tens or hundreds of gigabytes.
- CPU and operating system: Check that the computer meets the chosen tool’s current requirements and supports the necessary CPU instructions. Requirements can vary by build and platform.
- GPU and VRAM, if present: A GPU can improve responsiveness when the model and its working data fit in video memory. An older or small-VRAM GPU is not automatically useful for every model.
Also consider cooling. Sustained inference can keep a laptop’s CPU or GPU busy, so unobstructed vents and a hard, stable surface matter more than they do during brief everyday tasks.
How much RAM do you need for Ollama?
Ollama’s published baseline is 8 GB of available RAM for 7B models, 16 GB for 13B, and 32 GB for 33B. For an older computer, those figures are a starting point: choose a smaller model if memory is shared with other applications or if you want to keep longer prompts in context.
The model’s download size is not the same as the amount of memory it needs while running. For example, Ollama lists Llama 3.2 1B at 1.3 GB and 3B at 2.0 GB, Gemma 2 2B at 1.6 GB, and Mistral 7B at 4.1 GB. Those are download sizes, not total runtime memory requirements. Inference also needs working memory, and larger context settings increase that pressure.
| Computer | Reasonable starting point | What to expect |
|---|---|---|
| 8 GB RAM, CPU only | 1B–4B quantized model | Basic chat and short prompts; generation may be slow. |
| 16 GB RAM, CPU only | 4B–8B quantized model | More capable chat, with patience required. |
| 16 GB RAM and 6–8 GB VRAM | 7B–8B quantized model | Potentially better responsiveness if the model fits in VRAM. |
| 32 GB or more RAM | 13B–14B quantized model | More model or context options, subject to the computer’s speed. |
These bands are practical planning guidance derived from Ollama’s RAM recommendations and listed model sizes, together with the general rule that the model should fit within available memory. They are not guarantees for a particular machine. Leave memory headroom for the operating system and other running applications rather than treating all installed RAM as available to the model.
Also popular on Amazon
- 5-in-1 Connectivity: Equipped with a 4K HDMI port, a 5 Gbps USB-C data port, two 5 Gbps USB-A ports, and a USB C 100W PD-IN port. Note: The USB C 100W PD-IN port supports only charging and does not support data transfer devices such as headphones or speakers.
- Powerful Pass-Through Charging: Supports up to 85W pass-through charging so you can power up your laptop while you use the hub. Note: Pass-through charging requires a charger (not included). Note: To achieve full power for iPad, we recommend using a 45W wall charger.
- Transfer Files in Seconds: Move files to and from your laptop at speeds of up to 5 Gbps via the USB-C and USB-A data ports. Note: The USB C 5Gbps Data port does not support video output.
- HD Display: Connect to the HDMI port to stream or mirror content to an external monitor in resolutions of up to 4K@30Hz. Note: The USB-C ports do not support video output.
- What You Get: Anker 332 USB-C Hub (5-in-1), welcome guide, our worry-free 18-month warranty, and friendly customer service.
What is the best small model for an 8 GB computer?
For a first try, choose a small model such as Llama 3.2 1B or Llama 3.2 3B. Ollama lists their downloads at 1.3 GB and 2.0 GB respectively. Those file sizes make them more practical starting points than a 7B model on an 8 GB computer, but they do not guarantee a particular speed or leave a fixed amount of memory free.
Model choice is a trade-off: smaller models are easier to fit, while a larger model may produce more capable responses but demand more memory and time. A model that technically loads is not necessarily pleasant to use. Try a short, repeatable prompt before deciding whether to move up in size.
Quantization reduces the memory used by model weights by storing them at lower precision, with some loss of accuracy. Docker’s inference-engine guide describes Q4_K_M as about 4.5 bits per weight and a good general balance. If a very low-bit model fits but its answers disappoint, try Q4 or Q5 if the computer has enough memory; do not assume the quantized version will produce identical output to a higher-precision one.
Can you run a local AI without a graphics card?
Yes. CPU-only inference is supported by both Ollama and llama.cpp; a dedicated graphics card is not a prerequisite for trying a local model. The trade-off is responsiveness: on older CPUs, generation can be slow, especially with larger models or longer context.
Also popular on Amazon
- Note: Not suitable for MacBooks released after 2023 or devices with a protruding front camera; Not applicable to full-screen or notch-style tempered glass screen protectors; Do not use on the rear camera of the phone.
- 💻 Why Do You Need a Webcam Cover Slide? — Safeguard your privacy by covering your webcam with our reliable webcam cover when not in use. Don't let anyone secretly watch you. Stay protected!
- ✅ Thin & Stylish — Enhance your laptop's functionality and aesthetics with our 0.027" ultra-thin webcam covers. Seamlessly close your laptop while adding a touch of sophistication.
- ✅ Fits Most Devices — Compatible with laptops, phones, tablets, desktops! Keep your privacy intact on Ap/ple, Mac/Book, iPh/one, iP/ad, H/P, L/novo, De/ll, Ac/er, As/us, Sa/msung devices.
- ✅ 365 Days Protection — Our upgraded 3.0 adhesive ensures a strong hold that won't damage your equipment. Experience reliable, long-term privacy protection day in and day out.
A GPU helps most when the chosen model can use the available GPU memory. Ollama’s ollama ps command shows whether a loaded model is using 100% CPU, 100% GPU, or a split between them. A split can mean some of the work remains on the CPU, so having a GPU does not by itself guarantee that the model is fully accelerated.
llama.cpp supports CPU inference as well as multiple GPU and accelerator paths, including NVIDIA, AMD, Apple Silicon, and Vulkan. Which backend works depends on the specific hardware, drivers, and build. Docker’s inference-engine guidance also lists these options; check the current requirements for your device instead of assuming that every graphics card supports every backend.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Should you use Ollama or llama.cpp?
Choose based on whether you value a guided setup and convenient model management or more direct control over inference options. Both can run on a CPU; the practical differences are in setup, configuration, and how you inspect or manage the run.
| Consideration | Ollama | llama.cpp |
|---|---|---|
| Getting started | Offers a straightforward installer and commands such as ollama run. |
The project aims for inference with minimal setup across a wide range of hardware; the exact setup depends on the build and backend. |
| Model management | Provides named model pulls and a managed local model library. | Offers direct inference tooling; the setup and model-file handling depend on the chosen build and workflow. |
| Hardware paths | Can use CPU and supported GPU paths; actual placement depends on the machine and configuration. | Lists CPU inference and NVIDIA, AMD, Apple Silicon, and Vulkan support; support is hardware- and build-specific. |
| Quantization | Documents quantization and provides model variants; available options depend on the model. | Offers access to quantized model workflows; the amount of control depends on the selected tooling and model files. |
| CPU/GPU placement visibility | ollama ps reports CPU, GPU, or split placement. |
Placement and performance checks depend on the build and the method used to run it. |
| API access | Provides an API as part of the Ollama service. | API availability depends on the server or integration used with the selected build. |
Ollama is a sensible first choice if you want to try a model with a small number of commands and inspect placement using ollama ps. Pick llama.cpp if you want to work directly with its inference tools or a particular supported backend. The project README describes its goal as enabling LLM inference with minimal setup and state-of-the-art performance across a wide range of hardware; that goal is not a speed guarantee for an individual older computer.
Also popular on Amazon
- Anti-Slip Surface - Transform your laptop into a mobile workstation with the AboveTEK portable laptop lap desk. The anti-slip surface provides a strong grip for laptops up to 15.6 inches(Diagonal), while the double rubber strip on the bottom ensures a stable display or typing experience on your lap, couch, or bed.
- Retractable Mouse Pad - Retractable laptop mouse pad extends on both directions for the left/right handed with elevation along the edges for stopping mouse from falling off. The size of laptop tray is 14" X 9.7" and the size of mouse pad is 7.4" X 6.1".
- Effective Heat Shield - The effective heat shield made of sturdy and thick material protects your laptop from overheating. Prioritizes your comfort and safety, an ideal lap pad or board for working anywhere.
- EASY to Carry and Store - With an ergonomic and simplistic design, the lap desk is portable to store in a backpack. Only 15" in size, 2.2 lb of weight and with slim 0.6 inch thickness, it is ready to be easily carried around.
- Widely Applicable - The smooth platform accommodates laptops and tablets up to 15.6 inches(Diagonal), making it a versatile accessory and one of the best gifts for mom, dad, students and professionals. Perfect for use as a laptop bed tray or tablet holder anywhere at home, library, or park.
How do you set up a small local model?
Install one runtime, download one small model, and test it before changing settings or adding more models. The commands and platform details below reflect the cited Ollama documentation; installer and model labels can change, so check the current official instructions for your operating system.
- Check capacity first. Note installed and available RAM, free disk space, operating system, CPU support, and GPU VRAM if applicable. Close memory-heavy applications for the first test.
- Install a runtime. Use Ollama’s official installer. Its Linux Quickstart provides
curl -fsSL https://ollama.com/install.sh | sh. The cited Windows documentation says the native application supports Windows 10 22H2 or newer. If you prefer direct inference tools, set up llama.cpp using the project’s current instructions for your platform and hardware. - Start with a small model. In a terminal, run
ollama run llama3.2:1bfor the 1B variant orollama run llama3.2for the default Llama 3.2 model. The first run downloads the model, so allow for its file size and the time needed to fetch it. - Move model storage if necessary. If the internal drive is tight, Ollama’s
OLLAMA_MODELSsetting can point its model directory to another drive. Use a drive with enough free space; an external USB 3.x SSD is one option when the computer’s internal storage is limited. - Keep the first test modest. Use a short prompt and the default context before changing settings. Ollama’s FAQ documents a 2048-token default in the cited page and shows how to set
num_ctx. A larger context gives the model more text to work with but increases memory pressure. - Check where the work is happening. While a model is loaded, run
ollama ps. Read whether the model is on CPU, GPU, or split across them. Record the model, prompt, context setting, and placement so you can compare later tests on the same basis. - Keep the run simple. Stop models you are not using and avoid loading several models at once on a memory-constrained computer. Keep laptop vents clear during sustained generation.
Why is a local model so slow?
There is no universal tokens-per-second figure that predicts performance on every older computer. Speed varies with CPU or GPU, model architecture and size, quantization, context length, and backend. The useful first step is to identify whether the model is running on CPU, GPU, or both, then change one constraint at a time.
- The model is too large for the machine. Try a smaller parameter count or a lower-bit quantization. If quality suffers too much, return to a better-balanced quantization and shorten the context instead.
- The context is longer than you need. Reduce
num_ctxor use shorter prompts. Context length adds to memory pressure even when the downloaded model file looks small. - The GPU is not being used as expected. Check
ollama psfor CPU/GPU placement, then confirm that the runtime’s backend and required drivers support the specific hardware. A model that does not fit available VRAM may be split with system memory. - Other applications are competing for memory or processor time. Close unnecessary work, stop unused models, and test again with the same prompt. This helps separate a resource bottleneck from a change in model or settings.
- The computer is heat-constrained. Keep vents clear and avoid soft surfaces that block airflow. If performance changes during a long run, compare a short test with a sustained run under the same settings.
Do not compare two runs unless the model, prompt, context, and backend are the same. A larger model may answer better but generate more slowly; a lower-bit quantization may fit more easily but reduce answer quality. Those are trade-offs to measure on your own setup, not problems that a single universal speed number can settle.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.What can you upgrade if the first test is not usable?
Make the cheapest reversible change first: reduce the model size or context before buying hardware. If storage is the obstacle, move the model directory to a suitable SSD. If memory is the obstacle and the computer supports an upgrade, verify the exact RAM generation, slot count, and maximum supported capacity before choosing memory.
Also popular on Amazon
- Spacious Design: Measuring 21.1" wide and 12" deep, our lap desk comfortably fits most laptops up to 15.6". Extra room for accessories ensures convenience.
- Enhanced Functionality: Packed with handy features, including a 5x9" precision tracking mouse pad and a built-in phone slot for seamless work or video calls. Plus, enjoy laptop support with the integrated device ledge.
- Cool Comfort: Enjoy a stable surface with our lap desk's dual bolster cushion, designed for comfort and airflow, keeping your lap cool during extended use.
- Durable Surface: Work with confidence on our lap desk's solid surface, featuring a blush pink color, ensuring optimal air circulation to prevent your laptop from overheating.
- On-the-Go Convenience: With an integrated handle and lightweight design (2.14 lbs), our lap desk is portable for travel or moving around the house, offering flexibility in any space.
A GPU upgrade is harder to justify for an experiment on an older computer. It only helps if the model can use that GPU through a supported backend and has enough VRAM for useful placement; the value also depends on compatibility, power, cooling, noise, and total upgrade cost. A refurbishing or component decision should be based on the machine’s exact specifications and the tasks you expect it to handle, not on a promise of a particular generation speed.
What are the key limitations of a home LLM setup?
A local model is a practical way to experiment, but it is not automatically fast, capable, or private in every configuration. This guide’s hardware bands are planning estimates, not performance benchmarks. Model sizes, runtime defaults, driver requirements, and backend support can change between releases.
Free tools Windows power users keep installed
One-click scans. No signup required.
For a fair assessment, use the same prompt, model, context length, and backend for repeated tests. Note the answer quality as well as the time it takes to respond; the smallest model may be adequate for short, straightforward tasks, while more demanding work may justify more memory or a different machine.
Frequently Asked Questions
Do I need an internet connection every time I use a local LLM?
You need a connection for installation and for downloading a model the first time. Keep the required runtime and model files on the computer if you want to try using them without reconnecting; the cited setup documentation does not specify offline behavior for every version or configuration.
Can I use a local LLM on a laptop with integrated graphics?
A dedicated graphics card is not required: CPU inference is supported. Whether integrated graphics can accelerate a particular model depends on the runtime’s supported backend, drivers, and available memory.
Can I use my local model from another app?
Ollama provides an API, so a compatible application can connect to its service. The exact integration steps depend on that application and its current Ollama support.
Recommended Free Tools
Will a local model keep my conversations private?
Running inference locally means the model is processed on your computer, but privacy also depends on the applications and services you connect to it. Check those tools’ data handling and avoid sending sensitive information to integrations you have not assessed.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
