You can run an AI model on your own computer by installing a local runtime, downloading model weights it supports, and loading them in a chat app or local server. For the easiest first run, use LM Studio; for a terminal-based workflow, use Ollama or llama.cpp. Whether a model fits and runs smoothly depends on its format, quantization, context length, and your computer’s available memory—not just its parameter count.
“Open-source” is often used loosely for models whose weights can be downloaded. Weight access does not mean every model has the same license or degree of openness, so check the exact model and its terms before using it, especially for commercial work.
As an Amazon Associate I earn from qualifying purchases.
What you need to run a model locally
A local setup has three parts: a runtime, model weights, and enough free disk space and memory to download and load them. The runtime is the software that runs inference; it is separate from the model. Installing an app does not automatically mean a particular model is installed or compatible.
Quick wins for a faster PC:
Clear out junk files and repair common Windows errorsFree Scan →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →- A runtime: LM Studio provides a graphical interface; Ollama offers a command-line workflow and local API; llama.cpp supports command-line use and a server.
- Compatible weights: The file format must be supported by your chosen runtime. LM Studio commonly uses model files such as
.ggufor.safetensors; llama.cpp requires GGUF files. - Resources: You need disk space for the download and enough available RAM or GPU memory to load the model, with additional memory needed for the context window and other runtime overhead.
For a first try on an ordinary laptop, start with a smaller instruction-tuned model. Move to a larger or more capable model only if your task needs it and your hardware can accommodate it.
#1 Best Overall
- EVOLUTION AMD RYZEN AI MAX+ 395 MINI PC - GMKtec EVO-X2 is the next evolution in AI mini PC Ryzen Strix Halo series. Thanks to AMD Simultaneous Multithreading (SMT) the core-count is effectively doubled, to 32 threads. Ryzen AI Max+ 395 has 64 MB of L3 cache and can boost up to 5.1 GHz, depending on the workload. The Ryzen AI Max+ 395 is currently rated as the "most powerful x86 APU" on the market for AI computing.
- AI NPU with XDNA 2 ARCHITECTURE - Powered by 16 “Zen 5” CPU cores, 50+ peak AI TOPS XDNA 2 NPU and a truly massive integrated GPU driven by 40 AMD RDNA 3.5 CUs, the Ryzen AI MAX+ 395 is a transformative upgrade and delivers a significant performance boost over the competition. The Ryzen AI Max+ 395 excels in consumer AI workloads like the llama.cpp-powered application: LM Studio. Shaping up to be the must-have app for client LLM workloads, LM Studio allows users to locally run the latest language model without any technical knowledge required and unleash their creativity and productivity.
- AMD RADEON 8090S iGPU GAMING PC - The AMD Radeon RX 8060S offers all 40 CUs with up to 2.9 GHz graphics clock and uses the new RDNA 3.5 architecture. The powerful iGPU is positioned between an RTX 4060 and 4070 laptop GPU and therefore enables gaming in FHD at maximum details in most demanding games. The 8060S can also utilize the full 128GB pool, which is perfect for running LLMs such as Deepseek 70B Q8, which runs comfortably on this machine.
- EIGHT CHANNEL LPDDR5X - LPDDR5X is a new ground breaking memory small form factor installed on-board. With blazing speeds up to to 8000MT/s, it runs 1.5x faster than the DDR5 SODIMMs; 90% better performance over DDR5 SODIMMs in video conferencing and photo editing; 30% better performance in productivity apps; 12% better performance in digital content workloads.
- QUAD SCREEN 8K DISPLAY SUPPORT - EVO-X2 AI Mini PC support 4-screen 4K/8K output via HDMI 2.1 (8K@60Hz), DisplayPort 1.4 (4K@60Hz), and dual USB 4 40Gbps Transfer speed (supporting PD3.0/DP1.4/DATA). Ideal for gaming, video editing, and multitasking, it provides expansive and crisp multi-display support.
How to run a model with LM Studio
LM Studio is a practical starting point if you prefer to choose and load a model through a graphical interface.
- Check system requirements. LM Studio’s current requirements recommend 16 GB or more of RAM for Apple Silicon Macs running macOS 14 or newer. The page says Macs with 8 GB may work with smaller models and modest context settings. On Windows, LM Studio supports x64 and Snapdragon X Elite ARM systems; x64 requires AVX2. It recommends 16 GB RAM and at least 4 GB dedicated VRAM. These are runtime recommendations, not a guarantee that every model will fit or run well.
- Install LM Studio. Download and install the version for your operating system from the LM Studio website.
- Find and download a model. Open Discover, search for a model, and select a compatible variant. Check its format and license information before downloading.
- Load the model. Open Chat and select the downloaded model in the loader. Loading allocates memory for the weights and other settings, so it can fail if too little memory is free or the selected context is too large.
- Start a conversation. Send a prompt. If loading fails or performance is poor, try a smaller or more heavily quantized variant, lower the context setting, or close other memory-intensive applications.
How to run a model with Ollama
Ollama is a command-line option for downloading and running models. Because its library changes, choose a current model and variant in the Ollama model library and use the command shown for that entry rather than relying on a model name copied from an older guide. A catalog listing is not proof that the originating model is fully open source; check its model card and license.
- Install Ollama. Follow the current instructions for your operating system at Ollama’s download page.
- Select a model. Find a suitable model in the library, review its variant and terms, and copy its current run command.
- Run it from a terminal. Enter the command listed for that model. Ollama downloads the required files if needed and starts an interactive session.
- Check storage before downloading. Ollama’s Windows documentation says its binary needs at least 4 GB and downloaded models can require tens to hundreds of GB. If internal storage is limited, the documentation describes moving the model directory by setting
OLLAMA_MODELS.
On Windows, Ollama runs as a native application and documents a local API at http://localhost:11434. Keep that endpoint local unless you understand the authentication and access controls needed to expose a service beyond your computer.
Recommended Free Tools
Rank #2
- Built for Local AI Development: AMD Ryzen AI Halo is designed for local AI development and inference, featuring 128GB unified memory and support for up to 200B parameter models to build and run intensive AI workloads locally.
- 128GB Unified Memory: Features 128GB LPDDR5x unified memory at 8000 MT/s with 256 GB/s memory bandwidth, providing a shared memory pool across the CPU, GPU, and NPU to support larger AI models.
- AMD Ryzen AI Max+ 395 Processor: Features 16 cores, 32 threads, and Zen 5 architecture, paired with AMD Radeon 8060S integrated graphics featuring 40 RDNA 3.5 compute units and an AMD XDNA 2 NPU with up to 50 TOPS.
- Linux AI Developer Platform: Purpose-built for Linux-based AI development with full AMD ROCm software support and preloaded tools, models, and workflows optimized for local AI development.
- Compact, Connected Design: Includes a 2TB M.2 SSD, 10GbE LAN, Wi-Fi 7, Bluetooth 5.4, USB-C connectivity, and HDMI 2.1b.
How to use llama.cpp from the command line or as a server
llama.cpp is a lower-level option for people comfortable with command-line tools. It can be installed through a package manager, Docker, a prebuilt release, or a source build. The project supports CPU and multiple accelerator backends, including hybrid CPU/GPU inference; the appropriate installation and backend depend on your platform.
Use a compatible GGUF model file. The project README documents a local-file example:
llama-cli -m my_model.gguf
It also documents a server example that fetches a model using Hugging Face syntax:
Rank #3
- EVOLUTION RYZEN AI MAX+ 395 MINI PC - GMKtec EVO-X2 is the next evolution in AI mini PC Ryzen Strix Halo series. Thanks to AMD Simultaneous Multithreading (SMT) the core-count is effectively doubled, to 32 threads. Ryzen AI Max+ 395 has 64 MB of L3 cache and can boost up to 5.1 GHz, depending on the workload. The Ryzen AI Max+ 395 is currently rated as the "most powerful x86 APU" on the market for AI computing.
- AI NPU with XDNA 2 ARCHITECTURE - Powered by 16 “Zen 5” CPU cores, 50+ peak AI TOPS XDNA 2 NPU and a truly massive integrated GPU driven by 40 AMD RDNA 3.5 CUs, the Ryzen AI MAX+ 395 is a transformative upgrade and delivers a significant performance boost over the competition. The Ryzen AI Max+ 395 excels in consumer AI workloads like the llama.cpp-powered application: LM Studio. Shaping up to be the must-have app for client LLM workloads, LM Studio allows users to locally run the latest language model without any technical knowledge required and unleash their creativity and productivity.
- AMD RADEON 8090S iGPU GAMING PC - The AMD Radeon RX 8060S offers all 40 CUs with up to 2.9 GHz graphics clock and uses the new RDNA 3.5 architecture. The powerful iGPU is positioned between an RTX 4060 and 4070 laptop GPU and therefore enables gaming in FHD at maximum details in most demanding games. The 8060S can also utilize the full 128GB pool, which is perfect for running LLMs such as Deepseek 70B Q8, which runs comfortably on this machine.
- EIGHT CHANNEL LPDDR5X - LPDDR5X is a new ground breaking memory small form factor installed on-board. With blazing speeds up to to 8000MT/s, it runs 1.5x faster than the DDR5 SODIMMs; 90% better performance over DDR5 SODIMMs in video conferencing and photo editing; 30% better performance in productivity apps; 12% better performance in digital content workloads.
- QUAD SCREEN 8K DISPLAY SUPPORT - EVO-X2 AI Mini PC support 4-screen 4K/8K output via HDMI 2.1 (8K@60Hz), DisplayPort 1.4 (4K@60Hz), and dual USB 4 40Gbps Transfer speed (supporting PD3.0/DP1.4/DATA). Ideal for gaming, video editing, and multitasking, it provides expansive and crisp multi-display support.
llama-server -hf ggml-org/gemma-3-1b-it-GGUF
Check the current llama.cpp README for up-to-date command syntax and supported backends for your operating system. A model that can load with CPU/GPU hybrid inference may still run some or all work on the CPU, which can affect speed.
Do these 3 things before closing this tab:
1Repair Windows errors before they cause bigger problems2Scan for outdated or missing drivers - takes under a minute3Clear out junk files and repair common Windows errorsHow to choose a model that fits your computer
Compare the specific model variant against your hardware and task. Parameter count is a useful clue, but it does not tell you the complete memory requirement or whether the model supports the capability you need.
- Task capability: Choose a model variant intended for your use. For coding, image input, audio, long context, or tool use, verify that the exact variant supports the capability; do not infer it from the model family name alone.
- Runtime and format: Confirm that the runtime supports the downloaded file format and the model’s architecture.
- Available memory: Account for the weights, context window, runtime overhead, and memory already used by other applications. RAM and dedicated GPU memory are not always interchangeable.
- Quantization: Lower-precision or quantized weights can reduce memory needs, with possible trade-offs in output quality. Compare the available variants rather than assuming the smallest file is best.
- Context length and speed: Longer context uses more memory. Actual speed depends on the model, runtime, settings, and your computer; there is no universal performance ranking established for these tools.
- License and terms: Read the exact model card and license, particularly before redistributing a model or using it commercially.
Example: Gemma 4 memory estimates
Google’s Gemma 4 overview, last updated July 8, 2026, gives approximate GPU/TPU memory figures for loading three variants. The estimates include the page’s stated 20% overhead for additional loading items, but exclude supporting software and context-window memory. Google notes actual figures vary with the inference tool and environment, and longer context increases memory use.
| Gemma 4 variant | BF16 | SFP8 | Q4_0 |
|---|---|---|---|
| E2B | 11.4 GB | 5.7 GB | 2.9 GB |
| E4B | 17.9 GB | 8.9 GB | 4.5 GB |
| 12B | 26.7 GB | 13.4 GB | 6.7 GB |
These are Google’s approximate loading estimates for these Gemma 4 variants, not a general memory calculator for other models. Google describes Gemma 4 as ranging from E2B and E4B variants aimed at edge devices to larger variants for consumer GPUs and workstations. Its model card lists text and image support across the family and audio support for E2B, E4B, and 12B. It lists 128K context for E2B and E4B and 256K for 12B and 31B; Google’s overview describes 256K context for the 26B A4B as well. The 26B A4B is a mixture-of-experts model with 25.2B total and 3.8B active parameters. Active parameter count does not mean only those active parameters need to be resident: Google says all 26B parameters must be loaded for fast routing and inference.
What to check if something goes wrong
- The model will not load: Check free RAM and VRAM, the selected quantization, context length, and whether other applications are using memory. Try a smaller variant or lower context.
- The model loads but is slow: Check whether the runtime is using an available accelerator. Some work may fall back to the CPU, and llama.cpp can use hybrid CPU/GPU inference.
- The download or load fails: Confirm that the model format is supported by the runtime and that the downloaded file is complete.
- The model behaves unexpectedly or its terms are unclear: Check the exact variant’s model card and license rather than relying on a library label or family name.
Does running a model locally make it private?
Local inference can keep the model workflow on your computer, but local use alone does not prove that data stays there. Review the app’s settings and network behavior, including telemetry, extensions, cloud-connected features, and any services connected to the model. Do not expose a local API or server to a public network without understanding how access is restricted.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
