Free tools Windows power users keep installed
One-click scans. No signup required.
The setup that holds up over time is simple: pick the Gemma 4 size your memory can load, install a maintained runtime, confirm a short text prompt works, and only then add a chat window, a local API, image input, or an app. Ollama and LM Studio are the two chat-oriented routes Google documents, and the model you choose matters more than the app.
Step 1: Choose a model your memory can load
Gemma 4 comes in five sizes: E2B, E4B, 12B, 26B A4B, and 31B. Larger models and higher-precision files need more memory and more compute. Google’s Gemma 4 model overview (Google AI for Developers, accessed 2026) gives approximate model-loading memory for each size at three precisions:
As an Amazon Associate I earn from qualifying purchases.
| Model | BF16 | SFP8 | Q4_0 |
|---|---|---|---|
| E2B | 11.4 GB | 5.7 GB | 2.9 GB |
| E4B | 17.9 GB | 8.9 GB | 4.5 GB |
| 12B | 26.7 GB | 13.4 GB | 6.7 GB |
| 26B A4B | 57.7 GB | 28.8 GB | 14.4 GB |
| 31B | 69.9 GB | 34.9 GB | 17.5 GB |
These are estimates of what it takes to load the model. They include a 20% loading overhead, and the real figure changes with the inference tool and environment. They do not guarantee comfortable use at long context lengths or with several requests running at once. Leave headroom for your operating system and the context you plan to use.
Recommended Free Tools
For a rough rule: if the Q4_0 figure for a size is well under your available memory, that size is the realistic starting point. If it is close to or above your total memory, step down.
#1 Best Overall
- 【Your private database】: NAS N5 MAX, equipped with AMD Ryzen AI Max+395 processor, adopts 16x Zen 5 architecture and 16-core 32-thread design, single frequency up to 5.1GHz, supports multi-user access, simultaneous retrieval of multiple files, and ultra-high-speed decoding of audio and video playback. Say goodbye to the cumbersome operation of traditional hard drives and build your data management center, providing centralized storage, automatic backup, remote access and rich RAID options.
- 【200TB Enormous Storage Capacity】: The N5 MAX NAS comes pre-installed with 64 GB of LPDDR5x RAM (non-expandable) and features five 3.5-inch SATA drive bays, each supporting up to 32 TB, for a total capacity of 160 TB. Additionally, five M.2 NVMe slots support SSDs with up to 40 TB of capacity. This ensures rapid data access and enhances the performance of system applications, models, and caches, enabling the system to keep pace with steadily increasing data demands
- 【Versatile Connectivity Options】: The NAS is equipped with a variety of high-speed connectivity ports, including USB4 (80Gbps), HDMI 2.1 for up to 8K resolutions, and multiple USB connections. This wide array of interface options guarantees compatibility with a multitude of devices, facilitating ease of integration into existing systems and ensuring a smooth user experience through flexible connectivity solutions
- 【Dual 10GbE Networking】: The NAS includes dual 10GbE network ports, delivering exceptional data transfer speeds and the ability to handle simultaneous access from multiple devices without lag or disruption. This feature ensures that large files can be transmitted in seconds, providing a responsive and efficient multi-user environment for businesses that require high-performance networking for collaboration and data sharing
- 【Efficient Cooling System】: Featuring a comprehensive three-zone cooling architecture with advanced CPU heat pipes, independent HDD ventilation, and SSD/power fans to ensure optimal temperature management during extended operations. This thoughtful design minimizes noise levels while maximizing efficiency, allowing for quiet operation even in shared workspaces, enhancing user comfort
On the 12B model specifically, Google’s developer guide by André Susano Pinto, Research Engineer (published June 3, 2026), states: “Gemma 4 12B is small enough to run locally on dedicated GPU laptops with 16GB VRAM or unified memory.” That sentence describes the 12B model only. It is not a minimum for the other sizes or for heavier workloads.
What quantization changes
Quantized GGUF files store model values at lower precision. That reduces memory and compute cost, which is why Q4_0 figures are much smaller than BF16 figures in the table above. The trade-off is output quality. Google’s Ollama integration guide says that using less precise data in quantized models generally lowers output quality while also lowering compute costs.
How much quality drops depends on the quantization method and the task. A summarizing job may look fine while code generation or multi-step reasoning degrades. Before you settle on a file, run a handful of prompts that resemble your real work, and compare against a higher-precision build if your hardware allows it.
Quick wins for a faster PC:
Clear out junk files and repair common Windows errorsFree Scan →Scan for outdated or missing drivers - takes under a minuteDriver Scan →Repair Windows errors before they cause bigger problemsFix Now →Rank #2
- [Powerful Performance] Zen 5 Gen Ryzen AI Max+ 395 3.00GHz Processor (upto 5.1 GHz, 64MB Cache, 16-Cores, 32-Threads, ); AMD Radeon 8060S Integrated Graphics
- [High Speed and Multitasking] 128GB OnBoard RAM; Bluetooth 5.4, RJ-45, No
- [Superior Machine] 240W PSU; Black Color
- [Enormous Storage] 1TB PCIe NVMe SSD; 2 USB 2.0, 1 x HDMI 2.1, 1 Display Port, SD Reader, Headphone/Microphone Combo Jack
- Windows 11 Pro-64,
Step 2: Choose a runtime
Google’s run guide groups local tools by purpose. The table below summarizes that grouping; it does not rank these tools on speed, and Google’s sources do not establish a current cross-runtime speed comparison.
| Runtime | Best fit in Google’s guidance | Interface | Notes |
|---|---|---|---|
| LM Studio | Local chat UI | Desktop app | GGUF QAT checkpoints are mapped to LM Studio for local CPU, Apple Silicon, or consumer-GPU use. |
| Ollama | Local chat UI and local API | Command line, local HTTP API | Official integration guide documents pull, run, and a local endpoint. |
| llama.cpp | Efficient edge use | Command line | Uses GGUF QAT checkpoints; more direct configuration. |
| LiteRT-LM | Efficient edge use, desktop and on-device | Command line and local API | Mobile-optimized LiteRT formats are listed for E2B and E4B. |
| MLX | Efficient edge use on Apple Silicon | Framework | Apple-focused framework. |
| Transformers, Keras, Tunix, Unsloth | Development and fine-tuning | Code and Python | For custom applications and training, not a chat setup. |
For a first run, choose LM Studio if you want a window and no terminal work, or Ollama if you are comfortable in a terminal and plan to call the model from code. Choose llama.cpp or MLX when you need closer control over configuration or are on Apple Silicon and want an Apple-focused stack.
Setup with Ollama
This is the route Google’s Ollama integration guide documents. Run each step in order and confirm the result before moving on.
- Install Ollama for your operating system from Ollama’s official download page and its installer instructions.
- Open a terminal and run
ollama --version. If the command is not found, the Ollama executable is not on your system path. Reinstall, or add its install folder to the path, then open a new terminal. - Download the default model with
ollama pull gemma4. - Run
ollama list. The model should appear in the output. Check the size and tag it shows. - Test with a short prompt:
ollama run gemma4 "roses are red", or runollama run gemma4for an interactive session. A small text reply confirms the model loads and generates.
The Ollama guide lists the tags gemma4:e2b, gemma4:e4b, gemma4:26b, and gemma4:31b. Google’s overview also lists a 12B model, but the Ollama guide does not name a 12B tag. If you need a specific size, check the current Ollama model library for the exact tag before you pull it, since tag names can change.
Using the local API
Ollama serves a local HTTP API. The generate endpoint documented in Google’s guide is http://localhost:11434/api/generate. A minimal request looks like this:
curl http://localhost:11434/api/generate -d '{"model": "gemma4", "prompt": "Summarize quantization in one sentence.", "stream": false}'Treat this endpoint as local-machine access. Do not expose port 11434 to a wider network unless you add deliberate access controls such as a firewall rule or an authenticated proxy.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Setup with LM Studio
LM Studio is the desktop option Google lists for a local chat interface. Install it, search its model browser for Gemma 4, and pick a GGUF build at the size your memory supports. Menu labels change between app versions, so follow what the current interface shows rather than a fixed path. Start with a short chat prompt, then adjust context length only after the basic reply works.
Developer path: LiteRT-LM and local API servers
Google’s developer guide demonstrates importing a Gemma 4 12B LiteRT-LM checkpoint and launching litert-lm serve, which starts an OpenAI-compatible local API server. Use this route when an existing app expects an OpenAI-style endpoint. Confirm the exact import and launch steps in the current LiteRT-LM documentation, because command options are tied to the version you install.
Troubleshooting
- The command
ollamais not found. The executable is not on your path. Reinstall, or add the install folder to the path, and open a new terminal. - No model appears in
ollama list. Runollama pull gemma4, then runollama listagain. - The model fails to load or the system slows heavily. The file is too large for your memory. Move down one size, or choose a lower-precision build, and check the table above.
- Replies are weak or inconsistent. Quantization may be the cause. Try a higher-precision build of the same size if memory allows, and test with prompts from your real work.
- Speed is disappointing. Speed depends on hardware and runtime, and no fixed tokens-per-second figure applies across setups. Compare one configuration against another on the same machine.
- The API is unreachable from another device. That is by design for the default local endpoint. Keep it local, or put it behind access controls you manage.
Once a short prompt works in your chosen runtime, add a chat window, an API client, or image input one piece at a time. A change that breaks the setup is then easy to trace.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
