Recommended Free Tools
Yes, a small language model can run through Ollama on the original Jetson Nano, but the measured result is modest and the setup matters. In K. Kreier’s 2025 test of a 2019 Nano, Ollama 0.6.4 ran in CPU mode and generated TinyLlama 1.1B tokens at 5.37 tokens per second. That is a configuration-specific result—not evidence that Ollama currently uses the GPU on every Nano installation.
What this benchmark tested
K. Kreier tested the same original Jetson Nano machine from 2019, without overclocking, using the prompt “Explain quantum entanglement.” The comparison included Ollama 0.6.4, released in April 2025, running in CPU mode, as well as CPU and CUDA-enabled builds of llama.cpp. The test is a report from that benchmark author, not a new test of the hardware for this article. Kreier’s benchmark details.
| # | Preview | Product | Price | |
|---|---|---|---|---|
| 1 |
|
NVIDIA Jetson AGX Orin 64GB Developer Kit with Ethernet, USB, Display Port | $3,389.00 | Buy on Amazon |
As an Amazon Associate I earn from qualifying purchases.
The distinction between the runtimes and execution modes is central: the Ollama figure below is a CPU-mode result. The benchmark’s GPU figures belong to CUDA-enabled llama.cpp runs, not Ollama.
The Tool Desk
Outbyte Driver Updater FREEScan for outdated or missing drivers - takes under a minuteDriver Scan →Outbyte PC Repair FREEClear out junk files and repair common Windows errorsFree Scan →How fast was Ollama?
For TinyLlama-1.1B-Chat Q4_K_M (listed model size: 669 MB), Kreier reports 5.37 generated tokens per second for Ollama on the Jetson Nano in CPU mode. The CUDA-enabled llama.cpp comparison is listed at 6.28 tokens per second for the same model, which the author describes as around 20% faster.
#1 Best Overall
- The NVIDIA Jetson AGX Orin 64GB Developer Kit makes it easy to get started with Jetson Orin. Compact size, lots of connectors, and up to 275 TOPS of AI performance make this developer kit perfect for prototyping advanced AI-powered robots and other autonomous machines.
- The developer kit includes a Jetson AGX Orin 64GB module, and can emulate all the Jetson Orin modules. It supports multiple concurrent AI application pipelines with the NVIDIA Ampere GPU architecture, next-generation deep learning and vision accelerators, high-speed IO and fast memory bandwidth. Now you can develop solutions using your largest and most complex AI models to solve problems such as natural language understanding, 3D perception, and multi-sensor fusion.
- Jetson runs the NVIDIA AI software stack, and use-case specific application frameworks are available, including Isaac for robotics, DeepStream for vision AI, and Riva for conversational AI. You can save significant time with NVIDIA Omniverse Replicator for synthetic data generation (SDG), and by using NVIDIA TAO toolkit to fine-tune pretrained AI models from the NGC catalog.
- Jetson ecosystem partners offer additional AI and system software, developer tools, and custom software development. They can also help with cameras and other sensors, as well as carrier boards and design services for your product.
- With the computing capability of more than 8 Jetson AGX Xavier systems in a developer kit that integrates the latest NVIDIA GPU technology with the world’s most advanced deep learning software stack, you’ll have the flexibility to create tomorrow’s AI solution as well as today’s.
| Runtime and mode | Model | Reported generation speed |
|---|---|---|
| Ollama 0.6.4, CPU mode | TinyLlama-1.1B-Chat Q4_K_M (669 MB) | 5.37 tokens/s |
| llama.cpp, CUDA-enabled | TinyLlama-1.1B-Chat Q4_K_M (669 MB) | 6.28 tokens/s |
These are the benchmark author’s measurements under the stated test setup, not guaranteed performance for other Nano units, software versions, prompts, or power conditions. As Kreier puts it, “The main metric to compare here is the token generation.” The table’s comparison suggests that the tested CUDA-enabled llama.cpp configuration was quicker for this small model; it does not show that Ollama achieved that GPU acceleration.
What happened with larger models?
Model size and quantization affected whether a model could run and how much memory the test used. Kreier reports 1.9 GB of Jetson memory use for Gemma 3 1B and 2.8 GB for the tested Gemma 3 4B Q4 variant. Those figures describe the author’s setup and tested variants, not general memory requirements for every installation.
The benchmark reports practical limits with larger Gemma 3 variants. In its llama.cpp 4B tests, full GPU offload succeeded only for a Q2 quantization attempt. In the described 4B case, Ollama 0.6.4 ran in CPU mode through some tested variants and crashed with Google’s version. The result is a warning that “4B” alone does not determine compatibility: the quantization, runtime, available memory, and degree of GPU offload all matter.
Quick wins for a faster PC:
Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Repair Windows errors before they cause bigger problemsFix Now →Does Ollama use the Nano’s GPU?
Not in the reported Ollama measurements: Kreier explicitly identifies Ollama 0.6.4 as running in CPU mode. The benchmark also discusses GPU-enabled llama.cpp behavior separately. Its general summary says that the Jetson Nano used the GPU to generate tokens at 100% GPU load, with 1.5 GB GPU memory and 4 watts of power, while the CPU was around 14% load and 0.6 watts. Those utilization and power figures are for the GPU-enabled llama.cpp behavior described there, not the Ollama CPU-mode run.
So the benchmark supports a narrow conclusion: Ollama could generate TinyLlama output on the original Nano in CPU mode, while a separate CUDA-enabled llama.cpp setup delivered the faster reported result. It does not establish current, universal Ollama GPU acceleration on the original Nano.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Why Orin Nano setup instructions do not settle compatibility
The original Jetson Nano is a 2019 board; the Jetson Orin Nano is a separate, newer platform. NVIDIA’s Ollama on Jetson tutorial describes native installation and a Docker option for the supported devices it lists, which include Orin-family hardware rather than the original Nano. It therefore should not be read as confirmation of current support for the older Nano. The tutorial’s broad performance characterization is not a benchmark of Ollama on the original Nano.
NVIDIA’s JetPack setup guidance for the Jetson Orin Nano Developer Kit likewise concerns Orin Nano. JetPack’s libraries and tools for that device do not, by themselves, demonstrate software compatibility with the original Nano.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
A historical Ollama issue about JetPack 4 support for the Jetson Nano records reports of CPU-only behavior and describes CUDA 10 and compiler-toolchain compatibility complications. It is useful context for why older Nano setups can be difficult, but issue discussion and user reports are not a definitive current Ollama compatibility policy.
Quick Recap
Who should consider running it?
- For an experiment with a small model: the benchmark provides evidence that TinyLlama 1.1B can generate text through CPU-mode Ollama on a 2019 Nano, at the reported speed under that setup.
- For GPU acceleration: do not assume the Ollama result benefits from CUDA. The cited faster result came from a CUDA-enabled llama.cpp build.
- For larger models: expect model variant, quantization, memory use, and runtime behavior to constrain what loads and runs reliably.
- For a new installation: check guidance for the exact board generation and software stack. Orin Nano instructions are not evidence that the original Nano is supported in the same way.
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
