October DealsAmazon USOctober deal check: compare before you payAmazon US: current deals, useful picks and tech finds.Check DealsPC HealthRecommendedCrashes, freezes, slowdowns? Check your PC nowSpot repairable issues before they interrupt work.Check PCOctober DealsAmazon USDeal season is back - check today's better picksAmazon US: current deals, useful picks and tech finds.See Picks×
Skip to content
World desk10 min

How to Run Ollama on CPU Only with Docker Compose

A practical guide to running Ollama without GPU access in Docker Compose, including the exact YAML, model commands, persistent storage, API checks, CPU overrides, and recovery steps.
Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Run Ollama without GPU access by using the official ollama/ollama image, a persistent /root/.ollama volume, and port 11434. A CPU-only Compose file contains no GPU reservations, NVIDIA runtime options, ROCm image tag, or device mappings.

Save the Compose file, start it with docker compose up -d, then launch a model with docker compose exec ollama ollama run llama3.2. The instructions below also show how to keep model files across container replacement, verify the API, restrict network exposure, and diagnose common failures.

As an Amazon Associate I earn from qualifying purchases.

What’s actually slowing this PC down?

Pick the symptom - the matching free tool is one click away.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

What you need before starting

You need Docker Engine with the Compose plugin, or a Docker Desktop installation that provides the docker compose command. The host must have enough memory, processor capacity, and disk space for the model you intend to run. Model downloads are stored locally, so the required disk space grows as you add models.

CPU compatibility

Ollama can select an appropriate CPU runner automatically. Most installations should leave that autodetection untouched. If you later force a specific runner, the CPU must support the instruction set required by that runner; the documented cpu_avx2 override, for example, requires AVX2 support and a matching runner in the image.

#1 Best Overall
Sale
TOZO A1 Wireless Earbuds Bluetooth 5.3 Light Weight in Ear Headphones | IPX5 Waterproof,2 Mics for AI Calls,Immersive Premium Sound Bass Headset with Charging Case,32 EQ via App Black
  • [Ultra-Lightweight Ear Buds Designed for Small Ears] Each earbud weighs only 3.7g and features a compact, ergonomic in-ear design made especially for small ears. Secure, low-profile, and comfortable for workouts, all-day wear or overnight sleeping.The ultra-slim profile sits flush in your ear with no hard edges poking or rubbing, so you can even sleep on your side without discomfort.
  • [Immersive Stereo Sound with TOZO OrigX Technology] TOZO OrigX tuning delivers clear vocals, balanced mids, and natural stereo sound for music, podcasts, and videos.
  • [Long Battery Life for Daily Use] Get up to 7 hours of playtime on a single charge, with up to 32 hours total using the charging case—ideal for workdays, commuting, and extended listening sessions.
  • [Bluetooth 5.3 & Stable Connection] Bluetooth 5.3 provides fast pairing, stable wireless performance, and reduced dropouts as you move around home or office.
  • [Deep Bass with Clear Vocals] High-performance drivers produce punchy bass while keeping vocals clean and detailed for everyday listening.

Why Compose is useful here

Compose keeps the image, port, volume, and restart policy in one declarative file. You can recreate or upgrade the container without manually reconstructing command-line flags, while the named volume keeps downloaded models separate from the disposable container filesystem.

The minimal CPU-only docker-compose.yml

Create a directory for the service and save this file as docker-compose.yml or compose.yml:

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
services:
  ollama:
    image: ollama/ollama:latest
    container_name: ollama
    ports:
      - '11434:11434'
    volumes:
      - ollama:/root/.ollama
    restart: unless-stopped

volumes:
  ollama:

This is intentionally small. The official image supplies the Ollama server, the port mapping makes the service available on port 11434, and the named volume maps persistent storage into Ollama’s standard data directory.

What each setting does

Setting Purpose Operational consequence
image: ollama/ollama:latest Uses the official Ollama container image. The image includes CPU runtime support. The latest tag can change when a newer image is published.
container_name: ollama Assigns a predictable container name. Commands such as docker logs ollama address this container directly.
ports Publishes host port 11434 to container port 11434. Clients on the host can use http://localhost:11434.
ollama:/root/.ollama Mounts the named volume at Ollama’s data directory. Downloaded models survive container recreation.
restart: unless-stopped Asks Docker to restart the service after routine failures and host restarts. The service comes back automatically unless you explicitly stop it.

Start the container and run a model

  1. Open a terminal in the directory containing the Compose file.
  2. Start the service in the background:
    docker compose up -d

    Compose creates the named volume if it does not already exist, downloads the image when necessary, and starts the container.

  3. Check the service state:
    docker compose ps

    The ollama service should be listed as running. If it is still starting, inspect its logs before trying a model.

  4. Launch a model from inside the container:
    docker compose exec ollama ollama run llama3.2

    The first run downloads the model into /root/.ollama. Subsequent runs use the copy in the volume. Replace llama3.2 with another model name supported by your Ollama installation.

Call the HTTP API from the host

After the server is running, query the tags endpoint:

curl http://localhost:11434/api/tags

A successful response is JSON describing models already present in the volume. An empty model list is normal before the first download; a connection error means the host cannot reach the published port and should be investigated using the checks in the troubleshooting section.

Rank #2
Sale
Smart Watch for Men Women Compatible with iPhone Samsung Android Phone 1.69 inch Full Touch Screen IP68 Waterproof Bluetooh Fitness Tracker Heart Rate/Sleep Monitor (Black)
  • 【IP68 Waterproof Multifunctinal Smart Watch】Torjalph fitness activity tracker smart watch is IP68 waterproof technology ,you can wear it when swimming,handwashing,it's rain-proof which means that you perform multiple exercise modes outside without worrying about sweat or rain corroding your fitness smart watch,and this watch equipped with a 230 mAh large capacity battery magnetic charging ,and this sports activity tracker watch can be used 5-7 days If fully charged.
  • 【Larger 1.83'' Full Screen Watch & Multi-dial Selection】1.83 inch full HD color touch screen smart watch for men / women provides excellent 240*284 resolution image quality ,This Bluetooth smart watch offers a better touch experience and remarkable immersion and ,more than 200 watch faces for you to choose from App"Da Fit ", you can change the dial style according to your mood to wear keeping the world on your wrist
  • 【Accurate Activity Tracker Smart Watch 】sports watch activity tracker uses sweat and dust resistant breathable silicone strap, our Fitness Tracker Smart Watch for Android iOS Phones supports 12 sports modes,walking,running,bicycle,swimming,badminton,basketball,football,rope skipping. This bluetooth calls smart watch for men women is lightweight and stylish, bringing more convenient and relaxing sports experience while also providing more protection.
  • 【Your 24 Hour Personal Daily Health Monitor】Fitness watch for women men full touch screen IP67 waterproof bluetooth smartwatch for Android iOS Phones with heart rate blood pressure monitor multifunction Andriod smart watch with steps counter sleep monitor sports fitness watch, monitoring blood oxygen saturation allows you to understand your physical condition in real time to track fitness progress ,to know your health status in time.
  • 【More Functions Bluetooth Call Smart Watch】This fitness activity smart watch full screen for iPhone and Android phones includes a bunch of new features including:The bluetooth smart watch can hang up the phone, receive and display WhatsAPP /Text /Facebook /Twitter /Messenger reminder.Support sedentary reminder, alarm clock, stopwatch, timer,weather information,music control, remote camera control compatible with Android 5.0 and above or iOS 9.0 and above smartphones.

Why this configuration is CPU-only

CPU mode is achieved by not granting the container any accelerator access. Docker does not expose a host GPU to a container merely because one exists; an explicit runtime, reservation, or device mapping is normally required. The Compose file above contains none of those settings.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
Acceleration type Settings normally required What the CPU-only file does
CPU Standard container start with the official image and no accelerator mapping. Uses this path by omission of GPU-specific options.
NVIDIA GPU A GPU-enabled Docker configuration, such as a --gpus=all style runtime request or an equivalent Compose reservation. Contains no NVIDIA runtime or GPU reservation.
AMD GPU The rocm image variant plus access to /dev/kfd and /dev/dri. Uses neither the ROCm tag nor those device mappings.

Do not add accelerator settings “just in case.” They change the execution path and can make a file fail on hosts that do not have the corresponding runtime or devices. A CPU-only deployment should also avoid copying GPU examples from a different operating-system guide.

Bind the API only where it is needed

The short port form in the example is convenient for a single-user setup. Docker commonly publishes that form on all host interfaces. If the service is intended only for applications on the same machine, bind the host side explicitly to loopback:

ports:
  - '127.0.0.1:11434:11434'

With this mapping, requests from the local host can use http://localhost:11434, while other machines cannot reach the published port through the host’s network addresses. If another local service already uses port 11434, choose a different host port while keeping the container side at 11434, for example 127.0.0.1:11435:11434; clients would then connect to port 11435 on the host.

Persist models with the named volume

Ollama stores downloaded models below /root/.ollama. The ollama named volume maps that directory outside the container’s writable layer. Removing and recreating the container therefore does not require every model to be downloaded again.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
Rank #3
New Bee Guitar Cable 10ft Electric Instrument Bass AMP Cord
  • [Low Noise High Fidelity] New bee guitar cord with 22AWG oxygen-free-copper center conductors provides enhanced signal transfer, the PVC insulation material and the high-density stranded copper double shield provide noise rejection for quiet operation
  • [Ideal for Choice] New bee 1/4 straight and right angle metal connectors, ensures a secure connection in any instrument, pedal, or amp jack. L shape great fro on stage guitar stands
  • [Wide Compatible] Suitable for mixer, power amplifier, microphone, effector, decoder, equalizer, electoronic organ, electric guitar, electric bass and other electronic equipments. Widely used in the installation of smart home, karaoke speaker, professional stage audio, etc
  • [Easy Disassemble] Making incidental repairs quick and easy! Very nice, flexible cable
  • [Dont Hesite to Order] 24 hours customer services and professional technology team are standing by

Operations that keep the volume

  • docker compose down removes the Compose-managed container and network but, by default, leaves the named volume.
  • docker compose up -d attaches the existing volume when the project and volume definition are unchanged.
  • docker compose up -d --force-recreate replaces the container while retaining the same volume.

Operations that delete model data

Deleting the volume, changing the volume name, or mounting a different host path means the new container no longer sees the old model files. Treat commands that include volume removal as destructive and verify that you have another copy before using them.

Verify the service and read its logs

Use these checks in order:

  1. Confirm Compose sees the service:
    docker compose ps
  2. Follow the service output while it starts or downloads a model:
    docker compose logs -f ollama
  3. Test the API endpoint from the host:
    curl http://localhost:11434/api/tags

Container output is written to standard output and standard error, so Compose logs and Docker logs show the same primary diagnostic information. Because the file assigns the name ollama, the equivalent direct command is:

docker logs ollama

Stop following a live log stream with the terminal interrupt key; this does not stop the container itself.

Troubleshoot common failures

The API is unreachable

  • Run docker compose ps. If the container is stopped or repeatedly restarting, read docker compose logs ollama before changing the file.
  • Check that the host port in the Compose file is not already occupied. A changed host port must also be used in the URL you test.
  • Confirm that the service was started from the directory containing the intended Compose file. Running Compose from another project can create or address a different container and volume.
  • If you bound the port to 127.0.0.1, test from the same host. Other machines will not be able to use that loopback binding.

Models download again after recreation

Inspect the Compose file for an unchanged ollama:/root/.ollama mount. A typo in either side of the mapping, a renamed volume key, or a different project directory can point the replacement container at a new empty volume. List Docker volumes if necessary and compare the active mount shown by docker inspect ollama with the intended path.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

The container appears to use an accelerator

Remove GPU reservations, runtime settings, device mappings, and accelerator-specific image tags. The CPU image should be the standard ollama/ollama image, not the rocm variant. Recreate the service so that old container settings are not retained:

Rank #4
BSIMB 15.6" Digital Family Calendar & Chore Chart, No Subscription Fees
  • Digital Family Calendar: BSIMB digital calendar wall touch screen puts an end to your chaotic life. Designed for busy families, the digital calendar keeps you organised and in step with your family. Assign different colours and tasks to each family member to make family scheduling easier and more intuitive.
  • Easy Setup and Auto-Sync: Plug in, connect to WLAN and auto-syn your electric calendar in minutes. Digital wall calendar compatible with Google calendar, iCloud calendar, Outlook calendar, Cozi calendar and Yahoo calendar. Add events directly on the device or through the ecalendar app. Electronic calendar stays current whether you're at work or traveling—no manual updates needed.
  • All-In-One Home Assistant: Smart calendar builds habits and encourage independence with personalised chores. Display daily meal plan to reduce the daily ‘What's for dinner?’ stress. To-do lists keep track of everything you want. Turning chores into valuable achievements with stars and rewards, making life easier and more enjoyable.
  • Manage Anytime Anywhere: BSIMB electronic calendar for family stay connected with the powerful ecalendar app. Access and manage your digital wall calendar anytime, anywhere with free features like stars & rewards, photo screensaver, chore charts, meals plans, to-do lists etc. Electronic planner paid features like Magic Import tailor advanced tools for users with special needs.
  • Stay in Touch with Loved Ones: Share photos or videos from your phone to electronic wall calendar with app, instantly update your life moment, and keep in touch with your family, especially those who are far away; you can also surprise your smart calendar digital wall by sending him a wish card; it's also great for keeping tech-illiterate elders in touch with the youngsters.
docker compose down
docker compose up -d --force-recreate

Then inspect the startup logs and run a model again. A fresh container is important because editing the YAML file does not modify an already-created container until Compose recreates it.

The service restarts continuously

Read the earliest log messages, not only the latest restart line. Common causes include an invalid image pull, an unsupported forced CPU runner, a port collision, or insufficient host resources. Temporarily remove nonessential environment overrides, confirm the standard image name, and start the service in the foreground with docker compose up so the first error remains visible.

A model command fails while the API is healthy

Separate server reachability from model availability. First query /api/tags to see what is installed. Then run the model command inside the same container with docker compose exec ollama ollama run model-name, replacing model-name with the exact model identifier. A model that is not present may need to be downloaded, and a model that exceeds available CPU memory may fail even though the server itself responds.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Support on Ko-Fi

When to force the CPU runner

Autodetection is the preferred default. If troubleshooting shows that autodetection selected an unsuitable library, the documented override can be added to the service:

services:
  ollama:
    image: ollama/ollama:latest
    container_name: ollama
    ports:
      - '127.0.0.1:11434:11434'
    volumes:
      - ollama:/root/.ollama
    environment:
      - OLLAMA_LLM_LIBRARY=cpu_avx2
    restart: unless-stopped

volumes:
  ollama:

OLLAMA_LLM_LIBRARY=cpu_avx2 is an override for autodetection, not a universal compatibility switch. Use it only when the image contains that runner and the host CPU supports the required AVX2 instruction set. If the container fails after adding it, remove the variable, recreate the service, and return to autodetection.

Upgrade the image without losing models

The named volume and the container lifecycle are independent. To fetch a newer image and recreate the service:

  1. Pull the image declared by the file:
    docker compose pull ollama
  2. Recreate the service in the background:
    docker compose up -d
  3. Check the logs and API, then run a previously downloaded model to confirm that the existing volume is attached.

Using latest follows new image releases automatically when you pull. If predictable upgrades and rollbacks matter, replace it with a version or digest that you have chosen and keep that value in source control. Whatever tag you use, do not change the volume mount while upgrading.

Free tools Windows power users keep installed

One-click scans. No signup required.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

CPU performance and resource planning

There is no single CPU performance figure that applies to every Ollama deployment. Response speed depends on the selected model, model quantization, available system memory, CPU architecture and instruction support, concurrent requests, and storage speed during loading. Official documentation does not publish a universal CPU benchmark suitable for predicting tokens per second on an arbitrary host.

Practical planning points

  • Start with one model and observe memory use and response latency before adding concurrent workloads.
  • Keep model storage on a reliable local disk; repeated downloads caused by a missing volume waste both time and bandwidth.
  • Expect the first invocation of a model to take longer because the files must be downloaded and loaded.
  • Choose a model that fits the host’s available memory. A healthy HTTP endpoint does not guarantee that every model will load successfully.
  • For sustained workloads, monitor the host rather than assuming that a GPU-oriented example or benchmark predicts CPU behavior.

Compose versus a native Ollama installation

Both approaches run the same Ollama service, but they put operational responsibility in different places.

Decision axis Docker Compose Native installation
Installation isolation Ollama and its runtime are packaged in a container, separate from most host libraries. Ollama integrates directly with the host operating system and its service manager.
Model-volume management A named Docker volume explicitly owns /root/.ollama and follows the Compose lifecycle. Models live in the native installation’s configured data location.
Upgrade and rollback control Image tags or digests can be changed independently of model data; recreation is declarative. The host package or installer controls the update path.
Networking Port publishing determines how the container API reaches the host and other clients. The native service listens directly according to its host configuration.
Startup behavior restart: unless-stopped lets Docker restart the container after failures and reboots. Startup and restart depend on the operating system’s service configuration.
CPU instruction compatibility The container image supplies the runtime, but the host CPU still must support the selected runner. The native runtime also depends on host CPU support and its installed libraries.
Operational complexity Requires Docker, Compose, image management, and volume awareness. Avoids container administration but places more responsibility on host packages and services.

Choose Compose when repeatable container configuration, isolated dependencies, or easy service recreation matter most. A native installation can be simpler when Docker is not already part of your operating environment.

Platform and scope notes

The CPU-only file is deliberately platform-neutral at the Compose level: it does not request Linux GPU devices or an accelerator runtime. Docker Desktop behavior differs by host operating system, so do not paste Linux GPU mappings into a CPU deployment merely because the host documentation shows them.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Official guidance distinguishes CPU-only Docker from NVIDIA and AMD configurations. It also notes that GPU acceleration is unavailable for Docker Desktop on macOS. A CPU-only container avoids that GPU limitation because it does not depend on GPU passthrough.

Operational checklist

  • Use the official ollama/ollama image.
  • Keep the ollama:/root/.ollama volume mount unchanged.
  • Publish container port 11434, preferably as 127.0.0.1:11434:11434 for a local-only service.
  • Do not add --gpus-style settings, GPU reservations, /dev/kfd, /dev/dri, or the rocm tag.
  • Start with docker compose up -d and verify with docker compose ps.
  • Use docker compose logs ollama as the first diagnostic step.
  • Run models with docker compose exec ollama ollama run model-name.
  • Use OLLAMA_LLM_LIBRARY=cpu_avx2 only when autodetection is unsuitable and the CPU and image support that runner.
  • Recreate the container after configuration changes so old runtime settings are removed.

Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.

Leave a Reply

Your email address will not be published. Required fields are marked *

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

More from the Wire

  1. World desk4 min
    How to Spot an AI Voice Scam Before Sending MoneyDon’t rely on how a caller sounds. Pause, call back through a known number, and verify the emergency with another trusted person before sending money.
  2. Mountain View desk4 min
    Google’s SynthID Detector: How to Check AI-Generated Images, Video and AudioGoogle’s SynthID Detector looks for an embedded watermark in supported images, video and audio. Here is what its results do—and do not—show.
  3. Redmond desk20 min
    How to create a link to File or Folder in Windows 11Windows 11 gives you several ways to point to a file or folder without moving or duplicating it. You can create a desktop shortcut,…
Recommended PC Tool
Recommended PC Tool
PC Slower Than It Used to Be?Free scan - under a minute
Outdated Drivers Are Slowing You DownFree scan - exact matches

Two free Windows tools

One Free Minute Could Fix That PC

Before you go - each of these free tools takes about a minute and tackles what quietly slows a Windows PC down.

Special offer. View Outbyte info, uninstall instructions, EULA, and Privacy Policy.