The Tool Desk
Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →You can self-host Ollama on a VPS by installing Coolify on a fresh Linux server, adding that server to Coolify over SSH, and deploying Ollama from Coolify’s service catalog or as a Docker Compose resource. The critical choices are machine sizing, persistent storage for /root/.ollama, and whether the host will run inference on its CPU or an attached GPU.
Antihost offers Linux VPS plans with full root access, unlimited traffic on a 1 Gbps port and a 14-day free trial.
Coolify’s own control plane needs 2 CPU cores, 2 GB of RAM, and 10 GB of free disk. That is only the management minimum: Ollama’s current quickstart recommends at least 8 GB RAM for 7B models, 16 GB for 13B models, and 32 GB for 33B models. Plan additional memory and disk for the operating system, Docker, Coolify, model files, and concurrent requests.
As an Amazon Associate I earn from qualifying purchases.
Free tools Windows power users keep installed
One-click scans. No signup required.
What the deployment looks like
Coolify is the management layer; Ollama remains the model server. Coolify connects to your selected Linux machine over SSH and performs Docker operations there. Ollama runs in a container, listens on its API port, and stores downloaded models beneath /root/.ollama inside the container. A named Docker volume or a host-backed persistent volume maps that directory to storage that survives container recreation.
A typical path is:
- A fresh Linux VPS or dedicated server with root or sudo access runs Docker and Coolify.
- Coolify’s dashboard is used to select that server and create an Ollama service.
- The service uses the official
ollama/ollamaimage for CPU inference, or a GPU-specific image and device configuration. - You pull a small model first, verify a response, then move to larger models after checking memory, disk, and latency.
The dashboard is normally reached on port 8000 during initial setup. Do not leave the dashboard or an Ollama API broadly reachable just because the installation works; restrict access with your firewall, private networking, and the reverse-proxy and TLS controls you intend to use.
#1 Best Overall
- Real Boost Clock- 1882 Mhz Memory Detail- 12288 Mb Gddr6.
- Real-Time Ray Tracing In Games For Cutting-Edge, Hyper-Realistic Graphics.
- Dual Fans Cooling Offer Higher Performance Cooling And Much Quieter Acoustic Noise.
- Package weight - 2.1 Lbs
Choose the VPS before installing anything
Minimum resources and practical sizing
Coolify documents a baseline of 2 CPU cores, 2 GB RAM, and 10 GB free disk for the control plane. Ollama’s current quickstart gives separate memory guidance by model size. These figures are not a complete server budget: Coolify, the operating system, Docker, the model process, and request buffers all share the machine.
| Workload or component | Published requirement or size | How to interpret it |
|---|---|---|
| Coolify control plane | 2 CPU cores, 2 GB RAM, 10 GB free disk | Coolify’s stated self-hosting baseline; it is not enough RAM for every Ollama model. |
| Ollama 7B model | At least 8 GB RAM | Use as a lower bound for a single 7B-class workload, with headroom for the host. |
| Ollama 13B model | At least 16 GB RAM | Allocate more if other services or multiple requests share the VPS. |
| Ollama 33B model | At least 32 GB RAM | Keep additional capacity for Coolify, Docker, and the operating system. |
| Model files | Varies by model and tag | Disk capacity must cover every downloaded model plus temporary and container data. |
CPU, disk, network, and access checks
Before creating the server, verify the CPU architecture, available RAM, SSD capacity and I/O, region, bandwidth limits, SSH access, and whether the provider supports GPU passthrough. A fresh Linux installation reduces conflicts with existing Docker versions, firewall rules, and services. Keep enough disk for the model files themselves; deleting and re-pulling models is slow and does not solve a full filesystem.
Crashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minuteWindows Errors? Fix Them Before They Spread
Repair common Windows errors and clear accumulated junk for a smoother, more stable PC - no reinstall needed.Free scan · no reinstallFor a GPU deployment, confirm that the provider exposes the required device to the operating system rather than merely advertising a GPU option. The documented Coolify example was tested with an NVIDIA RTX 3060 containing 12 GB of VRAM and recommends at least 32 GB of system RAM for that setup. That is an example configuration, not a performance guarantee for every model or request pattern.
Prepare SSH and firewall access
- Log in to the new Linux server as root or as a sudo-capable user.
- Apply the distribution’s security updates and confirm that you can open a second SSH session before changing access rules.
- Install or confirm your SSH key, then disable password login only after key-based login has been tested.
- Allow SSH from your administration network and allow port 8000 only for the initial Coolify dashboard access if you need it remotely.
- Open only the ports required by the applications you will publish. Keep the Ollama API on a private network unless a controlled, authenticated path is required.
Coolify can place a reverse proxy in front of published services. Decide whether the Ollama endpoint should be private, limited to an internal application network, or exposed through a domain with TLS and access controls. The correct choice depends on who will call the API; an internet-reachable inference endpoint can consume your CPU, RAM, GPU time, and bandwidth.
Install Coolify on the VPS
Run the documented installer
From an SSH session on the prepared Linux server, run Coolify’s automated installer:
curl -fsSL https://cdn.coollabs.io/coolify/install.sh | bashFollow the terminal output until the installer reports completion. The process installs and configures Coolify’s Docker-based control plane. Record any generated credentials or paths shown by the installer.
Create the first administrator and protect the secrets
Open the dashboard at the address and port reported by the installer, initially port 8000 unless you changed the network path. Create the first administrator account immediately. Back up /data/coolify/source/.env; Coolify stores installation secrets there, and losing that file can complicate recovery. Store the backup in a protected location rather than in a public repository or an unencrypted shared folder.
Add the deployment server in Coolify
Coolify may run on the same VPS as Ollama or manage another server. In the dashboard, add the target server, provide its SSH connection details, and let Coolify validate the connection. If validation fails, check the server address, SSH port, key permissions, sudo privileges, and firewall rules before attempting a deployment. A successful connection means Coolify can issue Docker commands on that host; it does not prove that a GPU or a model will fit in memory.
Rank #2
- Chipset: NVIDIA GeForce RTX 3060
- Video Memory: 12GB GDDR6
- Memory Interface: 192-bit
- Output: DisplayPort x 3 (v1.4a) / HDMI 2.1 x 1.Avoid using unofficial software
- Digital maximum resolution: 7680 x 4320
Deploy Ollama from Coolify
Use the one-click service
In the Coolify dashboard, create a new service on the selected server and choose Ollama from the service catalog. Review the generated configuration before deploying. Confirm which port is published and where the model directory is mounted. Replace any ephemeral container path with a named volume or host-backed persistent volume mounted at /root/.ollama.
Use a Docker Compose resource when you need explicit settings
If you need to control volumes, ports, or GPU reservations directly, create a Docker Compose resource in Coolify. A minimal CPU-oriented definition is:
services:
ollama:
image: ollama/ollama
ports:
- "11434:11434"
volumes:
- ollama_data:/root/.ollama
volumes:
ollama_data:The official image is ollama/ollama. The named volume keeps downloaded models when Coolify recreates or updates the container. If you publish port 11434, restrict it at the firewall or place it behind the network boundary you selected; publishing a port is not the same as securing an API.
Verify persistence before downloading large models
- Deploy the service and confirm the container is running in Coolify.
- Open a terminal on the server and check that the volume exists and has free space.
- Pull a small model and run one request.
- Redeploy or restart the service from Coolify.
- Run the same model again. It should be available without downloading it a second time.
If a restart causes the model to disappear, inspect the volume mapping first. The mount target must be exactly /root/.ollama, and the volume must be attached to the Ollama service rather than to a temporary build container.
Pull a model and perform a first test
Start with a model whose memory and disk requirements fit comfortably within the VPS. The current Ollama quickstart lists these model download sizes:
| Model example | Quickstart download size | Use during validation |
|---|---|---|
| Llama 3.2 3B | 2.0 GB | Small first pull for checking connectivity, storage, and response handling. |
| Llama 3.1 8B | 4.7 GB | A larger test after the container and persistent volume are confirmed. |
| Llama 3.1 70B | 40 GB | Requires substantially more disk and memory planning than the small examples. |
| Llama 3.1 405B | 231 GB | Needs very large storage and is unsuitable for a minimally sized VPS. |
Exact downloads depend on the model tag you select. Treat the published sizes as planning figures for the current quickstart, not as a promise that every variant has the same footprint.
Do these 3 things before closing this tab:
1Fix the driver behind crashes, sound loss and screen glitches2Repair Windows errors before they cause bigger problems3Scan for outdated or missing drivers - takes under a minuteFrom a shell in the running container, the documented Docker usage is:
docker exec -it ollama ollama run llama3.2In Coolify, the container name may differ from ollama. Use the service’s terminal or the actual container name shown by Docker, then run the equivalent command. A successful response confirms that the service can start a model; it does not by itself prove that requests are using a GPU.
Rank #3
- NVIDIA Ampere Streaming Multiprocessors
- 2nd Generation RT Cores
- 3rd Generation Tensor Cores
- Powered by GeForce RTX 3060
- Integrated with 12GB GDDR6 192-bit memory interface
Run CPU-only inference
CPU-only mode is the simplest deployment. Keep the official ollama/ollama image, provide sufficient RAM, and watch CPU utilization and response latency under your expected request load. CPU inference avoids driver and device-mapping work, but larger models can require considerably more memory and patience than the small validation model.
Do not size solely from the model file’s disk size. The 4.7 GB download listed for an 8B example is not a statement that a 4.7 GB RAM VPS can run it. Use Ollama’s memory guidance—8 GB for 7B, 16 GB for 13B, and 32 GB for 33B—as the starting point, then add headroom for Coolify and the host.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Add an NVIDIA GPU
Install the host-side components
An NVIDIA deployment requires more than selecting a GPU in a VPS control panel:
- An NVIDIA driver installed on the Linux host.
- The NVIDIA Container Toolkit installed on the host.
- Docker configured to use the NVIDIA runtime.
- A Compose device reservation that requests the GPU.
Complete these steps on the host that actually runs the Ollama container. Restart Docker if the toolkit’s installation instructions require it, then verify the runtime before deploying Ollama.
Test GPU visibility independently
Run this Docker test on the host:
docker run --gpus all ubuntu nvidia-smiThe command should display the GPU and driver information. If it cannot see the GPU, Ollama will not be able to use it; fix the driver, toolkit, Docker runtime, or provider passthrough first.
Quick wins for a faster PC:
Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Clear out junk files and repair common Windows errorsFree Scan →Add a Compose GPU reservation
In the Coolify Compose resource, add a reservation similar to this and keep the persistent model volume:
services:
ollama:
image: ollama/ollama
ports:
- "11434:11434"
volumes:
- ollama_data:/root/.ollama
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: all
capabilities: [gpu]
volumes:
ollama_data:Coolify’s published example used an RTX 3060 12 GB and recommends at least 32 GB of system RAM for that GPU setup. VRAM capacity limits which models can be fully accelerated, while system RAM and the model’s own requirements still matter. Do not infer tokens-per-second from the GPU model alone; speed depends on the exact model, quantization, hardware, and concurrency.
Add an AMD GPU
Ollama’s Docker instructions use the ROCm image for AMD acceleration:
Rank #4
- Strong performance thanks to NVIDIA Ampere with NVIDIA GeForce RTX 3060, 12GB GDDR6, ray tracing and DLSS support
- Display outputs: DisplayPort v1.4a x 3 / HDMI 2.1 x 1
- DUAL FAN COOLING Two fans, combined with a huge heatsink, make the VENTUS series quiet and powerful
- Award winning MSI TORX Fan 3.0 design for high static pressure and effective cooling
- MSI Dragon Center: With just a few clicks, performance can be monitored and optimized in real time
services:
ollama:
image: ollama/ollama:rocm
ports:
- "11434:11434"
volumes:
- ollama_data:/root/.ollama
devices:
- /dev/kfd
- /dev/dri
volumes:
ollama_data:The /dev/kfd and /dev/dri mappings expose the host’s relevant devices to the container. Confirm that the VPS provider exposes those devices and that the host’s ROCm-compatible driver stack is installed before deploying this variant. If the devices are absent, the container cannot use the GPU regardless of its image tag.
Recommended Free Tools
Expose the API safely
Keep internal traffic private where possible
If another application on the same Coolify server is the only client, attach both services to an internal Docker network and avoid publishing Ollama’s port to the public internet. If a remote client must connect, publish through the reverse proxy with a domain, TLS, and an access-control design appropriate to that client.
Limit dashboard and API reachability
Allow port 8000 only during initial administration or from a trusted management network. Apply firewall rules at both the VPS provider and the operating system when both layers are available. Keep SSH restricted to known source addresses where practical, and review which ports Coolify opens when you publish a service.
Assume model endpoints consume resources
Every request can consume CPU, RAM, GPU time, and bandwidth. Monitor request volume and set network controls before sharing an endpoint with untrusted users. Avoid placing secrets in prompts or logs, and decide where application and proxy logs are retained before moving beyond a private test.
Operate the deployment after it works
Monitor the four failure signals
- Disk: model pulls and container logs can fill the filesystem even when RAM is healthy.
- Memory: swapping or out-of-memory kills indicate that the model or concurrent workload exceeds the VPS budget.
- GPU utilization: on accelerated hosts, verify that the GPU is being used rather than assuming the reservation worked.
- Container state: repeated restarts usually point to invalid configuration, missing devices, permissions, or insufficient resources.
Back up what is actually needed
Back up the Ollama volume if you need to preserve downloaded models, and back up Coolify’s /data/coolify/source/.env because it contains installation secrets. Also retain your Compose definition, domain configuration, firewall rules, and a record of the model tags you deployed. Test restoration on a spare server instead of assuming that a backup file is usable.
Upgrade deliberately
Pin an Ollama image tag when reproducibility matters; using a moving tag can change the server between deployments. Before upgrading Ollama, Coolify, Docker, or a GPU toolkit, confirm compatibility in their current documentation, take backups, and record the running image and configuration. Roll back to the previous image and Compose definition if a new deployment fails.
Troubleshooting checklist
The Coolify installer or dashboard is unreachable
- Confirm the VPS is running and that SSH still works.
- Check that port 8000 is allowed from your current IP and is not blocked by a provider firewall.
- Read the installer’s terminal output for the actual dashboard address and any failed Docker service.
- Use a trusted network or SSH tunnel for initial administration rather than opening the dashboard globally.
Coolify cannot connect to the deployment server
- Verify the hostname or IP address, SSH port, username, and key.
- Check key file permissions and that the account has the sudo privileges Coolify expects.
- Confirm the host firewall allows SSH from the Coolify machine.
- Test an interactive SSH connection with the same account before retrying the dashboard validation.
The model vanishes after a redeploy
- Inspect the service definition and confirm a named or host-backed volume is mounted at
/root/.ollama. - Make sure the volume is attached to the long-running Ollama service, not only to a temporary job.
- Check that the volume is mounted on the same server where the service is being redeployed.
- Verify free disk space and volume permissions.
The container starts but the GPU is unused
- Run
docker run --gpus all ubuntu nvidia-smion NVIDIA hosts. If that fails, repair the host driver, NVIDIA Container Toolkit, Docker runtime, or provider passthrough first. - Check that the Compose reservation uses the
nvidiadriver and requests GPU capabilities. - For AMD, confirm the ROCm image and both
/dev/kfdand/dev/drimappings. - Inspect the service logs for device-permission or runtime errors, then redeploy after correcting the host configuration.
The VPS runs out of memory or disk
- Stop pulling larger models until the current model and host remain stable.
- Compare available RAM with Ollama’s guidance for the model class: 8 GB for 7B, 16 GB for 13B, and 32 GB for 33B, before adding host overhead.
- Remove unused model files and rotate logs only after confirming that the files are not part of your backup plan.
- Resize the VPS or move the service to a host with more RAM, VRAM, or disk I/O.
What this setup can and cannot tell you
The documented requirements establish a safe starting point, not a universal performance result. Monthly VPS pricing, provider GPU availability, bandwidth limits, and inference speed vary by region, hardware, model tag, quantization, and concurrency. Measure the exact workload you intend to serve before committing to a production capacity plan.
A CPU-only VPS is usually the least operationally complex option and works well for small models and private experiments. A GPU-backed VPS adds driver, toolkit, device, and provider-passthrough dependencies but can make larger or interactive workloads practical. In either case, persistent storage and a restricted network boundary are more important than a one-click deployment that loses its models on the next restart.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
