Free tools Windows power users keep installed
One-click scans. No signup required.
Running a GPU workload on Kubernetes takes more than adding a GPU to a worker node. Kubernetes supplies the scheduler and resource model, but the administrator must install a vendor driver and a device plugin (or an operator that manages one). The plugin registers an extended resource such as nvidia.com/gpu or amd.com/gpu with kubelet. Pods then request that resource in their container limits. You must also decide between exclusive GPUs, hardware partitioning such as NVIDIA MIG, and software time-sharing, then account for utilization, node compatibility, and the cost of idle accelerator capacity.
What Kubernetes provides—and what it does not
A Kubernetes control plane can schedule a GPU resource only after a node advertises that resource. A stock Kubernetes installation does not contain the proprietary kernel driver, user-space libraries, or vendor device plugin needed to discover a physical accelerator. Those components belong on the GPU worker node.
As an Amazon Associate I earn from qualifying purchases.
The allocation path is:
- The operating system loads the vendor’s GPU driver and exposes the device to the container runtime.
- A vendor device plugin runs on the node and registers an extended resource with kubelet.
- Kubelet reports allocatable capacity to the API server, for example
nvidia.com/gpu. - The scheduler places a pod whose resource request fits the node’s advertised capacity.
- Kubelet invokes the device-plugin interface so the selected device is made available inside the container.
Kubernetes documents AMD and NVIDIA GPU scheduling as stable from Kubernetes v1.26. Stability of the Kubernetes API does not mean every driver, operating-system image, GPU model, or plugin release is interchangeable; those combinations remain vendor-specific.
Prepare a GPU worker node
Confirm hardware and platform compatibility
Inventory the exact accelerator model, host architecture, operating-system kernel, container runtime, and Kubernetes version before installing anything. Check the vendor’s support matrix for that combination. A node can join the cluster successfully while still being unusable for GPU workloads if the driver cannot load or the runtime cannot pass devices into containers.
#1 Best Overall
- AI Performance: 767 AI TOPS
- OC mode: 2632 MHz (OC mode)/ 2602 MHz (Default mode)
- Powered by the NVIDIA Blackwell architecture and DLSS 4
- Axial-tech fan design features a smaller fan hub that facilitates longer blades and a barrier ring that increases downward air pressure
- A 2.5-slot design maximizes compatibility and cooling efficiency for superior performance in small chassis
- Use a server or cloud instance with the intended GPU physically attached and sufficient power, cooling, and PCIe capacity.
- Choose an operating-system image and kernel supported by the driver release.
- Use a container runtime configured according to the vendor’s instructions.
- Keep driver and user-space library versions aligned; a plugin cannot repair a driver that fails to initialize.
Install the driver before the plugin
The driver is the foundation. Validate it on the host with the vendor’s diagnostic utility and inspect kernel logs for initialization errors. Do this before deploying Kubernetes components: if the host itself cannot enumerate the GPU, the device plugin has nothing reliable to register.
Install a device plugin or an operator
A stand-alone device plugin is the minimum Kubernetes integration. It discovers devices, registers a resource, and participates in allocation. An operator deploys and reconciles a broader set of components, commonly including driver management, the device plugin, runtime configuration, and monitoring hooks. The exact bundle varies by vendor and release, so follow the operator’s compatibility matrix rather than mixing manifests from different versions.
Device plugin versus operator
| Decision axis | Stand-alone device plugin | Operator-managed stack |
|---|---|---|
| Primary job | Registers GPU resources and allocates devices to pods. | Manages the plugin plus some combination of drivers, runtime settings, validation, and lifecycle. |
| Operational effort | You install, upgrade, and coordinate each node component. | One controller reconciles several components, reducing manual coordination but adding operator-specific dependencies. |
| Upgrade control | Fine-grained; each component can be changed independently. | Centralized; the operator’s supported version combinations determine upgrade paths. |
| NVIDIA path | Use the NVIDIA device plugin and configure any supported sharing mode. | NVIDIA’s GPU Operator documents a managed component set and GPU-sharing configuration. |
| AMD path | Use the AMD device plugin where appropriate. | AMD documents an operator-managed device-plugin route and a Dynamic Resource Allocation (DRA) driver alternative. |
An operator reduces repetitive node work; it does not eliminate compatibility checks. Verify the supported Kubernetes release, operating system, driver branch, GPU generation, and container runtime before enabling it on production nodes.
Requesting a GPU from a pod
Extended GPU resources are requested in a container’s resources.limits. For these resources, Kubernetes uses a limit as the request when no separate request is supplied. If both fields are present, they must be equal; a request without a corresponding limit is not allowed.
apiVersion: v1
kind: Pod
metadata:
name: gpu-smoke-test
spec:
restartPolicy: Never
containers:
- name: test
image: your-gpu-enabled-image:tag
command: ["sh", "-c", "your-gpu-check-command"]
resources:
limits:
nvidia.com/gpu: 1
The resource key in this example is NVIDIA-specific. Use the exact key shown by your installed plugin; an AMD cluster may advertise amd.com/gpu, and a sharing configuration may expose a different resource name. Do not copy a resource name from another cluster without checking kubectl describe node.
A request for 1 means one schedulable unit as defined by the plugin. In exclusive mode that is normally one whole physical GPU. In a partitioned or time-sliced configuration, the unit may represent an instance or a replica instead. The scheduler does not infer application memory needs, model size, or expected throughput from the integer alone.
How scheduling works in a real cluster
Advertised capacity is the first filter
The scheduler considers a node only when its allocatable extended-resource count can satisfy the pod’s request. CPU and memory requests, pod affinity, taints, and other constraints are evaluated at the same time. A node with a physically installed GPU but no registered extended resource is indistinguishable, for scheduling purposes, from a node with no usable GPU.
Recommended Free Tools
Separate different GPU populations
When a fleet contains multiple GPU generations or vendors, label nodes with attributes that matter to the workload and use an explicit placement rule. A minimal pattern is:
apiVersion: v1
kind: Pod
metadata:
name: labeled-gpu-job
spec:
nodeSelector:
accelerator.example.com/class: training
containers:
- name: trainer
image: your-gpu-enabled-image:tag
resources:
limits:
nvidia.com/gpu: 1
For dedicated GPU pools, taint the nodes and add a matching toleration to GPU workloads. Affinity rules are useful when you need preferences or multiple acceptable classes. The correct label keys, taints, and affinity expressions are cluster-specific; test them against your Kubernetes version and admission policies rather than assuming a universal recipe.
Rank #2
- Powered by the NVIDIA Blackwell architecture and DLSS 4. System Requirements: Minimum 850W PSU with 16-pin 12V-2x6 (12VHPWR) connector required. Verify before purchasing.
- Military-grade components deliver rock-solid power and longer lifespan for ultimate durability. Compatibility: 348mm (13.7") length, 3.6 slots, 4.3 lbs. Confirm case clearance and slot spacing. GPU bracket included.
- Protective PCB coating helps protect against short circuits caused by moisture, dust, or debris
- 3.6-slot design with massive fin array optimized for airflow from three Axial-tech fans
- Phase-change GPU thermal pad helps ensure optimal thermal performance and longevity, outlasting traditional thermal paste for graphics cards under heavy loads
Use node pools as an operational boundary
Separate pools can isolate training, inference, visualization, and batch jobs, each with its own driver cadence and sharing policy. This also makes it easier to drain a pool for a driver upgrade without evicting unrelated CPU workloads. Keep enough spare capacity for rolling maintenance if the workload cannot tolerate interruption.
Choose an allocation and sharing model
| Mode | What a pod receives | Isolation | When it fits | Important limitation |
|---|---|---|---|---|
| Exclusive full GPU | An entire schedulable GPU. | Strongest practical separation between tenants using the device. | Large training jobs, latency-sensitive inference, or workloads that need predictable capacity. | Low-utilization jobs can leave expensive hardware idle. |
| NVIDIA MIG | A hardware-partitioned GPU instance. | NVIDIA describes memory and fault isolation between instances. | Multiple workloads need predictable slices of a compatible NVIDIA GPU. | Only supported GPU models and profiles can create the required partitions; capacity is profile-specific. |
| NVIDIA time-slicing | A plugin-exposed replica that interleaves access to the same GPU. | Does not provide MIG’s memory or fault isolation. | Short, bursty, or development workloads where sharing is more valuable than strict isolation. | Requesting several replicas does not promise a proportional share of compute or throughput. |
Sharing changes the meaning of capacity. Time-slicing can raise apparent allocatable counts while several containers contend for the same physical execution resources. Treat those counts as admission units, not guaranteed performance units. Benchmark the actual model, batch size, and concurrency before setting service-level expectations.
MIG and time-slicing are not interchangeable. MIG’s hardware partitioning is intended to isolate memory and faults; time-slicing primarily multiplexes access. Choose based on tenant trust, failure impact, latency targets, and whether the workload can tolerate noisy neighbors.
NVIDIA and AMD are not configured identically
Kubernetes offers a common scheduling model for both vendors, but the node integration is different. NVIDIA deployments commonly register nvidia.com/gpu and may add MIG or time-slicing configuration through the device plugin or GPU Operator. AMD deployments use the AMD-supported plugin and resource naming, with an operator and DRA option documented by AMD.
Do not assume that an NVIDIA manifest, sharing mode, diagnostic command, or operator value can be transferred to AMD hardware. Confirm the resource name visible on the node, the supported driver/runtime combination, and the vendor’s procedure for partitioning or sharing. If a workload image depends on vendor-specific libraries, select the matching node pool explicitly.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.A practical installation and validation workflow
- Inventory the target. Record GPU model, host OS and kernel, Kubernetes version, runtime, and intended sharing mode.
- Build or select the node image. Include the supported vendor driver and any required runtime integration. Reboot if the driver installation requires it.
- Validate the host. Confirm the operating system can enumerate the GPU and that the vendor diagnostic utility reports it without errors.
- Deploy the plugin or operator. Pin versions that the vendor documents as compatible with your Kubernetes release. Wait for the DaemonSet or operator-managed components to become ready on every GPU node.
- Inspect registration. Run
kubectl describe node <gpu-node>and check that the expected extended resource appears under capacity and allocatable. - Run a smoke-test pod. Request one unit using the exact resource key and an image containing a vendor-compatible diagnostic or minimal GPU application.
- Verify placement and visibility. Check pod events, the assigned node, and the application’s view of the device. A scheduled pod that cannot open the device indicates a runtime, driver, permissions, or plugin problem rather than a scheduler decision.
- Apply placement policy. Add labels, taints, tolerations, or affinity so workloads land only on compatible pools.
- Measure the chosen sharing mode. Observe memory pressure, latency, and throughput under realistic concurrency before advertising capacity to users.
- Document upgrades and rollback. Record the driver, plugin/operator, Kubernetes, and image versions together. Drain and canary a node before changing the production pool.
Troubleshooting common failures
| Symptom | Likely cause | Checks and corrective action |
|---|---|---|
Insufficient nvidia.com/gpu or equivalent |
No free registered units, wrong resource key, or the pod is constrained to the wrong pool. | Inspect node allocatable values and pod events. Confirm the manifest uses the plugin’s exact resource name and review selectors, affinity, taints, and tolerations. |
| GPU node shows no extended resource | Plugin DaemonSet is absent, crash-looping, or unable to contact kubelet; the driver may also be missing. | Check plugin logs and node conditions, then validate the host driver independently. Fix the node integration before changing the workload. |
| Pod schedules but the application sees no GPU | Container runtime integration, device mounts, permissions, or user-space libraries are incorrect. | Inspect the pod’s events and container logs, compare the image libraries with the host/plugin requirements, and run the vendor’s minimal diagnostic image. |
| Sharing produces inconsistent latency | Time-sliced replicas contend for the same physical GPU. | Reduce concurrency, move latency-sensitive work to exclusive or MIG-backed capacity, or set expectations using measured workload results. |
| Upgrade breaks all GPU pods | Driver, plugin/operator, runtime, and Kubernetes versions are outside a supported combination. | Use the documented compatibility matrix, canary a replacement node, and keep the previous known-good image and manifests available for rollback. |
What GPU capacity really costs
GPU instances and servers are expensive infrastructure, and setup guidance for Kubernetes distributions such as kOps explicitly warns administrators to minimize workloads on GPU nodes. There is no universal current price or cost-per-workload figure: region, hardware generation, utilization, power, and operating model change the answer.
The Tool Desk
Outbyte PC Repair FREEClear out junk files and repair common Windows errorsFree Scan →Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →Build a calculation from your own numbers:
- Infrastructure: hourly cloud instance or amortized server and GPU cost.
- Utilization: the percentage of time the accelerator performs useful work rather than waiting, warming, or sitting idle.
- Supporting resources: CPU, memory, local storage, network bandwidth, images, and data-transfer charges.
- Owned-hardware overhead: electricity, cooling, rack space, spares, and hardware maintenance.
- Operations: driver and plugin upgrades, monitoring, incident response, and engineering time.
Exclusive allocation is simplest to reason about but can strand capacity when jobs are small. MIG can divide a compatible GPU into isolated instances, while time-slicing may improve admission density for tolerant workloads; neither mode guarantees lower cost unless utilization and workload behavior improve in practice. Compare modes using measured utilization and service objectives, not the advertised replica count.
Production checklist
- Every GPU node passes a host-level driver test.
- The plugin or operator is ready and registers the expected extended resource.
- Pod limits use the exact vendor resource name, and requests equal limits when both are specified.
- Node labels and taints prevent incompatible workloads from landing on a pool.
- The selected sharing mode’s isolation and performance characteristics are documented for users.
- Monitoring covers node health, plugin errors, allocatable capacity, utilization, memory pressure, and pod-level failures.
- Driver, plugin/operator, Kubernetes, runtime, and image versions are tracked as one tested combination.
- Capacity planning includes idle time, maintenance headroom, and the cost of supporting CPU, storage, networking, power, and cooling.
The reliable mental model is straightforward: the vendor makes the GPU usable on the node, the device plugin turns it into a Kubernetes resource, and the scheduler places pods that request that resource. Everything beyond that—partitioning, sharing, isolation, and economics—must be chosen for the hardware and workload you actually operate.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
