Short answer: Use a GPU node pool to decide which accelerator-equipped machines are available, and use GPU sharing to decide how workloads divide a device on those machines. Choose a separate pool for a particular GPU model, zone, driver lifecycle, autoscaling policy, or dedicated-capacity requirement. Add MIG, MPS, or time-slicing inside that pool only when the workloads can tolerate the selected isolation and performance behavior.
These are complementary decisions, not competing deployment models. Kubernetes schedules vendor-advertised extended resources such as nvidia.com/gpu; the device plugin, node labels, taints, tolerations, and workload requests determine placement. Sharing then controls whether a device is partitioned or multiplexed. Kubernetes GPU scheduling has been stable since v1.26, but driver, device-plugin, operator, GPU-model, and cloud-provider behavior still has to be verified for your cluster.
As an Amazon Associate I earn from qualifying purchases.
What a GPU node pool controls—and what it does not
A GPU node pool is a group of worker machines with a defined accelerator configuration. It answers the placement question: which machines may run this pod? A provider can give the pool its own GPU model, zones, autoscaling policy, operating-system image, driver lifecycle, and taints. Google Kubernetes Engine (GKE), for example, recommends separate GPU pools, autoscaling, and pools dedicated to different accelerator types in its GPU guidance. That is GKE guidance, not a universal requirement for every Kubernetes distribution.
A pool does not automatically mean that one pod owns an entire physical GPU. Whole-device allocation, MIG partitions, MPS limits, and time-slicing are separate resource-allocation choices. You can run one workload per GPU in a shared pool, or share devices among several pods in a pool created for one accelerator model.
#1 Best Overall
- NVIDIA Volta GV100 Architecture — 4,608 CUDA Cores, 640 1st-Gen Tensor Cores delivering 14 TFLOPS FP32 and 112 TFLOPS deep learning performance for AI training, inference, HPC, and scientific computing workloads
- 32GB HBM2 ECC Memory — 900 GB/s Bandwidth — High-bandwidth memory on a 4096-bit bus with ECC error correction provides the memory capacity and throughput required for the largest AI models, simulations, and datasets
- PCIe 3.0 x16 Interface — 250W TDP — Standard PCIe Gen3 connectivity with passive cooling designed for enterprise rack server deployment in HPE ProLiant, Dell PowerEdge, and Supermicro platforms with adequate chassis airflow
- NVLink — Scale to 96GB Unified Memory — Connect two V100 GPUs via NVLink at 300 GB/s bi-directional bandwidth to scale GPU memory from 32GB to 96GB for larger AI training and HPC workloads
- Multi-Precision Computing — Supports FP64 (7 TFLOPS), FP32 (14 TFLOPS), FP16 (112 TFLOPS) and INT8 precision modes for flexible deployment across training, inference, and scientific simulation workloads
Node-pool separation is also a scheduling boundary rather than a complete security boundary. Taints and tolerations keep unsuitable pods away from GPU nodes, while labels and node-placement rules steer compatible pods toward the right hardware. Neither mechanism alone guarantees that mutually untrusted workloads cannot exploit another path in the cluster.
How Kubernetes schedules GPUs
Kubernetes does not discover a GPU by itself. A vendor device plugin, together with the driver stack, advertises an extended resource to the kubelet. The standard Kubernetes explanation is in the device-plugin documentation. A common NVIDIA resource name is nvidia.com/gpu.
Once the resource is advertised, a pod requests it in its container resource limits. A minimal request looks like this:
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
apiVersion: v1
kind: Pod
metadata:
name: gpu-worker
spec:
containers:
- name: worker
image: your-image
resources:
limits:
nvidia.com/gpu: 1The example shows the resource request only; the image, driver compatibility, command, and sharing-specific resource names depend on your environment. Kubernetes places the pod only on a node that advertises the requested resource. Placement constraints then narrow that choice to the required accelerator type, zone, pool, or lifecycle.
Install and validate the driver and device plugin before troubleshooting scheduling. If no node advertises the resource, a pod requesting it remains unschedulable regardless of its node selector. If the wrong GPU type is advertised, a pod may schedule successfully and still fail at runtime because its software or memory requirement is incompatible.
The three main ways to share a GPU
Time-slicing
Time-slicing interleaves work from multiple workloads on the same physical device. It is useful when compatible, trusted jobs have intermittent demand and access to otherwise idle hardware is more valuable than strict per-pod guarantees. NVIDIA’s GPU Operator documentation states that, unlike MIG, time-sliced replicas have no memory or fault isolation; the same document notes that this can still be preferable to not sharing at all: Unlike Multi-Instance GPU (MIG), there is no memory or fault-isolation between replicas, but for some workloads this is better than not being able to share at all.
See the versioned NVIDIA GPU sharing documentation.
A time-sliced replica is a scheduling entitlement, not a fractional GPU with a guaranteed percentage of compute. Requesting multiple replicas does not establish proportionally higher throughput. Kernel duration, memory pressure, context switching, and simultaneous demand determine the observed behavior, so benchmark representative jobs before promising latency or throughput.
Do these 3 things before closing this tab:
1Fix the driver behind crashes, sound loss and screen glitches2Clear out junk files and repair common Windows errors3Scan for outdated or missing drivers - takes under a minuteTime-slicing can support older GPUs that do not provide MIG, according to the NVIDIA Operator documentation. It is therefore the broadest hardware option, but also the option with the weakest isolation and predictability.
Multi-Instance GPU (MIG)
MIG divides a supported accelerator into hardware-level instances. Each instance has a defined slice of memory and compute resources, with stronger memory and fault isolation than time-slicing. NVIDIA’s DRA guide identifies Ampere-or-newer NVIDIA data-center GPUs as MIG-capable, but the exact model, operating mode, driver, firmware, and software stack must be checked before you design a cluster around it. The current guide is at NVIDIA’s MIG documentation.
MIG profiles are predefined by the hardware. You cannot assume that any arbitrary percentage of memory or compute can be carved out. Select a profile whose memory capacity and compute resources fit the workload, then verify how the device plugin or DRA driver exposes that profile to Kubernetes. A profile that is too small can cause out-of-memory failures; a larger profile may strand capacity when jobs cannot use the remainder.
Multi-Process Service (MPS)
MPS is a software-mediated sharing mode documented separately from time-slicing by the NVIDIA device plugin. Its documentation describes managing access while permitting compute and memory limits per workload. The plugin treats MPS and time-slicing as mutually exclusive configuration choices, so select one rather than enabling both on the same device-plugin configuration. Read the exact semantics and compatibility requirements in the NVIDIA device-plugin README.
Recommended Free Tools
MPS can be appropriate when concurrent processes need explicit resource controls but do not require MIG’s hardware partitions. Because behavior depends on the CUDA stack, driver, device-plugin release, and application process model, test memory enforcement and interference with the same workloads you will run in production.
Rank #3
- Colour: brown
- Brand: Nvidia
- Packed with features
- Best product in its class
Node pools and sharing compared
| Decision axis | Dedicated GPU node pool | GPU sharing inside a pool |
|---|---|---|
| Primary control | Which GPU-equipped machines can accept a workload; pools can be specialized and scaled independently in provider implementations such as GKE. | How several workloads access a physical device or a hardware partition on one machine. |
| Isolation | Separates workloads by node placement, but taints and pools are not a complete security boundary. | Time-slicing has no memory or fault isolation; MIG supplies hardware-isolated instances; MPS supplies software controls described by its plugin documentation. |
| Utilization | Reserves expensive hardware for matching jobs, but intermittent demand can leave nodes idle. | Can use otherwise idle capacity when contention and shared-fault behavior are acceptable. No universal utilization gain is established. |
| Predictability | Whole-GPU allocation is preferable when software expects exclusive access or performance must be tightly bounded. | Depends on the mode. Time-slicing does not promise proportional compute for multiple replicas; MIG profiles are fixed; MPS limits require workload testing. |
| Hardware fit | Useful when GPU models, zones, images, or driver lifecycles differ. | MIG requires a supported model; time-slicing also covers older GPUs; MPS depends on the NVIDIA software path. |
| Operations | Requires pool provisioning, placement rules, autoscaling, and driver lifecycle management. | Adds sharing configuration, resource-semantic validation, interference monitoring, and version pinning. |
Choose the architecture by workload behavior
Large training jobs
Use whole-device allocation when a training job needs the GPU’s full memory and sustained compute, or when its framework assumes exclusive access. Place those jobs on a pool containing the required accelerator type and enough capacity for the job’s topology. Sharing can create unpredictable interference and memory pressure during long-running training steps.
Latency-sensitive inference
Prefer dedicated GPUs when tail latency, admission predictability, or repeatable throughput matters more than packing density. A separate pool can keep inference hardware away from batch jobs and can scale it under a distinct policy. If the model fits comfortably in a MIG profile, MIG is the sharing option to evaluate; do not assume time-slicing will preserve latency under contention.
Bursty or intermittent inference
Time-slicing may fit small, trusted services whose demand peaks at different times and whose owners accept shared memory and fault domains. Treat each replica as permission to use the device, not as a guaranteed fraction of it. Measure queueing and tail latency during simultaneous requests before setting service-level objectives.
Free tools Windows power users keep installed
One-click scans. No signup required.
Concurrent processes needing limits
Evaluate MPS when processes must run concurrently and you need explicit compute and memory controls without a hardware partition. Confirm the current device-plugin and CUDA compatibility, then test what happens when one process reaches its limit or crashes.
Mutually untrusted or sensitive workloads
Do not use time-slicing merely because it increases packing density. Its replicas share memory and fault domains. Prefer dedicated whole-GPU allocation, and combine node placement with the cluster’s established identity, admission, runtime, and network security controls. A separate pool by itself does not eliminate every privilege-escalation path; GKE makes this limitation explicit in its dedicated-node guidance.
Mixed accelerator generations
Create separate pools when applications need different GPU models, zones, driver branches, or operating-system images. Use labels and placement constraints so a pod cannot land on a model that lacks the required memory, instruction set, or MIG capability. Sharing can then be configured independently within each compatible pool.
Rank #4
- Bulk Pack without retail box
When combining a node pool with sharing is the best answer
Combine the mechanisms when you need both hardware selection and higher utilization. For example, create a pool for one supported accelerator model, taint it so only GPU workloads tolerate the taint, label it for placement, and configure MIG or time-slicing on those nodes. The pool answers “where can this run?” while the sharing mode answers “how is capacity divided there?”
This combination is especially useful when several small services require the same GPU generation but do not each justify a whole device. It also lets an autoscaler add or remove a homogeneous group instead of mixing incompatible accelerators. Keep the pool homogeneous enough that a pod’s resource request has the same meaning on every node in the group.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.A rollout procedure that avoids common scheduling mistakes
- Describe workload requirements. Record required GPU memory, expected concurrency, latency tolerance, whether the process assumes exclusive access, and whether workloads are mutually trusted. Decide whether the requirement is a full device, a MIG profile, an MPS-limited share, or a time-sliced entitlement.
- Inventory hardware and software support. Confirm the exact GPU model, firmware, driver, CUDA compatibility, and operator or device-plugin release. For MIG, verify that the model is supported and that the chosen mode is available. For MPS, verify the plugin’s documented compatibility. For time-slicing, verify that the application tolerates shared memory and fault domains.
- Install and verify the driver and device plugin. Follow the Kubernetes device-plugin model described at kubernetes.io. Check that every intended node advertises the expected extended resource and that a test pod can initialize the CUDA runtime.
- Create or select the GPU pool. In a managed service, choose zones where the desired accelerator is available and configure autoscaling for that pool. GKE’s GPU documentation specifically recommends separate pools, autoscaling, and accelerator-aware zone selection. Do not transfer those provider-specific settings unchanged to another Kubernetes service.
- Apply placement controls. Label nodes with the accelerator identity used by your platform, taint GPU nodes to repel unrelated workloads, and add matching tolerations and node-selection rules to GPU pods. Check that the labels, taints, and pool membership agree; a typo can leave a pod pending even when a suitable GPU is idle.
- Configure one sharing mode. Select MIG, MPS, or time-slicing according to the isolation and control requirements. In the NVIDIA device plugin, MPS and time-slicing are mutually exclusive choices. Treat a sharing configuration as part of the node image or operator lifecycle so replacement nodes receive the same behavior.
- Request the resource explicitly. Use the resource name advertised by the installed plugin, commonly
nvidia.com/gpu, in the container’s limits. Sharing modes may expose resources differently, so inspect the plugin’s versioned documentation rather than assuming that a whole-GPU request and a shared-instance request have identical names. - Run interference and failure tests. Start one workload, then add concurrent workloads until the expected contention level is reached. Measure memory usage, queueing, latency, throughput, and recovery after a process or node failure. Repeat after upgrades because driver and plugin changes can alter resource discovery or sharing semantics.
- Set operational guardrails. Pin compatible operator and device-plugin versions, document the GPU model and sharing mode, alert on unschedulable GPU pods and device-plugin errors, and keep a rollback path to whole-device allocation. Do not publish a utilization or cost target until representative jobs have been measured on the actual platform.
GKE-specific considerations
On GKE, the provider recommends separate GPU node pools and autoscaling, restricting pools to zones where the selected accelerator is offered, and using taints, tolerations, and accelerator labels for placement. Its documentation also describes version-dependent NVIDIA driver installation behavior, so check the current control-plane version and the driver-installation path before choosing a node image or upgrade sequence: GKE GPU node pools.
Dedicated-node scheduling can reduce efficiency when demand is intermittent. Google states: Using a separate node pool to isolate your workloads can impact your cost efficiency, autoscaling, and resource utilization.
That trade-off is a reason to measure pool occupancy and pending time, not a reason to place incompatible workloads on the same hardware.
Performance, utilization, and cost trade-offs
No single architecture has a universal cost or performance break-even point. The outcome depends on GPU model and price, workload concurrency, memory footprint, pool startup time, autoscaler behavior, and the isolation level you require. Official documentation does not establish a universal percentage improvement for sharing.
Dedicated pools can strand capacity when jobs arrive sporadically, but they make capacity and performance easier to reason about. Sharing can reduce idle periods, yet contention, memory pressure, and shared failure domains can increase tail latency or recovery work. Compare the two designs with the same representative workload mix and record both successful work and operational overhead.
Quick Recap
Troubleshooting by symptom
The pod is pending with an insufficient-GPU message
- Confirm that the driver and device plugin are running on the intended nodes.
- Inspect node capacity and allocatable resources to verify that the advertised resource name matches the pod request.
- Check taints, tolerations, labels, selectors, zones, and pool autoscaler limits.
- For MIG or sharing, verify that the configured mode exposes the resource type the pod requests; a whole-GPU name may no longer represent a shared instance.
The pod schedules but the application cannot initialize the GPU
- Check driver, CUDA, container runtime, and framework compatibility.
- Verify that the pod landed on the intended accelerator model rather than merely any GPU node.
- Inspect device-plugin and operator logs for registration or health errors.
- Reproduce with a minimal CUDA test before debugging the application itself.
Throughput falls when replicas increase
- With time-slicing, expect contention and remember that multiple replicas do not guarantee proportional compute.
- With MIG, check whether the selected profile has enough memory and compute for the model.
- With MPS, verify the configured limits and observe whether one process is throttled or causing memory pressure.
- Run the same concurrency test on whole-device allocation to separate application limits from sharing overhead.
A workload reaches the wrong pool
- Compare the node label used by the pod with the labels actually applied to nodes.
- Check that the pod tolerates only the intended GPU-pool taint.
- Remove stale selectors after a pool migration and verify zone availability for the requested accelerator.
A practical decision checklist
- Choose a dedicated GPU pool when you need a particular accelerator type, zone, driver lifecycle, autoscaling boundary, or predictable whole-device performance.
- Choose MIG when the exact accelerator supports it and you need hardware-isolated, predefined partitions.
- Choose MPS when supported concurrent processes need software-enforced compute and memory controls.
- Choose time-slicing when workloads are compatible and trusted, can tolerate shared memory and fault domains, and better access to idle devices outweighs strict per-pod guarantees.
- Combine a specialized pool with sharing when you need both homogeneous hardware placement and higher utilization.
- Do not treat a node pool, taint, or toleration as a complete security boundary.
- Pin and verify the driver, device-plugin, operator, GPU model, and configuration after every upgrade.
- Use measured representative workloads—not a generic utilization promise—to set capacity, latency, and cost expectations.
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
