The Tool Desk
Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →Outbyte PC Repair FREEClear out junk files and repair common Windows errorsFree Scan →A Kubernetes dashboard that shows every GPU on a node as allocated is accurate, and it still cannot tell you whether those GPUs produced anything useful. The “lying” in the headline is a measurement problem, not a fault in the tools. Kubernetes records which devices a Pod asked for and where it placed them. Utilization, queue delay, interference between workloads and cost per output each need a separate measurement, and each uses a different denominator. Most arguments about AI infrastructure spend start when one of those numbers is quietly read as another.
What Kubernetes actually counts
Kubernetes includes stable support for GPUs through device plugins. The official scheduling documentation says: “Kubernetes includes stable support for managing AMD and NVIDIA GPUs (graphical processing units) across different nodes in your cluster, using device plugins.” GPU scheduling support has been stable since Kubernetes v1.26 (Kubernetes, “Schedule GPUs”).
As an Amazon Associate I earn from qualifying purchases.
A workload requests a GPU the way it requests CPU or memory, by declaring a resource in its Pod spec. For NVIDIA devices the resource name is nvidia.com/gpu:
resources:
limits:
nvidia.com/gpu: 1That line is a placement instruction. The scheduler uses it to find a node with a free device and reserves that device for the Pod. It does not measure what the Pod does with the device once it runs. Two Pods can carry identical lines and generate very different GPU activity, while the Kubernetes object looks the same for both.
#1 Best Overall
- Axial-tech fans now feature a smaller fan hub that facilitates longer blades and a barrier ring that increases downward air pressure
- 2.5-slot design allows for greater build compatibility while maintaining cooling performance
- 0dB technology lets you enjoy light gaming in relative silence
- Dual BIOS switch lets you toggle between Quiet and Performance BIOS profiles
- Dual ball fan bearings last up to twice as long as sleeve bearing designs
The reservation side is visible with kubectl describe node <node-name>, which shows the total resource requests and limits claimed by Pods on that node. Read that output as a reservation ledger. It answers how much of the node is spoken for, not how hard the hardware worked.
Five measurements that are not interchangeable
Teams often use “GPU utilization” to mean any of the following. Each answers a different question, and none of them answers the cost question on its own.
| Measurement | Where it comes from | What it answers | What it does not answer |
|---|---|---|---|
| Requested or allocated GPUs | Pod resource requests scheduled by Kubernetes | How much capacity is reserved for a workload | Whether the reserved device did any work |
| GPU memory assigned | Device memory reserved for a workload or replica | Whether the model and its working set fit | How busy the compute units were |
| Device busy percentage | Device telemetry sampled over a time window | Whether the accelerator was active during the sample | How many useful outputs completed, or which workload used that time |
| Allocated GPU-hours | Cost allocation over time | Reserved time to charge against an owner or budget | Whether that reserved time was productive |
| Completed output | Application metrics such as training steps, successful requests or generated tokens | Useful work per period | What the hardware cost, unless joined to cost data |
A cost argument becomes unreliable when one row is used to justify a conclusion that only another row can support. A high allocation figure with a low busy figure suggests idle reservation, but it does not show which team or job caused it, and it does not show whether the work that ran was worth its cost.
Three cost denominators and why they disagree
The same GPU node produces three different unit costs depending on what you divide by. None of them is wrong. Each supports a different decision.
Rank #2
- Powered by the NVIDIA Blackwell architecture and DLSS 4
- Powered by GeForce RTX 5070 Ti
- Integrated with 16GB GDDR7 256bit memory interface
- PCIe 5.0
- WINDFORCE cooling system
Cost per provisioned GPU-hour
Total cost of GPU capacity over a period, divided by the GPU-hours that capacity was provisioned. Use it for procurement, capacity planning and fleet-level utilization. It charges idle reservation to the asset, so a GPU that sits empty still carries its full hourly cost.
Cost allocated to a workload
The share of cost assigned to a team, namespace or job under an allocation rule. Use it for showback, chargeback or team budgets. The result depends on the rule and on the ownership labels attached to Pods. Two teams that apply different rules to the same node will report different costs, and both can be internally consistent.
Cost per useful output
Total relevant cost divided by a defined output: completed training steps, successful inference requests, generated tokens, or another unit your product measures. This is often the most decision-relevant figure, but it needs application-level metrics and an explicit rule for retries, warm-up time, failed work and shared serving overhead. A failed request that consumed GPU time belongs in the cost, and it does not belong in the output count.
Recommended Free Tools
A hypothetical makes the gap concrete. Suppose a GPU carries an hourly cost of R and is provisioned for 24 hours, but it produces useful output during only 6 of them. Cost per provisioned GPU-hour is R. Cost per productive GPU-hour is 24R ÷ 6 = 4R. Neither figure is a measured result, and R stands in for whatever your provider charges in your region under your billing basis.
Rank #3
- Powered by the NVIDIA Blackwell architecture and DLSS 4
- Powered by GeForce RTX 5060
- Integrated with 8GB GDDR7 128bit memory interface
- PCIe 5.0
- WINDFORCE cooling system
How sharing changes safety, density and measurement
Sharing is the lever most teams reach for when utilization looks low. The three modes below make different trade-offs, and the differences matter more than the headline density number.
| Mode | Isolation | Density | Monitoring | Fit |
|---|---|---|---|---|
| Exclusive device assignment | Isolation at device allocation level | One workload per device; capacity can sit unused when workloads are small or bursty | Not stated in the cited Kubernetes scheduling page | Workloads that keep a whole device busy |
| GPU time-slicing | No memory or fault isolation between replicas | Oversubscribed; workloads interleave on one device | DCGM-Exporter cannot associate metrics to containers when time-slicing is enabled with the NVIDIA Kubernetes Device Plugin | Workloads that can tolerate shared memory and shared faults |
| Multi-Instance GPU (MIG) | Hardware memory and fault isolation on supported GPUs | Predefined instance shapes; less flexible density | Not stated in the cited NVIDIA sharing page | Workloads that need hard isolation and fit a predefined shape |
The isolation and density characteristics above come from NVIDIA’s GPU sharing documentation (NVIDIA, “Time-Slicing GPUs in Kubernetes” and related GPU sharing pages), and they describe how the mechanisms behave, not how much money any of them saves.
Time-slicing: density without isolation
NVIDIA’s documentation puts the core trade-off plainly: “Unlike Multi-Instance GPU, there is no memory or fault-isolation between replicas, but for some workloads this is better than not being able to share at all.” Source: NVIDIA, “Time-Slicing GPUs in Kubernetes.”
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
The counting trap is that requesting multiple time-sliced replicas does not guarantee proportionally more compute. The replicas are processes sharing the same underlying GPU. Asking for four replicas gives you four scheduling slots on one device, not four devices’ worth of throughput. Whether a shared device serves your workloads well has to be measured on those workloads.
Rank #4
- Powered by Radeon RX 9070 XT
- WINDFORCE Cooling System
- Hawk Fan
- Server-grade Thermal Conductive Gel
- RGB Lighting
MIG: isolation with fixed shapes
MIG splits supported GPUs into predefined instances with hardware-level memory and fault isolation. Planning it requires four checks: whether your GPU model is supported, which instance shapes it offers, how much memory each workload needs, and how the layout will pack over time. Packing is the easy part to miss. A partition layout that fits today’s jobs can strand capacity when tomorrow’s job needs a shape that no remaining slice provides.
Why utilization looks low on a shared cluster
Low numbers usually have several causes, and each points to a different fix. Check these in order before changing the sharing mode:
- Whole-device requests for small work. An exclusive assignment reserves a full device for a Pod that may use only part of it. NVIDIA lists low-batch inference, interactive notebooks, bursty rendering and CI among workloads that may benefit from sharing (NVIDIA utilization blog). The word to notice is “may.”
- Metrics at the wrong granularity. Under time-slicing, container-level numbers can mislead, as the monitoring column in the table above explains. Read the device-level figure before you assign blame to a team.
- Work waiting for admission. A GPU can sit idle while jobs queue behind quota or scheduling. Record queue delay as its own metric rather than folding it into utilization.
- Warm-up and data supply. Model loading, warm-up and data pipelines can leave an allocated device waiting. Measure them separately from steady-state compute.
Quotas make scarce GPUs governable, not cheaper
Quota decides who may take a scarce device and when. It does not create compute, and it does not guarantee savings. Kueue, a job-queueing controller for Kubernetes, documents charging GPU types against relative resource credits and using those credits to approximate monetary budgets. Its examples show configuration patterns, not current prices.
| Approach | What counts | Version and maturity notes |
|---|---|---|
| Count-based quota | Whole GPUs by count | Not stated in the cited sources |
| Relative credits per GPU type (Kueue quota example) | Credits weighted by GPU type, used to approximate budgets | Example from the Kueue v0.19 documentation (Kueue quota example); a configuration pattern, not a price list |
| Dynamic Resource Allocation (DRA) accounting | Device count, device counters or consumable capacity, depending on the device and configuration | Requires Kubernetes 1.34 or later per the Kueue documentation; some topology and device-feasibility behavior is alpha and feature-gated in Kueue v0.20 (Kueue DRA documentation) |
Before you enable DRA-based accounting, run kubectl version against the target cluster and confirm the server version is 1.34 or later. Then confirm the deployed Kueue release, because the feature-gate status described above is tied to specific releases. The documentation I checked is dated 7 October 2026, and it will drift.
Best Value
- Axial-tech fans now feature a smaller fan hub that facilitates longer blades and a barrier ring that increases downward air pressure
- Phase-change GPU thermal pad helps ensure optimal heat transfer, lowering GPU temperatures for enhanced performance and reliability
- 2.5-slot design allows for greater build compatibility while maintaining cooling performance
- Dual-ball fan bearings last up to twice as long as standard conventional sleeve bearings designs
- 0dB technology lets you enjoy light gaming in relative silence
Allocation with OpenCost, and why it is not the invoice
OpenCost is an open-source Kubernetes cost monitoring project. Its specification defines the accounting structure: an asset’s cost is the sum of its resource allocation and usage costs, cluster totals also include overhead, and costs split into workload costs and idle costs. For resources billed by allocation, the workload-level model uses the greater of requested and used resources. The specification recommends GPU usage metrics from chipset-specific sources (OpenCost specification). This gives you a consistent internal model. It is a documented allocation rule, not a physical law and not a reproduction of every provider’s invoice.
Cloud pricing enters through two channels, and they play different roles. On-demand price data is a list price. Billing integrations bring in actual charges, but billing data may take several hours to 24 hours to appear, and OpenCost’s configuration documentation states that it does not reconcile on-demand pricing with actual billing costs (OpenCost configuration). Treat a list-price estimate as an estimate until a billing export confirms it.
To reconcile before you claim a saving:
- Export billing line items for the same account, region and date range as your OpenCost report, and wait until the most recent day has landed in the billing data.
- Compare the estimate with billed totals by GPU node pool, not only at cluster level.
- Apply negotiated discounts and committed-use terms that the on-demand price does not reflect.
- Record how idle time, overhead and shared-node costs were allocated, and keep that rule fixed between periods.
- Only then compare cost per useful output across periods.
Where the tools fit
Place each tool against the problem it solves, and treat vendor claims as claims.
Outdated Drivers Are Slowing You Down
One free scan finds every outdated or missing driver and matches the right update for your exact hardware.Free scan · exact hardware matchPC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11- OpenCost handles allocation, idle-cost reporting and showback for Kubernetes workloads.
- Kueue handles admission, quota and queueing. Its DRA integration is version-gated as described above.
- NVIDIA GPU Operator documentation covers time-slicing and MIG configuration on Kubernetes.
- KAI Scheduler is an open-source scheduling solution that NVIDIA identifies in its portfolio.
- NVIDIA Run:ai is NVIDIA’s enterprise orchestration product (NVIDIA Run:ai). Its capabilities and any performance claims are vendor claims. Verify them on your own workloads before you budget around them.
Managed Kubernetes and cloud GPU capacity changes which line items you see, but not the denominators. Provider, region, hardware, discounts and utilization all move the result, so attach a source and date to any price you publish.
Quick Recap
A measurement routine that holds up
- Define the output unit before you measure: a completed training step, a successful request, a generated token, or another unit your business recognizes.
- Capture requested GPUs by namespace and owner label from Pod specs, and label that figure as reservation.
- Read device-level utilization first, and keep queue delay and warm-up time as separate series.
- Compute all three denominators, and write down the allocation rule you used.
- Reconcile against billing before you claim savings.
- When you change the sharing mode, measure throughput, latency, memory pressure and interference on your own workloads before and after. Sharing can reduce cost per output for suitable workloads and can raise latency for others.
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
