Recommended Free Tools
Yes, a Kubernetes workload can run at zero replicas and still wake on demand, but only when the scaler reads a signal that exists while no worker Pods are running, such as queue depth or another external or object metric. Scale-to-zero is not automatically cheaper, and it does not remove cold starts, request-holding problems, or GPU provisioning delays. The design choices below show where those costs appear and when keeping a warm minimum is the better decision.
How do I scale a Kubernetes workload to zero?
Scaling to zero needs two things: a controller that can set the replica count to zero, and a signal that still reports demand after the last worker Pod has gone. The second requirement shapes the whole design. CPU and memory metrics come from Pods, so once a Deployment has no Pods there is nothing to report them and nothing to wake the workload. Event-driven designs read demand from outside the workload instead, using queue depth, a topic backlog, a cloud metric, or the result of a database query.
| # | Preview | Product | Price | |
|---|---|---|---|---|
| 1 |
|
MINISFORUM MS-02 Ultra Workstation Mini PC, Intel Core Ultra 9 285HX (24C/24T, up to 5.5GHz), PCIe... | $1,659.00 | Buy on Amazon |
| 2 |
|
GMKtec EVO-X2 AI Mini PC Ryzen Al Max+ 395 Superchip 128GB LPDDR5X 2TB SSD | $3,649.99 | Buy on Amazon |
As an Amazon Associate I earn from qualifying purchases.
Three Kubernetes-native routes are in common use. Each reads demand differently, so choose by where the signal lives and how requests reach the workload.
Do these 3 things before closing this tab:
1Fix the driver behind crashes, sound loss and screen glitches2Clear out junk files and repair common Windows errors3Scan for outdated or missing drivers - takes under a minuteKEDA with the Horizontal Pod Autoscaler
KEDA monitors supported event sources and exposes their metrics to the Kubernetes Horizontal Pod Autoscaler (HPA). It handles the transition between zero and one replica: it activates the workload when a trigger is met and deactivates it when the trigger falls below its threshold. Above one replica, the HPA makes the scaling decisions. The KEDA project homepage lists more than 70 built-in scalers across cloud platforms, databases, messaging, telemetry, and CI/CD. That count describes catalog coverage, not performance or reliability.
#1 Best Overall
- High-Performance AI Processor：The MS-02 Ultra features an Intel Core Ultra 9 285HX (24C/24T, up to 5.5 GHz, 13 TOPS NPU), delivering fast and efficient performance for AI inference, algorithm development, and media workloads. A PCIe x16 expansion slot supports desktop-class GPU upgrades for advanced model training and accelerated computing tasks. It's ideal for creators, engineers, and teams handling intensive parallel workloads.
- 4 × M.2 PCIe 4.0 + 4 × DDR5 SODIMM slots:Four DDR5 SODIMM slots support up to 256 GB of memory, while ECC helps maintain data integrity in mission-critical environments. Four PCIe 4.0 M.2 slots support up to 24 TB of storage, supporting RAID 0/1/5/10, combining high-speed performance with data protection. It allows for the creation of independent scratch disks, media libraries, and project drives, providing high-throughput for production workflows.
- PCIe & USB 4.0 v2: Up to three PCIe slots can be equipped, including a dual-slot x16 GPU. The main slot supports PCIe 5.0, meeting the needs of high-bandwidth creative and computing workloads. USB 4.0 v2 (80Gbps) supports high-bandwidth external storage and displays.
- Ultra-fast Networking: Wi-Fi 7 further enhances wireless performance with next-generation speeds and low-latency stability. Intelligent bandwidth switching optimizes throughput in different network environments, ensuring optimal performance for enterprise or local networks. Dual 25GbE ports (providing up to approximately 3.125 GB/s bandwidth, about 25 times faster than traditional 1GbE), enabling seamless large-scale file transfers and parallel computing. 10GbE and 2.5GbE ports, with support for Intel vPro technology, ensure enterprise-grade remote management and deployment flexibility.
- Server-grade thermal architecture: Utilizing a dedicated CPU/GPU airflow design, equipped with a 6-pipe dual-fan cooler, it maintains stable performance even under sustained loads, delivering up to 140W Turbo power while maintaining a 100W TDP, and operating with noise levels as low as 36 dB. An integrated 350W power supply ensures stable and reliable output for demanding computing tasks and fully loaded extended configurations.
Kubernetes v1.37 HPA
The Kubernetes project post dated September 2, 2026 reports that v1.37 adds API support for scaling workloads to zero replicas through the core HPA. The post was written by Johannes Würbach, who states: “Kubernetes v1.37 includes API support for horizontal autoscaling of workloads down to zero replicas.” The feature is marked Beta and enabled by default in that release. It works only with object or external metrics. Confirm that your cluster version includes the feature and that your managed Kubernetes provider exposes it before designing around it.
Knative Serving
Knative Serving’s default autoscaler, the Knative Pod Autoscaler, responds to incoming request demand and can scale a service to zero when no traffic arrives, provided scale-to-zero is enabled. Set concurrency and scale bounds for each service. Use the minimum-scale annotation (autoscaling.knative.dev/min-scale) with a value of 0 on services that may idle, and keep a minimum of one on services that cannot absorb a cold start.
How can I scale an LLM workload to zero?
The same wake-up pattern applies to an LLM endpoint, but the first request now waits for more than a container start. Google Cloud’s GKE tutorial demonstrates an Ollama LLM deployment scaled with KEDA-HTTP. That design places an HTTP activation layer in front of the model and configures a GPU node pool with node autoscaling. The tutorial shows the topology; it does not publish a measured startup time for the model.
Free tools Windows power users keep installed
One-click scans. No signup required.
What the first request waits for
- Node provisioning. If the GPU node pool is also at zero, a machine must be created before any Pod can be scheduled.
- Image pull and container start for the inference server.
- Model loading. Weights must be read into host memory and onto the GPU before the readiness check passes.
- Activation delay. The scaler must observe the signal, on its polling interval, before it sets replicas above zero.
- Queue wait for requests that arrive while earlier replicas are still starting.
Official project and provider documentation does not publish a cold-start figure for an LLM on these platforms, so measure each stage on your own model, container image, and GPU type.
Interactive versus asynchronous inference
A queue is a natural buffer for work that can wait. A chat request cannot sit silently for minutes. The Kubernetes project’s v1.37 post states: “Kubernetes Services do not buffer requests while no Pods are ready, so HTTP and other request-driven workloads need a separate buffering layer.” For interactive inference, the activation and holding path is a component you must design. Options include an HTTP activation layer like the one in the GKE example, Knative Serving once you have verified how it holds requests during activation in your version, or a gateway that holds requests for a bounded time and returns a clear retry response when the budget is exceeded.
How do I autoscale from a queue?
Queue-driven scaling is the most direct case because the queue stores work while the workers are absent. The chain runs as follows: a producer submits work to a durable queue or topic; an external metric reports the backlog; KEDA or an HPA-compatible controller observes that metric and raises the worker replica count; workers consume jobs and write results; and when the backlog and trigger fall below the configured activation thresholds, the Deployment returns to zero. AWS’s EKS guidance describes this model, with a KEDA operator activating and deactivating the Deployment and providing custom metrics to the HPA. Google Cloud’s GKE example uses a Pub/Sub scaler for the same job.
Setup steps
- Choose a queue or topic that provides retention, redelivery, and a dead-letter path. The EKS example uses Amazon SQS; the GKE example uses Pub/Sub.
- Give the scaler a read-only identity for the queue metric through your cluster’s workload identity mechanism, rather than long-lived static credentials in the Deployment.
- Define the scaler as a KEDA
ScaledObjectthat targets the worker Deployment. The sample below uses an SQS trigger. - Set the scale-down cooldown and the maximum replica count. The maximum is also your cost ceiling and your GPU quota demand.
- Make workers acknowledge a message only after the result is durably written. A message acknowledged early is lost if its worker is terminated during scale-down.
- Set the queue’s visibility timeout, or the equivalent acknowledgement deadline, longer than worst-case processing time, or messages will redeliver while still in progress.
- Test the wake path: send one message to an empty queue, confirm the Deployment moves from 0 to 1, confirm the message completes, and confirm the Deployment returns to 0 after cooldown.
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
name: inference-worker
spec:
scaleTargetRef:
name: inference-worker
minReplicaCount: 0
maxReplicaCount: 8
cooldownPeriod: 300
pollingInterval: 30
triggers:
- type: aws-sqs-queue
metadata:
queueURL: https://sqs.REGION.amazonaws.com/ACCOUNT_ID/inference-jobs
queueLength: "5"
awsRegion: REGION
In this sample, queueLength is the backlog per replica that drives scaling. The field names follow KEDA’s AWS SQS scaler; confirm them against the scaler reference for your KEDA version.
Rank #2
- EVOLUTION RYZEN AI MAX+ 395 MINI PC - GMKtec EVO-X2 is the next evolution in AI mini PC Ryzen Strix Halo series. Thanks to AMD Simultaneous Multithreading (SMT) the core-count is effectively doubled, to 32 threads. Ryzen AI Max+ 395 has 64 MB of L3 cache and can boost up to 5.1 GHz, depending on the workload. The Ryzen AI Max+ 395 is currently rated as the "most powerful x86 APU" on the market for AI computing.
- AI NPU with XDNA 2 ARCHITECTURE - Powered by 16 “Zen 5” CPU cores, 50+ peak AI TOPS XDNA 2 NPU and a truly massive integrated GPU driven by 40 AMD RDNA 3.5 CUs, the Ryzen AI MAX+ 395 is a transformative upgrade and delivers a significant performance boost over the competition. The Ryzen AI Max+ 395 excels in consumer AI workloads like the llama.cpp-powered application: LM Studio. Shaping up to be the must-have app for client LLM workloads, LM Studio allows users to locally run the latest language model without any technical knowledge required and unleash their creativity and productivity.
- AMD RADEON 8090S iGPU GAMING PC - The AMD Radeon RX 8060S offers all 40 CUs with up to 2.9 GHz graphics clock and uses the new RDNA 3.5 architecture. The powerful iGPU is positioned between an RTX 4060 and 4070 laptop GPU and therefore enables gaming in FHD at maximum details in most demanding games. The 8060S can also utilize the full 128GB pool, which is perfect for running LLMs such as Deepseek 70B Q8, which runs comfortably on this machine.
- EIGHT CHANNEL LPDDR5X - LPDDR5X is a new ground breaking memory small form factor installed on-board. With blazing speeds up to to 8000MT/s, it runs 1.5x faster than the DDR5 SODIMMs; 90% better performance over DDR5 SODIMMs in video conferencing and photo editing; 30% better performance in productivity apps; 12% better performance in digital content workloads.
- QUAD SCREEN 8K DISPLAY SUPPORT - EVO-X2 AI Mini PC support 4-screen 4K/8K output via HDMI 2.1 (8K@60Hz), DisplayPort 1.4 (4K@60Hz), and dual USB 4 40Gbps Transfer speed (supporting PD3.0/DP1.4/DATA). Ideal for gaming, video editing, and multitasking, it provides expansive and crisp multi-display support.
Choosing a pattern
The patterns differ mainly in what they wake from and where requests wait. Read the table against your workload’s tolerance for a delayed first response.
| Pattern | Wakes from | Best fit | Benefits | Trade-offs to design for |
|---|---|---|---|---|
| KEDA with HPA | Queue depth, topic backlog, cloud, database, or other external metrics | Asynchronous consumers and batch workers | Broad event-source coverage; HPA handles scaling above activation | Identity and scaler permissions, thresholds, polling, minimum and maximum replicas, queue semantics; cold starts remain |
| Knative Serving | Incoming HTTP requests | Containerized HTTP services | Request-driven autoscaling with optional scale-to-zero | Concurrency and scale bounds; request holding during activation and the cold-start budget must be verified |
| Kubernetes v1.37 HPA | Object or external metrics that persist at zero | Teams that want the core HPA on a cluster that includes the feature | Scale-to-zero through the core HPA API, with no additional controller | Requires a signal that persists at zero; confirm cluster version and provider support |
| Managed KEDA add-on on AKS | The same triggers as upstream KEDA | Provider-integrated AKS clusters | Less installation work and provider identity guidance | Version and configuration limits; Microsoft documents limits on modifying some KEDA component values |
Evaluate candidate approaches on these axes before committing:
- Event metric support, including whether the metric exists while zero Pods are running.
- HTTP activation and request holding, and what the client sees during activation.
- Acceptable cold-start time, measured end to end.
- Queue durability, retry behavior, and dead-letter handling.
- Model load time and GPU availability.
- Identity and secret handling.
- Scale bounds and per-replica concurrency.
- Node-pool scaling behavior, including whether nodes return to zero.
- Total cost across idle and active periods.
Official project and provider documentation does not publish an apples-to-apples cost or latency comparison across these options, so that comparison has to come from your own measurements.
When should a workload keep a warm minimum?
Not every inference endpoint suits scale-to-zero. Use this checklist to decide per workload.
Quick wins for a faster PC:
Clear out junk files and repair common Windows errorsFree Scan →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Repair Windows errors before they cause bigger problemsFix Now →- Work can be queued and retried safely: scale-to-zero is a strong candidate.
- Requests carry an interactive latency budget and the measured cold start exceeds it: keep at least one replica warm.
- Model load is long and the GPU node is slow or costly to provision: keep one replica on that node pool rather than paying the full wake path on every burst.
- Traffic arrives in long, irregular gaps: scale-to-zero can remove a meaningful share of idle compute.
- Traffic is steady with short gaps: scale-to-zero removes little idle time, so the cold-start risk may not be worth taking.
- Requests have side effects that cannot be safely retried: settle idempotency before allowing the workload to scale down.
What scale-to-zero does not remove
Pod scale-to-zero is not cluster scale-to-zero, and it does not make an idle architecture free. Kubernetes documentation covers removing idle Pods. The GKE example handles GPU capacity separately, with a GPU node pool and node autoscaling. Other parts of the platform may stay provisioned while the workload sits at zero:
- Cluster nodes, including GPU nodes whose pool has not been configured to scale down
- The queue or broker, the HTTP activation layer or gateway, and ingress
- Storage for model weights, logs, and results
- Observability stacks and the control plane
Check each always-on component against its provider’s billing documentation before estimating savings. Scale-to-zero can also add cost through cold-start retries, extra activation components, and GPU time spent while a model loads. A workload that idles at zero while its GPU node pool keeps a minimum node running has removed only the Pod-level part of the bill.
Quick Recap
Troubleshooting
The workload never leaves zero
- Confirm the trigger uses a metric that exists while no Pods run. CPU and memory from the workload cannot wake it.
- Run
kubectl describe scaledobject <name>and read the conditions and events. Authentication or metric-retrieval errors should appear there. - Check that the activation threshold is not higher than the backlog your producers create.
- On the core HPA path, confirm the HPA uses an object or external metric and that the cluster supports the feature.
The first request times out
- Compare the client timeout with your measured cold-start time. If the timeout is shorter, raise it for this route or keep a warm replica.
- Confirm a holding or activation layer sits in front of the workload. Without one, an HTTP request arriving at zero replicas has nowhere to wait.
Replicas start, but the backlog keeps growing
- Determine whether model loading or GPU scheduling is the bottleneck. Replicas that take minutes to become ready will not clear a backlog quickly.
- Raise the maximum replica count only when the node pool maximum, GPU quota, and downstream services can absorb the extra load.
Messages are redelivered or lost during scale-down
- Look for redelivery spikes that line up with scale-down events. If they do, revisit the acknowledgement and visibility settings from the setup steps first.
- Set the Pod’s
terminationGracePeriodSecondslong enough for the longest job to finish.
GPU Pods stay Pending
- Confirm the GPU node pool has node autoscaling enabled with a maximum above zero.
- Check GPU quota in the target region, and compare the Pod’s node selector and tolerations with the node pool’s labels and taints.
Versions and limits to verify
- Kubernetes HPA scale-to-zero. Feature status is taken from the Kubernetes project post dated September 2, 2026. Check the release notes for your cluster version and your provider’s support matrix before relying on it.
- KEDA scalers. Confirm the specific scaler and its fields in the reference for your KEDA version.
- AKS managed KEDA. Microsoft’s current AKS documentation lists limits on changing some KEDA component values. Check it before depending on custom settings.
- Cloud behavior. Node scaling, managed queue behavior, and billing change over time. Re-check provider documentation at design time.
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
