Quick wins for a faster PC:
Repair Windows errors before they cause bigger problemsFix Now →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →GPU spot instances are cloud virtual machines that use spare accelerator capacity at a lower priority and usually a lower price than regular instances. The provider can reclaim that capacity, sometimes with only seconds or minutes of warning. If the VM is stopped, deallocated, hibernated, or deleted, any GPU job still running on it can stop before completion. Reliable use therefore depends on durable checkpoints, interruption monitoring, and an automated way to resume on replacement capacity.
What a GPU spot instance is
A GPU spot instance is a GPU-capable cloud VM offered from capacity that the provider does not currently need for higher-priority commitments. “Spot” describes the allocation priority, not a special type of graphics processor. The same GPU family may be available as a regular on-demand VM and as a spot VM, but the spot allocation can be reclaimed.
As an Amazon Associate I earn from qualifying purchases.
AWS describes Spot as spare EC2 capacity that is returned when AWS needs it. Azure says Spot VMs have lower priority than regular VMs and provide no availability guarantee after deployment. In practice, the inventory you can obtain depends on the exact GPU SKU, region, quota, and capacity pool at the time you request it. See the AWS interruption guidance and Azure Spot architecture guidance.
The lower price is compensation for accepting that uncertainty. AWS and Azure each advertise savings of up to 90 percent compared with their regular pricing, but those are provider-wide maximums, not a quote for a particular GPU, region, date, or workload. A specific pool can have a much smaller discount, become unavailable, or be interrupted more often than another pool.
#1 Best Overall
- AI Performance: 767 AI TOPS
- OC mode: 2632 MHz (OC mode)/ 2602 MHz (Default mode)
- Powered by the NVIDIA Blackwell architecture and DLSS 4
- Axial-tech fan design features a smaller fan hub that facilitates longer blades and a barrier ring that increases downward air pressure
- A 2.5-slot design maximizes compatibility and cooling efficiency for superior performance in small chassis
Why providers interrupt them
An interruption is a capacity-management event, not necessarily a defective GPU. Providers may reclaim a VM because they need the capacity for regular instances. Depending on the service and configuration, a price condition or a request constraint can also trigger the event. The platform then applies the interruption behavior you selected.
The result can be a graceful state change or permanent removal:
- Terminate or delete: the VM is removed, and ephemeral state is lost. Attached disks may also be removed when the platform is configured to delete them with the VM.
- Stop or deallocate: the VM is powered off but may be redeployed later. Allocation of the same capacity is not guaranteed, and persistent disks can continue to incur charges.
- Hibernate: the provider saves VM memory state according to the service’s rules and stops the instance. This can reduce restart work but does not make the capacity dependable.
A running training process, rendering task, simulation, or inference service is not automatically moved to another GPU. Unless the application has already written recoverable state elsewhere, the work in memory at the moment of interruption is gone.
Do these 3 things before closing this tab:
1Scan for outdated or missing drivers - takes under a minute2Clear out junk files and repair common Windows errors3Fix the driver behind crashes, sound loss and screen glitchesHow AWS, Azure, and Google Cloud signal an interruption
Warning behavior differs by provider. Do not design a checkpoint strategy around another provider’s timing.
| Provider | Signal and timing in the cited documentation | Possible outcome | Operational implication |
|---|---|---|---|
| AWS EC2 Spot | A two-minute interruption notice is documented. A rebalance recommendation can provide an earlier indication that interruption risk is elevated. | Depending on request configuration, EC2 can terminate, stop, or hibernate the instance. Reclamation can be related to capacity, price, or constraints. | A persistent request may be resubmitted, but a new GPU is not guaranteed immediately. AWS interruption details and how Spot works. |
| Azure Spot VMs | The Preempt signal is described as providing at least 30 seconds of advance notice in Azure’s architecture guidance. Azure also notes that a VM can be evicted immediately after creation; Scheduled Events delivery is best effort in the VM documentation. | Deallocate preserves the VM and disks for possible later deployment, but capacity is not assured and disk storage remains chargeable. Delete removes the VM and its underlying disks. Evictions can be capacity- or price-related, depending on configuration. | Use health checks and orchestration even immediately after provisioning. Azure architecture guidance and Azure Spot VM documentation. |
| Google Cloud Spot VMs | Metadata receives a preemption signal. Dedicated notice defaults to zero; Google documents an optional 120-second setting. After the signal, a best-effort shutdown phase can last up to 30 seconds. | If the VM has not stopped, it can receive a forced-off signal. The 120-second option’s current feature status should be checked before making it a requirement. | Assume no dedicated warning unless you have confirmed the setting for the exact VM type and region. Google Cloud Spot VM documentation. |
These timings are service-documentation behavior, not guarantees that a particular GPU job has enough time to save a checkpoint. A busy filesystem, large model state, or network congestion can consume the entire window.
What happens to a GPU job during an interruption
- The control plane marks the VM for interruption. The provider emits its documented signal through an instance metadata endpoint, scheduled-events channel, or equivalent mechanism.
- Your process keeps running briefly, if time is available. A monitor can stop accepting new work, flush logs, and request a final checkpoint. The signal is an opportunity for cleanup, not a guarantee of completion.
- The VM changes state. It may be terminated, deleted, stopped, deallocated, hibernated, or forced off according to the provider and your settings.
- Unsaved state disappears. GPU memory, CPU memory, temporary files, and any checkpoint that had not finished uploading are lost.
- Recovery requires another allocation. A scheduler, persistent request, autoscaling group, or user must obtain replacement capacity and start the workload from its last durable checkpoint.
For distributed training, losing one worker can invalidate the rendezvous or collective operation for the entire job. Plan for coordinated checkpointing and a restart of the group, rather than assuming that the surviving workers can continue independently.
How to make a GPU workload interruption-tolerant
The following design turns an unreliable VM into a replaceable worker. Exact checkpoint intervals depend on model size, storage bandwidth, restart time, and the amount of lost work you can afford; no provider specifies one universal interval.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Rank #2
- Chipset: GeForce RTX 3050
- Boost Clock / Memory: 1492 MHz / 14 Gbps
- Video Memory: 6GB GDDR6
- Memory Interface: 96-bit
- Output: DisplayPort x 1 (v1.4a) / HDMI 2.1a x 2
1. Put checkpoints outside the VM
Write model weights, optimizer state, scheduler state, data position, random-number-generator state, and the software configuration needed for reproducibility to durable storage that is not tied to the spot VM. Upload atomically: write a new object or directory, verify it, then update a small “latest successful checkpoint” pointer. Keep more than one known-good checkpoint so a failed upload cannot become the only recovery point.
2. Checkpoint before the warning window becomes critical
Save periodically during normal operation. Treat the provider signal as a trigger for a final best-effort save, not as the only time you persist. A checkpoint that takes longer than the warning period must already be covered by the regular cadence.
3. Monitor the provider signal
Run a lightweight agent or sidecar that polls the provider’s interruption channel, records the event, marks the worker unhealthy, and asks the application to drain. Add alerts for missed heartbeats and failed checkpoint uploads. Monitoring cannot prevent eviction, but it gives your scheduler a chance to stop assigning new work and to release dependent resources cleanly.
4. Make retries safe
Design each unit of work to be idempotent. Use deterministic job IDs, transactional output publication, and a manifest that records completed shards or steps. On a retry, the replacement worker should be able to detect already-published results instead of duplicating them or corrupting a shared dataset.
Free tools Windows power users keep installed
One-click scans. No signup required.
5. Automate replacement capacity
Keep the machine image, container, drivers, startup scripts, permissions, and data-mount configuration reproducible. A queue or orchestrator should request another compatible GPU, attach durable storage, restore the latest checkpoint, and rejoin the job. Do not assume that a persistent spot request or a new allocation will produce an immediate replacement GPU.
6. Test the recovery path deliberately
Stop a worker during training or revoke its lease in a test environment. Measure how much work is lost, how long checkpoint upload and restore take, whether outputs are duplicated, and whether a multi-node job can re-form its process group. A recovery path that has never been exercised is likely to fail under a real capacity event.
Provider-specific storage and shutdown details
AWS EC2 Spot
AWS states: “When Amazon EC2 interrupts a Spot Instance, it terminates the instance by default, unless you specify a different interruption behavior, such as stop or hibernate.” The selected behavior determines whether the VM and local state remain available after the event. Use the two-minute notice and any rebalance recommendation to drain work, but keep checkpoints on storage that survives instance replacement. AWS’s Spot best-practices guidance recommends architectures that tolerate interruption.
Rank #3
- Powered by the NVIDIA Blackwell architecture and DLSS 4. System Requirements: Minimum 850W PSU with 16-pin 12V-2x6 (12VHPWR) connector required. Verify before purchasing.
- Military-grade components deliver rock-solid power and longer lifespan for ultimate durability. Compatibility: 348mm (13.7") length, 3.6 slots, 4.3 lbs. Confirm case clearance and slot spacing. GPU bracket included.
- Protective PCB coating helps protect against short circuits caused by moisture, dust, or debris
- 3.6-slot design with massive fin array optimized for airflow from three Axial-tech fans
- Phase-change GPU thermal pad helps ensure optimal thermal performance and longevity, outlasting traditional thermal paste for graphics cards under heavy loads
Azure Spot VMs
Azure’s architecture guidance says: “Spot VMs have no availability guarantees or service-level agreements (SLAs).” With deallocation, the VM and disks can remain for a later attempt, but the platform does not promise that the VM can be allocated again and disk storage remains billable. With delete, the VM and underlying disks are removed. Configure deletion behavior deliberately, and keep the authoritative checkpoint in external durable storage.
Google Cloud Spot VMs
Google Cloud delivers the preemption signal through VM metadata. Dedicated notice is zero by default; documentation describes a 120-second option and a best-effort shutdown period of up to 30 seconds. Because the notice option’s availability can depend on the current feature state, verify it in the current documentation and your chosen machine configuration. Code the workload to handle an abrupt forced-off event even when a longer notice is enabled.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.When spot GPUs are a good fit
- Batch training with frequent checkpoints: Losing part of an epoch is acceptable when the next VM can resume.
- Hyperparameter sweeps: Each trial can be independently retried, and a failed trial does not invalidate the others.
- Distributed rendering or simulation: Work can be divided into idempotent frames, tiles, or scenarios.
- Development and experimentation: The price advantage can outweigh the inconvenience when a person can restart a job.
Spot is a poor fit for a single, long-running job that cannot checkpoint, a latency-sensitive production endpoint without failover, or a deadline where replacement capacity is not assured. A regular VM, reserved capacity, or a mixed fleet can be more appropriate when continuity matters more than the maximum discount.
How to compare a spot option responsibly
Compare the exact configuration, not a provider’s headline percentage. Use this checklist before committing a production workload:
- GPU model, GPU count, attached CPU and memory, region, zone, and quota.
- Current spot price versus the regular price for that same SKU and location.
- Recent interruption behavior for the specific capacity pool, where the provider exposes it.
- Signal delivery method, configured notice period, and whether the signal is best effort.
- Whether interruption stops, hibernates, deallocates, terminates, or deletes the VM and attached disks.
- Time required to obtain replacement capacity and to restore the checkpoint.
- Checkpoint storage, transfer, and restart costs.
- Operational effort for schedulers, monitoring, retries, and multi-node coordination.
No general GPU price, availability rate, or interruption rate applies across all regions and SKUs. Those values change, so obtain current figures for the exact pool you intend to use rather than treating “up to 90 percent” as an expected saving.
A practical readiness checklist
- Can a new VM resume from a checkpoint without manual edits?
- Are checkpoints stored outside the spot VM and verified before publication?
- Does the application save enough state to avoid restarting from the beginning?
- Does the interruption handler stop new work and finish only safe cleanup?
- Are retries idempotent and outputs protected from duplication?
- Will an orchestrator request compatible replacement GPUs automatically?
- For multi-node jobs, can all workers restart from one consistent checkpoint?
- Have you tested both a graceful warning and an immediate loss?
Spot capacity can be an effective way to reduce GPU compute cost, but it is not cheaper guaranteed capacity. Treat every VM as disposable, persist the state that matters, and make recovery routine rather than exceptional.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
