There is no universal cheapest cloud for AI inference. In an April 2026 comparison of on-demand NVIDIA T4 GPUs, Google Cloud was lowest at $0.35 per GPU-hour, versus $0.53 on AWS and $0.56 on Azure. That result applies to comparable GPU infrastructure, not to managed model APIs. For API-based inference, the lowest bill can change with the model, input/output token mix, region, latency target, data-residency requirement and utilization. The reliable way to choose is to compare like with like, then include every cost of keeping the service available.
The answer depends on which kind of inference you are buying
“AI inference” covers two different products:
- Managed inference APIs run the model for you. Amazon Bedrock, Microsoft Foundry and Google Vertex AI generally meter usage by input and output tokens, with additional meters for features such as caching, batch processing, provisioned capacity or fine-tuned hosting.
- Self-hosted inference means renting virtual machines and accelerators, deploying the model and paying for the hardware whether requests arrive or not. GPU-hour, host-VM, storage, networking, monitoring and orchestration charges all matter.
Comparing a token price from one category with a GPU-hour price from the other produces a misleading answer. A fair comparison holds the model, precision, prompt and completion sizes, region, latency objective and availability target constant.
As an Amazon Associate I earn from qualifying purchases.
Comparable on-demand GPU prices
The clearest cross-provider snapshot is a TCOIQ 2026 table for an NVIDIA T4 on-demand GPU. It is a hardware comparison, not a promise that the same model, software stack or throughput will cost the same on each cloud.
| Provider | GPU and pricing mode | Price per GPU-hour | What the figure represents |
|---|---|---|---|
| Google Cloud | NVIDIA T4, on demand | $0.35 | Comparative on-demand table, TCOIQ 2026 |
| AWS | NVIDIA T4, on demand | $0.53 | Comparative on-demand table, TCOIQ 2026 |
| Azure | NVIDIA T4, on demand | $0.56 | Comparative on-demand table, TCOIQ 2026 |
| Google Cloud | NVIDIA L4, on demand | $0.72 | Comparative on-demand table, TCOIQ 2026 |
On that specific T4 comparison, Google Cloud is approximately 34% below AWS and 38% below Azure. Those percentages describe only the listed accelerator-hour prices. The host machine, attached storage, egress, operating system, support plan and idle time can narrow or reverse the difference.
#1 Best Overall
- AI Performance: 767 AI TOPS
- OC mode: 2632 MHz (OC mode)/ 2602 MHz (Default mode)
- Powered by the NVIDIA Blackwell architecture and DLSS 4
- Axial-tech fan design features a smaller fan hub that facilitates longer blades and a barrier ring that increases downward air pressure
- A 2.5-slot design maximizes compatibility and cooling efficiency for superior performance in small chassis
Google Cloud’s published GPU options
Google Cloud’s official GPU pricing page lists accelerator charges separately from the machine type. As Google explains, “Each GPU adds to the cost of your instance in addition to the cost of the machine type.” The same page lists these NVIDIA T4 rates:
| Commitment type | NVIDIA T4 price per GPU-hour | Important qualification |
|---|---|---|
| On demand | $0.35 | Current page accessed October 18, 2026 |
| One-year commitment | $0.22 | Commitment rate; the obligation and eligible configuration must be modeled |
| Three-year commitment | $0.16 | Long-term commitment rate; not an hourly option without the commitment |
| Spot | Dynamic | Google states that most machine types and GPUs typically receive a 60–91% discount from on-demand pricing |
A commitment lowers the accelerator component but does not remove the machine-type charge or guarantee that demand will fill the GPU. Spot capacity can be interrupted and its price changes, so it is best suited to work that can checkpoint, retry or wait.
How each cloud bills managed inference
AWS: model-specific token meters
Amazon Bedrock publishes prices by model, with separate rates for input and output tokens per 1 million tokens. It also lists provisioned model-unit charges for workloads that need reserved capacity. Consequently, the cheapest AWS deployment depends first on the model and then on the ratio of input to generated output.
For an API estimate, count prompt tokens and completion tokens independently. A workload that sends long documents but produces short answers can have a different winner from a conversational workload that generates lengthy responses, even when request counts are identical. Add any caching, batch or provisioned-capacity meter shown for the selected model.
Azure: pay-per-token, batch and provisioned deployments
Microsoft Foundry documents token billing for model inputs and outputs and hourly hosting for some fine-tuned deployments. Its deployment choices have different operating and cost characteristics:
| Azure deployment type | Billing or behavior | Best fit |
|---|---|---|
| Global Standard | Pay per token | General online inference where demand varies |
| Global Batch | Pay per token; Microsoft documents a 50% discount for eligible batch workloads | Large asynchronous jobs that can tolerate delay |
| Provisioned throughput | Capacity purchased for predictable demand | Steady traffic with a measurable throughput requirement |
The 50% batch reduction is documented for eligible Azure Foundry models and Global Batch deployments; eligibility and model availability must be checked for the exact workload. Microsoft describes Global Batch as processing data in any Azure region, billed per token and intended for large asynchronous jobs.
Google Cloud: Vertex APIs versus Compute Engine
Google’s managed model services expose their own model catalog and token meters, while Compute Engine charges for the selected GPU per accelerator-hour in addition to the VM. Do not use the $0.35 T4 number as a proxy for a Vertex API request: a managed API price includes a different service, model and billing unit.
The Tool Desk
Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →For self-hosting on Google Cloud, calculate the GPU and machine-type charges together. Google publishes on-demand, commitment and Spot prices separately, and Spot pricing is dynamic. A lower accelerator rate is useful only if the selected VM shape, region and software stack can deliver the required throughput.
A reproducible comparison for managed APIs
Use the following process before choosing Bedrock, Microsoft Foundry or Vertex AI. Save the assumptions with the result because catalogs, prices and regions change.
Rank #2
- Chipset: GeForce RTX 3050
- Boost Clock / Memory: 1492 MHz / 14 Gbps
- Video Memory: 6GB GDDR6
- Memory Interface: 96-bit
- Output: DisplayPort x 1 (v1.4a) / HDMI 2.1a x 2
- Fix the model and precision. Compare the same model family, version and numerical precision. If a provider does not offer that model, mark the result as a different-model comparison rather than declaring a price winner.
- Describe the request. Record average input tokens, average output tokens, maximum context, requests per second, daily request volume and the percentage of traffic that is interactive versus asynchronous.
- Set the region and residency boundary. Use the same geographic requirement for every provider. A cheaper global endpoint may be unusable when data must remain in a particular country or legal region.
- Set the service objective. State the latency target, uptime requirement, concurrency and failover policy. A provisioned or multi-region design can cost more than a single best-effort endpoint.
- Apply the provider’s current meters. Multiply input tokens by the input rate and output tokens by the output rate for the selected model. Add documented charges for caching, batch, fine-tuned hosting, provisioned throughput, safety features or other enabled capabilities.
- Model peaks as well as averages. A daily average can hide a short traffic burst that requires extra provisioned capacity or a second region. Calculate the cost at normal, busy and failure-recovery volumes.
- Separate recurring and one-time costs. Include migration, prompt or model conversion, load testing, observability setup and reserved-capacity commitments separately from the steady-state request bill.
The result should be a cost per request, cost per 1 million input tokens, cost per 1 million output tokens and total monthly cost under the stated traffic pattern. Do not combine input and output into one token count when the provider prices them differently.
A reproducible comparison for self-hosted GPUs
For a model you deploy yourself, use a total-cost equation rather than the GPU line item:
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Monthly cost = accelerator hours + host VM hours + storage + network transfer + load balancing + monitoring and logs + orchestration/control plane + support + idle capacity.
Measure throughput on the exact model, precision, batching and context length you intend to run. Then divide the complete monthly cost by the number of successfully served requests or tokens. Include warm-up time, rolling upgrades, replicas for availability and capacity held for spikes.
Costs that are commonly omitted
- Machine type: The CPU, RAM, local SSD and attached disks can cost as much as, or more than, a lightly used accelerator.
- Storage: Model weights, tokenizer files, container layers, checkpoints and logs need persistent and sometimes regional storage.
- Networking: Egress to users, cross-zone traffic and inter-region replication can be material for large responses or multi-region deployments.
- Utilization: Paying for a GPU that is idle between bursts raises the effective cost per token. Track average, p95 and peak utilization.
- Operations: Autoscaling, health checks, rollout capacity, alerting, patching and incident response require both services and engineering time.
- Reliability: A single GPU may be cheapest but cannot provide the same redundancy as multiple replicas or zones.
What the T4 numbers mean in a full-month scenario
To make the hourly comparison concrete, 730 hours is a common approximation for one month of continuous operation. Multiplying only the published accelerator rates gives:
| Provider and T4 rate | 730 accelerator-hours | What is excluded |
|---|---|---|
| Google Cloud at $0.35/hour | $255.50 | VM, storage, network, monitoring, support and idle host costs |
| AWS at $0.53/hour | $386.90 | VM, storage, network, monitoring, support and idle host costs |
| Azure at $0.56/hour | $408.80 | VM, storage, network, monitoring, support and idle host costs |
These are arithmetic illustrations of the TCOIQ 2026 on-demand figures, not complete deployment bills. A different VM family, a second replica or a regional data-transfer path can change the ranking. Google’s published one-year and three-year T4 accelerator rates of $0.22 and $0.16 per hour would produce lower accelerator-only totals, but only for customers accepting those commitment terms.
Quick wins for a faster PC:
Repair Windows errors before they cause bigger problemsFix Now →Scan for outdated or missing drivers - takes under a minuteDriver Scan →Which pricing model tends to win by workload?
Low-volume or bursty online traffic
Pay-per-token APIs usually avoid paying for a GPU while no requests are arriving. They also remove much of the deployment work. Compare the API bill with the cost of keeping a self-hosted replica warm for your latency target; a low average request count can make the always-on GPU uneconomic even when its hourly rate looks attractive.
Spot GPUs are another option when interruptions are acceptable. Design the service to checkpoint and retry, and keep an on-demand fallback if a user-facing request cannot wait.
Steady, high-throughput traffic
When utilization is consistently high, reserved, committed or provisioned capacity can reduce the unit cost. The commitment must be amortized over the hours you will actually use, including expected growth and maintenance windows. Provisioned throughput is particularly relevant when a managed API must meet a predictable rate without relying on best-effort shared capacity.
Rank #3
- Powered by the NVIDIA Blackwell architecture and DLSS 4. System Requirements: Minimum 850W PSU with 16-pin 12V-2x6 (12VHPWR) connector required. Verify before purchasing.
- Military-grade components deliver rock-solid power and longer lifespan for ultimate durability. Compatibility: 348mm (13.7") length, 3.6 slots, 4.3 lbs. Confirm case clearance and slot spacing. GPU bracket included.
- Protective PCB coating helps protect against short circuits caused by moisture, dust, or debris
- 3.6-slot design with massive fin array optimized for airflow from three Axial-tech fans
- Phase-change GPU thermal pad helps ensure optimal thermal performance and longevity, outlasting traditional thermal paste for graphics cards under heavy loads
Large asynchronous or batch jobs
Batch processing allows queueing, retries and interruption. Azure Global Batch documents a 50% discount for eligible models and workloads. Check the current batch programs on AWS and Google for the same job before deciding; equivalent discounts, model coverage and regional availability can differ.
Strict locality, compliance or custom-model requirements
A regional endpoint or self-hosted model can be the right choice even when its headline hourly price is higher. Data residency, private networking, custom kernels, fine-tuning control and an approved model catalog may outweigh a small per-token or per-GPU saving. Record the compliance requirement as a hard constraint before optimizing price.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Spot, commitment and on-demand capacity
Do not put these options in one undifferentiated “cheapest” column:
| Capacity option | Cost behavior | Operational trade-off | Suitable workloads |
|---|---|---|---|
| On demand | Highest flexibility; pay the listed rate while running | Capacity and price are easier to change, but unit cost is higher | Uncertain demand, production fallbacks, experiments |
| Commitment or reservation | Lower rate in exchange for a one- or three-year obligation; Google’s T4 examples are $0.22 and $0.16 per hour | You can pay for unused capacity if forecasts are wrong | Stable, high-utilization services |
| Spot | Dynamic discount; Google cites a typical 60–91% discount for most machine types and GPUs | Interruption and price variability require checkpointing and retries | Batch, flexible queues and fault-tolerant inference |
An August 2026 nOps report put observed Spot-to-on-demand GPU prices for H100, A100 and V100 capacity across the three hyperscalers in a broad $1.95–$12.29 per GPU-hour range. That range spans different GPU models and providers, so it should not be substituted for the T4 figures or treated as a guaranteed quote.
Renting a GPU versus buying one
A physical NVIDIA T4 can look inexpensive after comparing cloud hourly rates, but the purchase decision requires a different calculation. Include the card’s purchase price, host chassis, power, cooling, networking, storage, replacement risk, physical space, software maintenance and the value of engineering time. Divide that total by realistic productive hours rather than by every hour on the calendar.
Recommended Free Tools
Ownership can make sense when utilization is high and predictable, the workload can remain in one location and you already operate suitable hardware. Renting is usually easier for experiments, variable demand, geographic distribution and workloads that need newer accelerators or rapid scale-out. Neither choice is automatically cheaper from the T4’s name or its cloud hourly rate alone.
Common comparison mistakes
- Mixing billing units: A token price and a GPU-hour price answer different questions. Convert both to cost per useful token or request under the same workload.
- Using different models: A cheaper model may be less capable or may require more tokens, retries or post-processing. Compare quality and output length as well as price.
- Ignoring output tokens: Long completions can dominate an API bill even when prompts are short.
- Comparing regions casually: Prices, quotas, model availability and data-transfer paths vary by region. Keep the region fixed or label the comparison as geographic trade-off.
- Assuming a discount is permanent: Batch, Spot and promotional rates have eligibility rules, expiration or dynamic behavior. Record the date and SKU.
- Forgetting idle replicas: A highly available service may run two or more GPUs while average traffic uses one. Charge the reserved capacity, not just the busy replica.
- Counting only inference: Embeddings, reranking, moderation, vector search, logging and data movement can be part of the production path.
- Relying on a stale calculator: Recheck provider pricing pages, model catalogs, quotas, Spot behavior and commitment terms immediately before deployment.
A practical decision framework
- Start with constraints. Write down model, precision, region, residency, latency, availability and maximum acceptable interruption.
- Estimate managed APIs first. They provide a useful baseline for bursty traffic and expose the real token mix.
- Benchmark self-hosting. Measure tokens per second, concurrency, memory use and cold-start behavior on the candidate GPU and VM.
- Price three capacity cases. Produce separate on-demand, commitment and Spot totals. Do not blend them into one average without stating the assumed usage.
- Calculate break-even utilization. Find the utilization at which the complete self-hosted monthly cost equals the managed API cost. Include replicas and operations in both sides where applicable.
- Stress-test the forecast. Recalculate for a traffic spike, longer outputs, a second region and a 20–30% lower utilization assumption. The option that remains affordable under those cases is usually safer than the one that wins a single-point estimate.
- Record the quote. Save provider, region, SKU, model version, pricing page date, discount assumption and currency so the comparison can be rerun when prices or catalogs change.
Bottom line for 2026
For the specific on-demand T4 comparison available in April 2026, Google Cloud is the low-cost infrastructure choice at $0.35 per GPU-hour, ahead of AWS at $0.53 and Azure at $0.56. Google’s published T4 commitment rates are lower still, at $0.22 for one year and $0.16 for three years, while Spot capacity can be substantially discounted but is interruptible.
That does not establish Google as the cheapest place to run every AI model. Managed APIs use provider-specific catalogs and token meters, and the exact same-model, per-million-token winner across all three clouds is not established by the available figures. Choose the provider only after fixing the model, token mix, region, latency and utilization, then adding the costs that the headline rate leaves out.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
Free tools Windows power users keep installed
One-click scans. No signup required.
