October DealsAmazon USOctober deal check: compare before you payAmazon US: current deals, useful picks and tech finds.Check DealsSlow PC?RecommendedPC slow today? Run a repair scan before it gets worseResolve common Windows issues and optimize system performance.Scan NowOctober DealsAmazon USDeal season is back - check today's better picksAmazon US: current deals, useful picks and tech finds.See Picks×
Skip to content
World desk5 min

Why Teams Choose GPUs Even When TPUs Can Cost Less per Token

TPU cost advantages depend on the workload and the complete deployment—not just the hourly chip price. Here’s why teams may still choose GPUs.
Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

TPUs can cost less per token when a workload suits their design and the full TPU setup delivers enough useful work. That is not a universal price advantage: teams must account for VM configuration, throughput, utilization, latency, software adaptation and access to capacity—not just an accelerator’s hourly rate. Those factors help explain why a lower reported cost does not automatically make TPUs the default, though the available evidence does not quantify industry-wide adoption or its causes.

Why a lower hourly rate does not settle the cost question

A chip-hour is not a token price. Google lists TPU rates per chip-hour, but Cloud TPU VMs are billed in VM-hours, and a VM can contain multiple chips. Google gives the example of a TPU v4 host with four chips costing $12.88 per hour on demand; that example is not a universal quote for every TPU generation or region. Check the Cloud TPU pricing details and compare the whole configuration you would actually run.

As an Amazon Associate I earn from qualifying purchases.

The useful comparison is the cost of a complete configuration divided by the useful tokens it produces under your quality and latency requirements. A lower resource price can lose its advantage if the workload makes poor use of the hardware, needs extra machines, or requires extensive engineering to run well. GPU prices also vary by region, and the machine configuration contributes to total cost, as Google notes in its GPU pricing documentation.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
  • Configuration: Compare full VM or machine costs, not isolated accelerator rates.
  • Delivered work: Measure useful tokens at the required output quality, throughput and latency.
  • Utilization: Include idle time and the workload’s actual batch and concurrency profile.
  • Engineering: Account for the work to port, operate and maintain the model on the chosen stack.
  • Capacity: Include whether the required hardware can be obtained in the needed region and timeframe.

TPUs suit some workloads better than others

TPUs are specialized accelerators built around matrix operations. Google recommends them for matrix-dominated models, large effective batch sizes and training workloads that do not depend on custom PyTorch or JAX operations in the main loop. Its Cloud TPU introduction identifies frequent branching, high-precision arithmetic and custom operations in that loop as poor fits.

#1 Best Overall
ASUS Dual Radeon RX 9060 XT 16GB GDDR6 Gaming Graphics Card | PCIe 5.0, HDMI 2.1b, DisplayPort 2.1a, 2.5-slot design, Axial-tech Fans, 0dB Technology
  • Axial-tech fans now feature a smaller fan hub that facilitates longer blades and a barrier ring that increases downward air pressure
  • 2.5-slot design allows for greater build compatibility while maintaining cooling performance
  • 0dB technology lets you enjoy light gaming in relative silence
  • Dual BIOS switch lets you toggle between Quiet and Performance BIOS profiles
  • Dual ball fan bearings last up to twice as long as sleeve bearing designs

That guidance matters economically: the relevant question is not whether a TPU can run a model at all, but whether the model’s operators, precision requirements, shapes and batch profile can keep the accelerator productively occupied. A workload that maps poorly may deliver fewer useful tokens for each paid hour.

What published comparisons do—and do not—show

A May 25, 2026 technical preprint by Jatin Kishnani, Mayank Goel, Amit Singh, Pulkit Agrawal and Sairanjan Mishra reports a substantial TPU advantage for its specific Gemma 4 31B setup. The figures below are the authors’ results, not a universal ranking of TPU and GPU hardware.

Rank #2
GIGABYTE GeForce RTX 5070 Ti Gaming OC 16G Graphics Card, 16GB 256-bit GDDR7, PCIe 5.0, WINDFORCE Cooling System, GV-N507TGAMING OC-16GD Video Card
  • Powered by the NVIDIA Blackwell architecture and DLSS 4
  • Powered by GeForce RTX 5070 Ti
  • Integrated with 16GB GDDR7 256bit memory interface
  • PCIe 5.0
  • WINDFORCE cooling system
Workload and comparison Reported result
LoRA training on TPU v5p-8 versus a two-H100 GPU baseline 1.61× faster training and 2.12× lower training cost
Inference on TPU v6e-8 in the authors’ serving comparison Throughput within 3% of the comparison setup; time to first token of 235 ms versus 475 ms
Representative combined training and service setup using the study’s training and TPU v6e-8 inference results 1.82× lower cost

These results are useful evidence that TPU economics can be compelling for a defined workload. They do not establish that every model, serving target or deployment will see the same advantage. See the authors’ Gemma 4 31B technical comparison for the setup and adaptation details.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Porting and operating the software stack takes work

In that Gemma study, the authors describe moving from a GPU-oriented PyTorch, Hugging Face TRL and FSDP recipe to JAX with Tunix and Qwix. The work included mesh configuration, sharding, checkpoint conversion, data-pipeline changes and other adaptations. Those are concrete costs in this example, but they do not prove that every TPU project requires the same migration or effort.

Rank #3
GIGABYTE GeForce RTX 5060 WINDFORCE OC 8G Graphics Card, Cooling System, 8GB 128-bit GDDR7, PCIe 5.0, Manufactured by NVIDIA, DisplayPort & HDMI - Video Output Interface, GV-N5060WF2OC-8GD Video Card
  • Powered by the NVIDIA Blackwell architecture and DLSS 4
  • Powered by GeForce RTX 5060
  • Integrated with 8GB GDDR7 128bit memory interface
  • PCIe 5.0
  • WINDFORCE cooling system

Google has announced vLLM support for TPU alongside its JetStream serving stack, which can reduce friction for some deployments. It does not make every model or deployment path a drop-in fit. Google’s own serving results also need to be read in context: its 2025 post reported 7× better prefill and nearly 3× better token generation for a disaggregated Llama 2 70B setup using multiple Trillium hosts, compared with interleaved prefill and decode on one server. The stated setup used MaxText, the OpenOrca dataset, maximum input and output lengths of 1,024 tokens, and measurements as of April 2025. This was a vendor-reported serving comparison—not a TPU-versus-GPU benchmark. Details are in Google’s TPU and GPU inference update.

Cloud capacity affects whether the economics work in practice

Cloud TPU is provisioned through Google Cloud services, so access is part of the decision. Google says on-demand requests are not guaranteed to have enough resources. Reservations can improve capacity assurance and may reduce price, but require advance commitment. Flex-start is for up to seven days, while Spot instances can be preempted. Review Google’s Cloud TPU resource-planning guidance before assuming a particular topology will be available when you need it.

Rank #4
Sale
GIGABYTE Radeon RX 9070 XT Gaming OC 16G Graphics Card, PCIe 5.0, 16GB GDDR6, GV-R9070XTGAMING OC-16GD Video Card
  • Powered by Radeon RX 9070 XT
  • WINDFORCE Cooling System
  • Hawk Fan
  • Server-grade Thermal Conductive Gel
  • RGB Lighting

A hardware option that looks economical on paper may not fit a team’s timeline, region or capacity needs. This is a practical constraint, not evidence that TPUs are inherently less capable or that capacity explains any particular share of GPU adoption.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Support on Ko-Fi

How to make a fair TPU-versus-GPU comparison

Hold the workload constant before comparing costs. A difference in model, precision, serving software, sequence length or evaluation method can change both performance and output quality.

Best Value
Sale
ASUS Prime Radeon RX 9070 XT 16GB GDDR6 OC Edition Gaming Graphics Card | PCIe 5.0, HDMI/DP 2.1, 2.5-slot, Axial-tech Fans, Dual-ball Fan Bearings, dual BIOS, ASUS GPU Guard
  • Axial-tech fans now feature a smaller fan hub that facilitates longer blades and a barrier ring that increases downward air pressure
  • Phase-change GPU thermal pad helps ensure optimal heat transfer, lowering GPU temperatures for enhanced performance and reliability
  • 2.5-slot design allows for greater build compatibility while maintaining cooling performance
  • Dual-ball fan bearings last up to twice as long as standard conventional sleeve bearings designs
  • 0dB technology lets you enjoy light gaming in relative silence
  1. Fix the task: Use the same model and checkpoint, precision, input and output lengths, and quality evaluation.
  2. Match the service target: Keep batch size, concurrency, throughput and latency requirements consistent.
  3. Price complete configurations: Compare the actual VM or machine setup in the relevant region, not a chip-hour against an entire system.
  4. Measure useful output: Record tokens produced at the required quality and latency, along with utilization, rather than relying on peak throughput.
  5. Include the path to production: Account for migration, checkpoint and data-pipeline work, ongoing maintenance, and capacity arrangements.

Google’s newer TPU-generation announcement is another reason to date any comparison carefully: its Next ’26 post described eighth-generation TPU systems as coming to Cloud customers soon. That announcement is not itself a current availability check or an independent performance comparison. See Google’s AI infrastructure announcement for its stated plans and claims.

Why “not popular” needs a careful qualification

The available sources do not provide a reliable industry-wide TPU-versus-GPU adoption statistic or isolate the reasons teams choose one platform over another. They do show several practical factors that can limit TPU adoption: workload fit, the engineering involved in adapting a GPU-oriented workflow, and cloud capacity planning. Those factors explain why a workload-specific cost win may not translate into a default choice; they do not quantify how much each factor matters across the industry.

Quick Recap

Bestseller No. 1
ASUS Dual Radeon RX 9060 XT 16GB GDDR6 Gaming Graphics Card | PCIe 5.0, HDMI 2.1b, DisplayPort 2.1a, 2.5-slot design, Axial-tech Fans, 0dB Technology
ASUS Dual Radeon RX 9060 XT 16GB GDDR6 Gaming Graphics Card | PCIe 5.0, HDMI 2.1b, DisplayPort 2.1a, 2.5-slot design, Axial-tech Fans, 0dB Technology
0dB technology lets you enjoy light gaming in relative silence; Dual BIOS switch lets you toggle between Quiet and Performance BIOS profiles
$529.99
Bestseller No. 2
GIGABYTE GeForce RTX 5070 Ti Gaming OC 16G Graphics Card, 16GB 256-bit GDDR7, PCIe 5.0, WINDFORCE Cooling System, GV-N507TGAMING OC-16GD Video Card
GIGABYTE GeForce RTX 5070 Ti Gaming OC 16G Graphics Card, 16GB 256-bit GDDR7, PCIe 5.0, WINDFORCE Cooling System, GV-N507TGAMING OC-16GD Video Card
Powered by the NVIDIA Blackwell architecture and DLSS 4; Powered by GeForce RTX 5070 Ti; Integrated with 16GB GDDR7 256bit memory interface
$1,249.99
Bestseller No. 3
GIGABYTE GeForce RTX 5060 WINDFORCE OC 8G Graphics Card, Cooling System, 8GB 128-bit GDDR7, PCIe 5.0, Manufactured by NVIDIA, DisplayPort & HDMI - Video Output Interface, GV-N5060WF2OC-8GD Video Card
GIGABYTE GeForce RTX 5060 WINDFORCE OC 8G Graphics Card, Cooling System, 8GB 128-bit GDDR7, PCIe 5.0, Manufactured by NVIDIA, DisplayPort & HDMI - Video Output Interface, GV-N5060WF2OC-8GD Video Card
Powered by the NVIDIA Blackwell architecture and DLSS 4; Powered by GeForce RTX 5060; Integrated with 8GB GDDR7 128bit memory interface
$497.06
SaleBestseller No. 4
GIGABYTE Radeon RX 9070 XT Gaming OC 16G Graphics Card, PCIe 5.0, 16GB GDDR6, GV-R9070XTGAMING OC-16GD Video Card
GIGABYTE Radeon RX 9070 XT Gaming OC 16G Graphics Card, PCIe 5.0, 16GB GDDR6, GV-R9070XTGAMING OC-16GD Video Card
Powered by Radeon RX 9070 XT; WINDFORCE Cooling System; Hawk Fan; Server-grade Thermal Conductive Gel
$814.99
SaleBestseller No. 5
ASUS Prime Radeon RX 9070 XT 16GB GDDR6 OC Edition Gaming Graphics Card | PCIe 5.0, HDMI/DP 2.1, 2.5-slot, Axial-tech Fans, Dual-ball Fan Bearings, dual BIOS, ASUS GPU Guard
ASUS Prime Radeon RX 9070 XT 16GB GDDR6 OC Edition Gaming Graphics Card | PCIe 5.0, HDMI/DP 2.1, 2.5-slot, Axial-tech Fans, Dual-ball Fan Bearings, dual BIOS, ASUS GPU Guard
0dB technology lets you enjoy light gaming in relative silence; Dual BIOS switch lets you toggle between Quiet and Performance BIOS profiles
$831.99

Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Leave a Reply

Your email address will not be published. Required fields are marked *

Free tools Windows power users keep installed

One-click scans. No signup required.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

More from the Wire

  1. World desk4 min
    How to Spot an AI Voice Scam Before Sending MoneyDon’t rely on how a caller sounds. Pause, call back through a known number, and verify the emergency with another trusted person before sending money.
  2. Mountain View desk4 min
    Google’s SynthID Detector: How to Check AI-Generated Images, Video and AudioGoogle’s SynthID Detector looks for an embedded watermark in supported images, video and audio. Here is what its results do—and do not—show.
  3. Redmond desk20 min
    How to create a link to File or Folder in Windows 11Windows 11 gives you several ways to point to a file or folder without moving or duplicating it. You can create a desktop shortcut,…
Recommended PC Tool
Recommended PC Tool
Crashes, No Sound, or Screen Glitches?Free driver scan
PC Slower Than It Used to Be?Free scan - under a minute

Two free Windows tools

One Free Minute Could Fix That PC

Before you go - each of these free tools takes about a minute and tackles what quietly slows a Windows PC down.

Special offer. View Outbyte info, uninstall instructions, EULA, and Privacy Policy.