Training and inference are different parts of an AI system’s life: training uses large-scale computation to develop a model’s parameters, while inference runs that trained model to generate an answer or prediction. A costly training run can therefore coexist with cheap individual requests—but “millions” and “cents” are framing, not universal prices. Actual costs depend on what is counted, the model and workload, hardware and rental rates, and how the service is configured.
What training and inference mean
Training builds the model
During training, a model processes data and adjusts its parameters so it can perform a task. Large-scale training can require substantial compute over an extended period. The compute bill is only one part of development: data preparation and storage, networking, security, and data-residency requirements can also add cost.
As an Amazon Associate I earn from qualifying purchases.
Inference uses the trained model
Inference is the repeated execution of a trained model to produce outputs. A single request may be inexpensive, but serving a product means paying for the systems that handle requests over time. The total depends on demand and the design of the service, including its performance requirements.
The Tool Desk
Outbyte Driver Updater FREEScan for outdated or missing drivers - takes under a minuteDriver Scan →Outbyte PC Repair FREEClear out junk files and repair common Windows errorsFree Scan →Why training estimates can reach millions
A training-cost figure is meaningful only when its scope is clear. The 2026 Economic Report of the President estimates cloud compute cost by multiplying historical cloud rental cost by training chip-hours. That is a compute estimate, not necessarily the full cost of developing and operating a model.
#1 Best Overall
One 2024 study, “The rising costs of training frontier AI models”, estimated that amortized costs of the most compute-intensive training runs had grown by 2.4 times per year since 2016, with a 90% confidence interval of 2.0–2.9 times. The estimate applies to that study’s selected, most compute-intensive runs; it is not a yearly cost rule for every AI model.
Project costs can extend beyond the training run itself. AWS’s overview of generative AI costs identifies data preparation and storage, pre-training, security, data sovereignty and residency, and low-latency networking as considerations. These needs vary by project, so two models with similar compute requirements can still have different total costs.
Why an individual inference request can cost cents—or less
Inference is usually discussed as a recurring cost because a deployed model serves requests over time. AWS notes that inference expenses vary with customer demand. Cost per request depends on the model, the amount of work in each request, the hardware and serving arrangement, and the service’s performance requirements. A low per-request figure does not reveal the total bill without the request volume and the costs included.
Efficiency improvements can change the unit economics, but benchmark claims are specific to their tested configuration. NVIDIA reported that B200 cost per million tokens for GPT-OSS-120B fell from $0.11 at launch to $0.02 within two months, attributing the figures to SemiAnalysis InferenceX benchmarks as of April 2026. That is a vendor-presented result for a named model and benchmark—not a general price for AI inference.
How to compare AI cost claims
Before comparing a training estimate with an inference price, check that the figures describe comparable scopes. Training is often reported as a project or run total; inference is often reported per request or per token. A unit price cannot be compared directly with a one-time total.
- Workload and model: Identify what is being trained or served and how much computation it requires.
- Volume: For training, look for training chip-hours; for inference, look for request or token volume.
- Hardware and rates: Check the hardware used and the rental or operating rates behind the estimate.
- Included costs: Establish whether the figure covers compute alone or also data, storage, networking, security, and other infrastructure.
- Performance requirements: Latency and serving capacity affect the system needed to meet demand.
- Benchmark context: For vendor results, retain the named model, hardware, software, benchmark, and date rather than treating the number as a market-wide rate.
What an energy estimate per prompt does—and does not—tell you
Per-prompt energy figures are also tied to a particular system and measurement boundary. Google Cloud’s May 2025 analysis estimated that a median Gemini App text-generation prompt used 0.10 Wh of energy, produced 0.02 gCO₂e, and used 0.12 mL of water. Google describes this as a non-comprehensive, point-in-time estimate that considers active TPU and GPU consumption. It is not a universal footprint for an AI prompt or a full accounting of every part of the service.
Quick Recap
Best Value
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
Do these 3 things before closing this tab:
1Fix the driver behind crashes, sound loss and screen glitches2Clear out junk files and repair common Windows errors3Scan for outdated or missing drivers - takes under a minute
