For a token-priced AI API, add the charge for each billed usage category in a request—usually input and output tokens, with separate terms for cached tokens or other billable features. To estimate a service’s cost, repeat that calculation across representative request types, weight by how often each occurs, and compare providers or hosting options using the same workload and service requirements.
Calculate the model charge for one API request
Use the rates for the provider, model, billing route, and endpoint your application will actually use. For each billed category, multiply its token count by its price per million tokens, then divide by 1,000,000:
As an Amazon Associate I earn from qualifying purchases.
Request model charge = Σ(category tokens ÷ 1,000,000 × category price per million tokens)
Free tools Windows power users keep installed
One-click scans. No signup required.
At minimum, separate input from output: they can have different rates. OpenAI’s enterprise pricing formula also distinguishes cached input from ordinary input; other rate cards may list cache writes, cache reads, or additional billable features. Apply each rate only to the usage category it covers. See OpenAI’s enterprise pricing formula, OpenAI pricing, and Anthropic pricing.
#1 Best Overall
Worked arithmetic example
Suppose a request uses 2,000 input tokens and 500 output tokens. If the applicable rates are I dollars per million input tokens and O dollars per million output tokens, its model charge is 0.002 × I + 0.0005 × O. This illustrates the calculation; it is not a quoted price or measured benchmark. If some input is billed at a cache-read rate, split those tokens out and apply that rate to them rather than charging all input at the ordinary input rate.
Estimate cost across real traffic
A single hand-picked prompt is rarely a useful estimate for a service whose context length, answer length, or cache behavior varies. First gather token usage from provider usage fields or your own request logs for representative traffic. Then calculate separate costs for the request classes that matter and weight them by their share of requests.
- Group the workload: include materially different prompt and completion lengths, cache hits and misses, and tool-using requests where relevant.
- Calculate each group: apply the matching rate categories to its observed token counts and billable features.
- Weight by traffic share: multiply each group’s cost by its proportion of requests and add the results to get an estimated average request cost.
- Scale to expected volume: multiply that weighted average by the expected number of requests for the period.
- Keep a range when inputs are uncertain: model low and high cases for request volume, completion length, or cache behavior instead of presenting a falsely precise point estimate.
Use measured token usage where possible rather than treating character count as a dependable proxy. NVIDIA’s sizing guidance also identifies request lengths, cache hit rate, concurrency, latency targets, model selection, and contract duration as sizing inputs: NVIDIA inference sizing guidance.
Quick wins for a faster PC:
Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Clear out junk files and repair common Windows errorsFree Scan →Scan for outdated or missing drivers - takes under a minuteDriver Scan →Check pricing route and modifiers
The same model name does not guarantee the same billing terms across endpoints or regions. Direct-provider pricing can differ from a cloud marketplace or model platform. AWS says OpenAI models on Bedrock are billed through AWS, while Anthropic notes that partner-operated Bedrock and Vertex AI pricing is independent of its direct API regional pricing. Confirm the route and geography your workload will use: AWS Bedrock pricing and Anthropic pricing.
Before finalizing an estimate, check whether the applicable rate card includes relevant adjustments for batch processing, priority or fast service, long-context brackets, geographic processing, or tools. Eligibility and discount interactions are provider-specific; do not assume that separate discounts combine. Recheck the official rate card before relying on an estimate because prices and service tiers can change.
Estimate self-hosted cost per completed request
For a model you operate, divide the serving cost allocated to a period by the number of completed requests served in that period:
Self-hosted cost per completed request = allocated serving cost for a period ÷ completed requests served in that period
Define the accounting boundary for allocated cost. It may include rented or amortized accelerators and associated operating costs. Measure throughput with the intended model, real workload, concurrency, and latency target, and account for paid capacity that sits idle. NVIDIA’s TCO guidance cautions that hourly hardware price alone obscures throughput and latency; lower utilization can increase effective token cost because infrastructure expense continues while output falls. See NVIDIA’s inference TCO guidance.
Best Value
Compare API and self-hosted options fairly
Compare measured cost per completed request or cost per token at the workload you need—not an API bill against a GPU hourly quote. Keep the comparison’s workload and service requirements aligned:
- Model capability and quality on the task
- Observed request mix, including input, output, and cache-adjusted usage
- Required peak concurrency, latency, throughput, and reliability
- Geography, batch use, and service-tier modifiers
- For self-hosting, operational overhead, paid capacity, and utilization
NVIDIA’s sizing and TCO guidance discuss these cost and performance factors, but they cannot determine which deployment is cheaper for a particular service without its workload and requirements: NVIDIA sizing guidance and NVIDIA TCO guidance.
How to interpret published inference benchmarks
NVIDIA’s inference page reports $4.20 per million tokens on its stated Hopper configuration and $0.12 per million tokens on its stated Blackwell configuration. These are vendor-presented benchmark claims tied to particular hardware and test conditions, not general market prices or a substitute for benchmarking your model and traffic pattern. See NVIDIA’s inference benchmark page. NVIDIA’s TCO guidance states, “AI inference economics depend on the cost per token and overall system throughput rather than raw hourly hardware rates”; treat that as the vendor’s guidance, not an independent standard.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
