Prompt caching can reduce the cost of repeated LLM input, but a 70–90% reduction in your total bill is possible only for workloads with a large reusable prompt, frequent cache hits, and relatively little uncached input and output. The discount applies to eligible cached input tokens—not automatically to every token or to the whole bill. Here’s how to calculate the difference.
What prompt caching reuses—and what it does not
Prompt caching lets a provider reuse work for eligible context that appears again, often a matching prompt prefix. Instead of processing that repeated portion as ordinary input on every call, the provider may charge a lower rate when it reads the cached tokens. OpenAI describes the feature this way: “Prompt caching reuses work when requests share the same prompt prefix.” See the OpenAI prompt caching guide.
As an Amazon Associate I earn from qualifying purchases.
The request still has costs. New or changed content must be processed, a cache miss may mean processing the full prompt, and generated output is billed separately. Depending on the provider and caching method, writing or keeping a cache can also add charges. A lower price for cache reads is therefore not the same as an equal percentage reduction in the total request cost.
How to calculate prompt-caching savings
Start with the same set of requests under two scenarios: ordinary input processing and caching. Include every billing category in both totals.
#1 Best Overall
- Professional AI & Creator Workstation: AMD Radeon AI PRO R9700 GPU with 32GB GDDR6 is engineered for AI development, professional content creation, and compute-intensive workloads.
- Massive 32GB Memory Capacity: 32GB of GDDR6 memory on a 256-bit bus provides ample bandwidth for large AI models, 8K video editing, and complex 3D rendering.
- Advanced RDNA 4 with AI Accelerators: 64 Compute Units with 3rd Gen Ray Tracing and dedicated 2nd Gen AI Accelerators for groundbreaking AI performance and visual computing.
- Professional Blower Cooling: Efficient single blower design exhausts heat directly out of the chassis, ideal for multi-GPU workstation and server configurations.
- Enterprise-Grade Thermal Solution: Vapor chamber heatsink with industrial Honeywell PTM7950 thermal interface material ensures reliable cooling under sustained professional loads.
- Uncached baseline input cost: total input tokens × standard input rate.
- Cached input cost: cache-write tokens × write rate + cache-read tokens × read rate + uncached tokens × standard rate + any applicable cache-storage charges.
- Total request cost: input cost + output-token charges + any other applicable charges.
- Savings percentage: (baseline total request cost − cached total request cost) ÷ baseline total request cost × 100.
Use the same model, token counts, output, and other charges in both scenarios. That isolates the effect of caching rather than attributing unrelated changes to it.
Worked example: a 71.4% reduction in input cost, not necessarily in the bill
This illustration uses Anthropic’s published cache-price multipliers, as listed on its pricing page accessed October 7, 2026: a 5-minute cache write costs 1.25× the base input rate, and a cache read costs 0.1× that rate. The dollar price depends on model and applicable context tier; this example uses rate units, not a quoted dollar price. See Anthropic pricing.
Rank #2
- NVIDIA Volta GV100 Architecture — 4,608 CUDA Cores, 640 1st-Gen Tensor Cores delivering 14 TFLOPS FP32 and 112 TFLOPS deep learning performance for AI training, inference, HPC, and scientific computing workloads
- 32GB HBM2 ECC Memory — 900 GB/s Bandwidth — High-bandwidth memory on a 4096-bit bus with ECC error correction provides the memory capacity and throughput required for the largest AI models, simulations, and datasets
- PCIe 3.0 x16 Interface — 250W TDP — Standard PCIe Gen3 connectivity with passive cooling designed for enterprise rack server deployment in HPE ProLiant, Dell PowerEdge, and Supermicro platforms with adequate chassis airflow
- NVLink — Scale to 96GB Unified Memory — Connect two V100 GPUs via NVLink at 300 GB/s bi-directional bandwidth to scale GPU memory from 32GB to 96GB for larger AI training and HPC workloads
- Multi-Precision Computing — Supports FP64 (7 TFLOPS), FP32 (14 TFLOPS), FP16 (112 TFLOPS) and INT8 precision modes for flexible deployment across training, inference, and scientific simulation workloads
Assume ten calls each contain a 10,000-token eligible stable prefix and 1,000 tokens of new input. The first call writes the prefix; the other nine calls read it within the 5-minute cache duration. Assume every later call hits the cache, and exclude output and any other charges to calculate input cost only.
Free tools Windows power users keep installed
One-click scans. No signup required.
| Input category | Calculation | Cost in base-input-rate units |
|---|---|---|
| Baseline: all input uncached | 110,000 tokens × 1.0 | 110,000 |
| First-call cache write | 10,000 tokens × 1.25 | 12,500 |
| Nine cache reads | 90,000 tokens × 0.1 | 9,000 |
| New input across ten calls | 10,000 tokens × 1.0 | 10,000 |
| Cached-workload input total | 12,500 + 9,000 + 10,000 | 31,500 |
Input savings are (110,000 − 31,500) ÷ 110,000 = 71.4%. This is not a 71.4% total-bill saving: the example excludes output charges. If output and other costs add up to 44,000 of the same rate units in each scenario, total savings become (110,000 − 31,500) ÷ (110,000 + 44,000) = 51.0%. The same saved input cost is a smaller share of a larger bill.
Rank #3
- Built for Running LLMs Locally: RDNA 4, 128 AI Accelerators, up to 1,531 TOPS (INT4) for fast inference and fine-tuning
- 32GB GDDR6 VRAM for Large AI Models: 256-bit, up to 640GB/s bandwidth, run large language and multi-modal AI models without offloading
- Multi-GPU Scaling for Local AI Clusters: PCIe 5.0 and 2-slot design support dense multi-GPU builds for local AI training and inference clusters
- Diecast Shroud and Backplate: Wave-pattern design cuts memory temperature by up to 16%, keeping clocks steady during long AI training runs
- Phase-Change GPU Thermal Pad: Delivers superior thermal conductivity for consistent performance and longevity under heavy AI loads
What it takes to reach 70–90% of the total bill
In the example, caching saves 78,500 rate units. To save at least 70% of the total request cost, all costs that are unchanged by caching—including output—must be no more than about 5,000 units: 78,500 ÷ 0.70 − 110,000. That is roughly 4.5% of the baseline input cost. The threshold changes with the workload and price structure, but the principle holds: output and uncached input dilute input savings.
A 90% total-bill reduction requires an even more favorable workload. Cache reads may be much cheaper than standard input, but the first write, changing content, misses, output, and any storage charges remain. With repeated calls, a large stable prefix, and little else in the bill, the cached-input cost can approach the read rate; total costs do not automatically do so. The headline range is a conditional outcome to calculate—not a general provider guarantee.
Rank #4
- 24GB GDDR7 ECC Memory: handles large AI, 3D and rendering files smoothly
- Powerful CUDA Compute - 8,960 CUDA cores for fast graphics and computing power
- AI & Ray Tracing Boost - Tensor of the 5th generation and RT cores of the 4th generation
- PCIe 5.0 x16 interface - fast data connection with modern systems
- 4 × DisplayPort 2.1 - Multi-monitor support for professional workflows
How caching differs across providers
Eligibility, cache controls, rates, expiry, and usage reporting depend on provider and model. The figures below reflect provider documentation accessed October 7, 2026; check the linked current documentation for the exact model and configuration you use.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
| Provider | How caching is described | Rates, eligibility, and retention details established in the cited documentation |
|---|---|---|
| OpenAI | Automatic caching for supported models when requests share an eligible prefix; the guide also describes cache controls and usage diagnostics. | The guide lists a 1,024-token minimum for GPT-5.6 and later. Minimums and behavior for earlier models depend on request settings. Breakpoints, rates, and retention options are model-dependent; consult the OpenAI guide. |
| Anthropic | Cache durations described include 5 minutes and 1 hour. | The pricing page accessed October 7, 2026 lists 5-minute writes at 1.25× base input pricing, 1-hour writes at 2×, and reads at 0.1×. Dollar rates depend on model and applicable context tier; see Anthropic pricing. |
| Google Gemini | Documentation covers implicit caching for supported models and explicit context caching. | Eligibility, token minimums, price reductions, and storage costs depend on model and configuration. Explicit caching may incur storage-duration charges. The cited documentation does not establish a universal hit rate or guarantee a saving: see Gemini context caching and API optimization. |
These differences affect the calculation. For example, a cache that expires before the next request may require another write, while a cache miss can turn an expected read into ordinary input processing. OpenAI’s Prompt Caching 101 cookbook explains the basic miss path: without a match, the full prompt is processed, and an eligible prefix may be cached for a later request.
How to improve hit rates and verify savings
- Identify repeated context. Separate instructions, tool definitions, and other stable material from request-specific content. Caching helps only where requests share eligible context.
- Keep reusable material early and consistent. Put stable content before changing content where the provider’s matching rules allow. Avoid unnecessary edits to the reusable prefix.
- Use provider-supported controls selectively. Set explicit breakpoints or use context objects only when the provider supports them and the expected reuse justifies any write or storage costs.
- Instrument the actual billing categories. Record cache-read and cache-write tokens, uncached input, output tokens, total cost, and latency. A repeated API call or an ongoing session alone does not prove that the provider recorded a cache hit.
- Compare representative runs. Compare otherwise similar cached and uncached workloads across enough calls to include misses and expiry. Use actual usage fields and charges, not a theoretical cache discount, to estimate savings.
Use live provider pricing for the model and configuration in production. OpenAI’s October 2024 announcement contains historical pricing examples; they should not be treated as current prices.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
