Prompt caching can meaningfully lower the input-side cost of an LLM application, but only for the part of each request that repeats. A 70% figure is best read as a cache-hit target or a result from one specific workload, not as a typical drop in total LLM spend. Cache-hit rate and bill savings are different metrics, and the gap between them is where most budgeting mistakes happen.
What prompt caching reuses
Prompt caching lets a provider reuse computation it has already done for input that is identical to an earlier request, usually a stable prefix at the start of the prompt. It does not store the answer. Each new request still processes its new content and generates a new response, and output tokens are billed as before.
As an Amazon Associate I earn from qualifying purchases.
The providers describe the mechanism differently. OpenAI states that its prompt cache stores key-value (KV) tensors, not the tokens themselves, and that a later request can reuse saved state for a matching prefix while still processing new input (OpenAI API prompt caching guide). Anthropic matches exact prompt segments up to a marked cache breakpoint (Anthropic prompt caching documentation). Google Cloud describes reusing precomputed input tokens for Gemini models (Google Cloud, Vertex AI context caching).
The Tool Desk
Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →In practice, the reusable material is usually system or developer instructions, tool definitions, output schemas, few-shot examples, and long reference documents. Anything that changes on every call, such as the user’s question, a timestamp, or a freshly retrieved snippet, sits outside the cached part and is billed at the normal input rate.
#1 Best Overall
What the 70% figure can and cannot mean
The number in the headline comes from several different kinds of source, and each one supports a different claim. The table below separates them.
| Figure | Source and date | What it supports |
|---|---|---|
| Up to 95% discount on reused cached input tokens | OpenAI API prompt caching guide (page does not state a publication date) | A discount on eligible cached input tokens. It is not a 95% reduction in the whole request or the invoice. |
| Approximately 70% token cache-hit rate in a single-turn judge example | OpenAI API prompt caching guide (same page) | An illustrative example. OpenAI says actual hit-rate ceilings depend on context and application usage. |
| Above 90% token cache-hit rate in a multi-turn agent example | OpenAI API prompt caching guide (same page) | Another illustrative deployment, not an average across applications. |
| 55% token-cost savings at a 70% cache rate | OpenAI Cookbook worked example, using a 1,100-token prompt (publication date not stated on the page) | A conditional calculation under that example’s assumptions. It is not a measured benchmark for all request costs. |
| Manus: cache hit rate moved from roughly 85% to consistently above 90% | OpenAI announcement on GPT-6 prompt caching, September 22, 2026 | A customer-reported change in hit rate. It is not a general result. |
| Wordsmith: cache hits rose from 83% to 91%; cache writes fell by roughly two-thirds; inference costs fell by 36% | Same OpenAI announcement, September 22, 2026 | A customer case with the same workload before and after. It is not independently audited. |
The gap between a hit rate and a bill is easiest to see in the Cookbook’s own arithmetic. A 70% cache rate produces 55% token-cost savings there, not 70%, because cached tokens are still a share of total spend, writes carry their own charge, and uncached input and output keep their full price. Your savings will depend on the share of input that repeats, how many requests arrive inside the cache window, current provider prices, write and storage charges, and how much output you generate.
Rank #2
Wordsmith’s engineer described the change in these terms: “Working with OpenAI, we moved our session agents to explicit cache breakpoints. In under a week, cache hit rates on our evaluations rose from 83% to 91%. This meant fewer cache writes and lower inference costs with the same workload.” (Eugene Mikhantyev, AI Engineer, quoted in OpenAI’s September 22, 2026 announcement.) The quote shows what moved in that case: the breakpoint placement and the number of writes, on a fixed workload.
Outdated Drivers Are Slowing You Down
One free scan finds every outdated or missing driver and matches the right update for your exact hardware.Free scan · exact hardware matchWindows Errors? Fix Them Before They Spread
Repair common Windows errors and clear accumulated junk for a smoother, more stable PC - no reinstall needed.Free scan · no reinstallHow the major providers differ
Caching is not one feature with one set of rules. Minimum lengths, retention, isolation, and price multipliers vary, and the same Claude model can behave differently on the Claude API than on Bedrock or Vertex AI. Verify the rules for your exact model and deployment platform before you build a cost estimate. The figures below are from provider documentation as of October 2026 and change over time.
Rank #3
| Provider | How caching is enabled and matched | Cost points to model |
|---|---|---|
| OpenAI API | Enabled by default for supported models. Newer controls include implicit and explicit breakpoints and diagnostics. Matching uses the rendered prefix, and minimum lengths are model-dependent (OpenAI API prompt caching guide). | Per the current guide, cache writes on GPT-5.6 and later cost 1.25× standard input and subsequent reads cost 0.1×, on most such models; some models are exceptions. Confirm current API pricing for your model. |
| Anthropic Claude API | Automatic or explicit caching on active Claude models. Segments must match exactly up to the marked block. Isolation is workspace-level for the Claude API, Claude Platform on AWS, and Microsoft Foundry (beta), starting February 5, 2026. Bedrock and Vertex AI keep organization-level isolation (Anthropic prompt caching documentation). | 5-minute writes at 1.25× base input; 1-hour writes at 2×; reads at 0.1×. A 5-minute cache is refreshed at no extra cost when it is used. The 1-hour option costs more up front and suits longer gaps between requests. |
| Google Cloud Vertex AI | Implicit caching is enabled by default. Explicit caching gives control and, according to Google, a guaranteed discount. Google’s Vertex AI post describes a 2,048-token minimum for the caching it covers, and implicit entries are deleted within 24 hours (Google Cloud, Vertex AI context caching). | Cached-token input is priced at 10% of standard cost for supported Gemini 2.5 and later models. Explicit caching also carries storage charges based on its TTL. Check model and region pricing and current limits. |
Implementation steps
- Find the repeated prefix. Log a sample of production requests and identify the instructions, tool definitions, schemas, examples, and reference text that recur word for word. A long prompt is not automatically cacheable; what matters is how much of it is identical across calls.
- Put reusable content first. Place stable material before the user’s question, timestamps, retrieved snippets, and tool outputs. Caching depends on exact prefix matching, so a change near the start of the prompt invalidates reuse for everything after it.
- Freeze serialization. Keep the text, field order, tool list, and relevant settings identical across requests. Do not put request IDs or timestamps into the shared prefix. Where the provider supports explicit breakpoints, set one after the stable block and leave a low-reuse suffix uncached so you do not pay write charges on content that will not be read again.
- Check the provider’s constraints for your model. Confirm minimum cacheable length, supported cache modes, retention window, region or platform behavior, and whether the cache is isolated per workspace or per organization.
- Pick a TTL from your request cadence. A 5-minute window suits traffic that arrives in bursts. A longer retention suits slower agent loops, but its write or storage cost is higher, so it only pays if the gaps are long enough to need it.
- Measure the invoice, not just hits. Compare cached input tokens, cache-write tokens, uncached input tokens, output tokens, latency, and total cost before and after the change. Include low-reuse and cache-miss traffic in the test, not only the best-case flow.
Estimating whether caching pays for your application
The break-even point depends on the write multiplier. Using Anthropic’s 5-minute multipliers and ignoring output and any uncached input, a prefix that is written once and then read n times costs 1.25 + 0.1(n − 1) units of base input, against n units without caching. That is cheaper from the second request inside the window onward. With the 1-hour write at 2×, the same calculation breaks even at roughly the third request.
This is arithmetic on published multipliers, not a measured saving. Real savings are lower when the cached share of the prompt is small, when output is a large part of the bill, or when requests are spaced too far apart for the cache to stay warm.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Why your cache is missing
- The prefix changes. A timestamp, a per-user ID, or a reordered tool list early in the prompt breaks the match for everything after it.
- The prefix is below the minimum length. Google’s described caching uses a 2,048-token minimum, and OpenAI’s minimums depend on the model. Shorter prefixes will not be cached.
- The gap exceeded the retention window. If requests arrive after the TTL has expired, each one writes again. Consider whether a longer TTL costs less than the writes it prevents.
- The platform isolates caches differently. Workspace-level and organization-level isolation mean that calls routed through a different account, project, or cloud platform may not share cache entries.
- The model or region changed. Cache behavior is tied to the specific model and deployment, so a model upgrade can reset hit rates.
Is it worth doing for your app?
Caching is most likely to pay off when a large, identical block of context is sent many times in a short period: a shared instruction set for a support bot, a fixed rubric applied to many evaluation items, a long document queried repeatedly, or an agent that resends its tool definitions on every turn. It is least likely to pay off when every request is unique, when prompts are short, or when most of the cost is output. Run the six steps above on a sample of real traffic, and treat the result as your own number rather than the headline figure.
Do these 3 things before closing this tab:
1Fix the driver behind crashes, sound loss and screen glitches2Clear out junk files and repair common Windows errors3Scan for outdated or missing drivers - takes under a minuteFor a single-stack team, the practical target is a measured reduction in input cost on the workload you actually run, with cache writes checked against the reads they produce. A 70% reduction in total spend is possible only when almost all input repeats and output is a small share of cost. Most applications will see smaller gains, and some will see none.
Current rates and supported models change often. Check each provider’s pricing page and documentation on the day you make the decision.
Primary sources: OpenAI API prompt caching guide, OpenAI Cookbook prompt caching example, OpenAI announcement on GPT-6 prompt caching, Anthropic prompt caching documentation, and Google Cloud Vertex AI context caching.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
Quick wins for a faster PC:
Repair Windows errors before they cause bigger problemsFix Now →Scan for outdated or missing drivers - takes under a minuteDriver Scan →
