Prompt caching can reduce the cost and processing time of repeated long prompts by reusing work on a stable prefix or saved context. It does not make new or changed input free: cache creation, cache reads, uncached additions, and sometimes storage each have their own billing rules. To tell whether it saves money, check the exact provider and model’s cache rules and compare the full cost across a realistic sequence of requests.
How does prompt caching work?
A language model normally processes the input tokens in each request. When a later request reuses a matching prefix, a provider may reuse computed state from that prefix rather than process it all again. OpenAI describes this as reusing key-value (KV) states; its guide says the cache stores KV tensors, not the prompt tokens themselves. Another pattern is explicit context caching: save shared content as a resource and refer to it in later requests, as Gemini supports.
As an Amazon Associate I earn from qualifying purchases.
In either case, only eligible repeated content can benefit. A changed prefix, unsupported model or endpoint, insufficient prompt length, expired cache, or misplaced breakpoint can prevent reuse. Keeping a conversation or session open does not by itself guarantee a hit.
PC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11Crashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minuteDoes prompt caching actually reduce your API bill?
It can, when the savings on repeated input outweigh cache creation, cache-read, storage, and miss costs. A high cache-hit rate alone is not proof that a workload is cheaper: the read may be discounted, but writing or storing the context can add charges, and request-specific input is still billed under the provider’s rules.
#1 Best Overall
There is no defensible universal savings percentage or single cheapest provider without specifying model, endpoint, region, cache lifetime, and reuse pattern. Compare the complete costs for your own workload, and use current provider pricing rather than assuming rates are interchangeable.
Estimate cost over a request sequence
- Count the stable tokens you expect to reuse and the changing tokens sent with each request.
- For the exact model and endpoint, find ordinary input, cache-write, cache-read, and any storage-duration rates.
- Estimate initial writes, later reads, uncached suffixes, cache misses, and rewrites over the cache’s lifetime.
- Compare that total with sending the full prompt uncached for the same requests.
- Validate the estimate with usage reports from representative requests, then recalculate if the prompt, traffic pattern, or configuration changes.
How the caching options differ by provider
These are different mechanisms, not one cross-provider feature with a shared syntax or billing model. The details below reflect official documentation available on October 7, 2026; supported models, thresholds, prices, and field names can change.
| Provider | How reuse is requested or applied | Lifetime and billing considerations | What to verify |
|---|---|---|---|
| OpenAI API | Prompt caching is enabled by default for supported models and reuses matching prompt prefixes. Breakpoints and controls vary by model; the guide describes prompt_cache_options for GPT-5.6-and-later behavior and retention options for earlier models. |
Uncached input, cache reads, and cache writes can have model-dependent rates. A model-dependent minimum input length applies. | Confirm model support, minimum length, prefix/breakpoint behavior, retention controls, current rates, and reported cache usage in the OpenAI prompt-caching guide. A session alone does not ensure a hit. |
| Anthropic Claude API | Use cache-control markers according to the model’s rules. | The documentation describes a default five-minute lifetime and an optional one-hour duration at additional cache-write cost. It lists cache creation and reads separately; multipliers depend on model. The documentation available October 7, 2026 describes five-minute writes at 1.25 times base input pricing, one-hour writes at 2 times base input pricing, and cache hits at a fraction of base input price. | Verify the model’s supported markers, applicable lifetime, current write/read multipliers, and base rate in Anthropic’s prompt-caching documentation. |
| Google Gemini API | Create cached content and reference that resource in subsequent requests; this is an explicit resource pattern. | Storage is billed according to TTL rules. Minimum token counts and supported models vary. | Check supported models, minimums, TTL and storage billing, and cached-token usage reporting in the Gemini context-caching guide and caching API reference. |
Anthropic’s listed multipliers are documentation-specific, not universal constants; verify current rates for the exact model before using them in an estimate. Provider mechanics differ, so do not transfer a marker, TTL assumption, or rate from one API to another.
How to structure requests for reuse
- Place stable system instructions, tool definitions, and shared reference material early in the prompt or in the provider’s cached-content structure.
- Keep user-specific or frequently changing material outside the reusable prefix when the API allows it.
- Follow the selected model’s exact cache breakpoint, cache-control, or resource-reference rules. There is no universal prompt-caching syntax.
- After representative requests, inspect the provider’s cache-related usage fields, such as cached-token counts, cache reads, and cache writes where available. Do not infer a hit merely from prompt length or an open session.
When should you use prompt caching?
It is most worth evaluating when many requests reuse a substantial, stable body of instructions or context. If prompts are short, change substantially each time, or are rarely repeated within the cache’s lifetime, setup, write, or storage charges may outweigh avoided input processing. The outcome depends on actual reuse and the applicable model and billing rules, so use the request-sequence estimate and usage data rather than a general savings claim.
Recheck model support, minimum cacheable length, pricing, usage fields, and TTL whenever you change provider, endpoint, model, or deployment. Those details are volatile and are not safe to assume from a prior configuration.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
