What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
If OpenAI API requests that look alike are not reusing prompt work, compare the actual rendered inputs from the first token onward, then check the request-level cache diagnostics and usage data. A small change near the start can prevent later text from matching; similar-looking prompts alone do not establish that a cache miss is a bug.
Why is my OpenAI prompt cache not hitting?
Prompt caching can reuse an unchanged prefix of a request’s input. The relevant comparison is the token-bearing content in the order it reaches the model, not whether two prompts look broadly similar in your application code. OpenAI also lists compatible model, service tier, and tools among the conditions for reuse. See the Prompt Caching guide and Prompt Cache Diagnostics guide.
As an Amazon Associate I earn from qualifying purchases.
Start by capturing two real requests that you expect to share context. Compare their fully rendered inputs from the beginning: system and developer content, tool definitions, conversation history, and all other content before the point you expect to be reusable. A dynamic value inserted early—such as a request-specific date or identifier—can shift the matching boundary, leaving later identical material outside the shared prefix. That is a consequence of the documented prefix-matching rule, not proof that any particular application has this cause.
Do these 3 things before closing this tab:
1Clear out junk files and repair common Windows errors2Fix the driver behind crashes, sound loss and screen glitches3Repair Windows errors before they cause bigger problems- Check for changed whitespace, reordered messages, tool-schema differences, and dynamic text in early messages.
- Check that the model and service tier match, and that the tools are compatible between requests.
- Confirm the prompt meets the relevant model’s cache eligibility threshold before treating a miss as unexpected.
How do I find prompt prefix drift?
- Choose representative requests. Select a pair that should share a prefix and retain the complete rendered request inputs and relevant settings—not just the prompt template.
- Compare from the first token-bearing content. Find the earliest difference in system/developer instructions, tools, history, or other input. The first divergence is the point after which later text may no longer belong to the same matching prefix.
- Inspect request-level diagnostics. Open Prompt Cache Diagnostics for the requests. Use it to examine prefix matching, compatible settings, and whether a cached prefix was hit.
- Check aggregate behavior separately. Use the Prompt Caching Dashboard to monitor cache-read hit-rate trends across the application. The dashboard gives a trend view; it does not, by itself, explain why one particular request missed.
- Validate with usage and cost. Compare cached tokens, total input tokens, any exposed cache-write tokens, latency, and realized cost for the same request set.
When early dynamic content fragments a reusable prefix, one implementation option is to put stable instructions and tool schemas earlier and volatile user-specific data later. Treat this as a design tactic to test, not a guarantee: eligibility, cache availability, settings compatibility, and request behavior still affect reuse. Preserve the intended meaning and behavior of the application when changing message order.
#1 Best Overall
How can I see cached tokens in the OpenAI API?
For Responses API requests, inspect usage.input_tokens_details.cached_tokens. This reports the number of input tokens reused from cache, not a yes-or-no promise that the entire input was cached. Where available, also record cache-write tokens, total input tokens, request latency, and cost. OpenAI’s diagnostics guide illustrates partial reuse: an example request with 2,500 input tokens has a 2,000-token matching prefix and 500 newly processed tokens. Those are illustrative values, not a benchmark.
For an aggregate view, the Usage API reference defines input_cached_tokens for aggregated text input usage. When calculating a hit rate, aggregate cached tokens and total input tokens across the same requests and time window; mixing different populations can make the ratio misleading.
Does my model meet the prompt-cache threshold?
Eligibility rules depend on model generation. OpenAI’s current Prompt Caching guide documents a minimum of 1,024 visible input tokens for GPT-5.6 and later. Hidden OpenAI-provided system tokens do not count toward that minimum. For earlier models, the minimum varies with request settings, and breakpoint behavior and cached-token reporting can differ across generations.
Recommended Free Tools
Check the guide for the exact model and settings you use rather than applying one older threshold across every model. A request that falls below its model’s applicable minimum is not evidence of prefix drift.
How do I tell whether caching is actually making requests cheaper?
Use the usage data from your own requests and the current rates for your exact model. OpenAI’s API pricing page lists model-specific uncached input, cached input, and cache-write rates; rates and cache-write pricing differ by model and can change. There is no single discount percentage or savings result that can be applied reliably to every application.
Compare realized cost over equivalent request sets, alongside cached tokens, cache writes where exposed, total input, and latency. A higher cache-read share is useful evidence of reuse, but cost depends on the model’s rates and the full request pattern. Do not infer expected savings from the eligibility threshold or the diagnostics guide’s illustrative example.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.What should I check before enabling extended prompt caching?
Extended prompt caching has a data-retention consequence. OpenAI’s data-controls documentation says that, for the endpoint use it describes, storing key/value tensors as application state is required and that this use is not eligible for Zero Data Retention. Check the endpoint-specific retention table and your organization and project controls before enabling extended retention.
Quick Recap
Best Value
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
