The security company ProjectDiscovery once saw a cache hit rate of just 7% on its AI agent. Its biggest fix did not involve a new model. It moved changing text from the middle of the prompt to the end, and with a few related cleanups the hit rate climbed to 84%. Caching saved 59% on LLM costs overall compared with full input rates, and 70% in the last 10 days it measured. (Sources: ProjectDiscovery, Digital Applied)
That is the honest story of prompt caching. It is one of the largest discounts in the AI API business, reaching 95% on cached tokens at OpenAI, Anthropic and Google for some current models, and it fails silently. Requests that miss still succeed. You only find out from a bill or a usage field. This guide covers how it works, what it really saves, how the big providers differ as of November 2026, and the 6 mistakes that cost the most.
Quick summary: Prompt caching reuses the already processed start of a prompt, so repeated tokens bill at a fraction of the input rate. Reads cost 0.05x input on Claude Opus 5.5 and GPT-6.1 Terra. Writes can cost more, 1.25x for short lifetimes. It only touches input, so a 95% token discount became a 75% bill cut in our example. Any change early in the prompt breaks the match, so static content goes first and anything that changes goes last.
How prompt caching works
Every request to a model starts by processing the prompt. When most of a prompt is the same as last time, such as a long system prompt, tool definitions, a reference document or the earlier turns of a chat, the provider can reuse that work. The match runs from the first token forward and needs to be exact. OpenAI's guide says the "entire rendered prefix" must match, and Anthropic requires 100% identical content up to the breakpoint. On Claude, the order is tools, then system, then messages, so an edit to a tool invalidates everything after it. (Sources: OpenAI, Anthropic)
This is different from caching answers. Anthropic says caching has no effect on output generation, and OpenAI describes it as reusing the processed prompt prefix: the model still generates a fresh response, and you still pay full price for those output tokens. Caches also are not shared between organizations at either company, and on the Claude API they are further isolated by workspace. (Sources: OpenAI, Anthropic)
How the providers compare
Prices below are per million tokens as listed in early November 2026, and they change often. Read price is shown as a multiple of the normal input rate.
| Model | How you enable it | Read price | Write price | Lifetime | Minimum prefix |
|---|---|---|---|---|---|
| Claude Opus 5.5, Sonnet 5.5 | One cache_control field, or up to 4 breakpoints | 0.05x ($0.20 on Opus 5.5) | 1.25x for 5 min, 2x for 1 hour | 5 min, refreshed on each read; 1 hour optional | 512 tokens |
| Claude Haiku 5.5 | Same as above | 0.1x ($0.01 up to 100K-token prompts) | 1.25x for 5 min, 2x for 1 hour | Same as above | 512 tokens |
| GPT-6.1 Sol, GPT-6.1 Terra | On by default | 0.05x ($0.05 on Terra) | 1.25x for 30 min, 2.5x for 12 hours | 30 min default; 12 hours in the Responses API | 1,024 visible tokens |
| Other GPT-5.6 and later | On by default | 0.1x ($0.01 on GPT-6 Luna) | 1.25x for 30 min, 2.5x for 12 hours | Same as above | 1,024 visible tokens |
| Gemini 4 Argon | Implicit only, on by default | 0.05x ($0.10, rising to $0.20 on Dec 1) | No write rate listed | Not stated; hits are not guaranteed | 4,096 tokens |
| DeepSeek V4.1 Flash 1017 | On by default | 0.02x ($0.006 peak, $0.003 off-peak) | No write rate listed | Best effort; cleared in hours to days | Not published |
One shift is easy to miss if you learned caching a year ago. OpenAI writes used to be free. On GPT-5.5 and earlier there is no extra write charge, but GPT-5.6 and later bill writes at 1.25x, so old advice that caching on OpenAI cannot cost you anything no longer holds. Google has moved too: explicit caching is not supported on Gemini 4 Argon or later, which leaves implicit caching as the only option there. (Sources: OpenAI, Flexera, Google)
What it really saves: two worked examples
A support bot on GPT-6.1 Terra
Assume a 10,000-token fixed prefix, a 200-token user question, a 500-token answer, and 1,000 evenly spaced requests a day. Terra costs $1 input, $5 output, $1.25 for 30-minute writes and $0.05 for reads. Without caching, the prefix costs $10.00, the questions $0.20 and the answers $2.50, for $12.70 a day. With caching, one write costs $0.0125 and 999 reads cost about $0.50, so the prefix falls to about $0.51 and the day totals about $3.21. That is a 95% cut on the prefix but a 75% cut on the bill, because output pricing does not move. These are our calculations from list prices, and the scenario is illustrative.
A 30-turn coding agent on Claude Opus 5.5
Agents benefit most because each step resends everything so far. Assume a 20,000-token start and 2,000 new tokens per turn, for 1.47 million input tokens across 30 turns. At $4 per million, that is $5.88 uncached. With caching, 78,000 tokens are written at $5 ($0.39) and 1.392 million are read at $0.20 (about $0.28), for about $0.67, an 89% cut on input. Now add one 6-minute pause before turn 16, longer than the default 5-minute lifetime. That turn rewrites all 50,000 tokens for $0.25 instead of about $0.02, and the session rises to about $0.90, or 34% more. A write costs 25 times a read on this model, which is why gaps matter so much. Output is excluded here, and these figures are our own math from list prices.
Independent testing supports the direction of these numbers, with a wider range. A January 2026 arXiv study of more than 500 agent sessions with 10,000-token system prompts, across OpenAI, Anthropic and Google, found API costs fell 41% to 80% and time to first token improved 13% to 31%. (Sources: arXiv, Digital Applied)
6 mistakes that quietly erase the savings
1. Putting something that changes at the top
A timestamp, request ID or user name in the system prompt changes the prefix on every call, so nothing after it can match. ProjectDiscovery's fix included freezing the date once per run, date only, rendering prompts with placeholders, and moving working memory to a message at the end. (Sources: ProjectDiscovery, Anthropic)
2. Serializing tools or JSON in an unstable order
If your code builds tool lists from an unordered structure, the bytes can differ between calls even when the meaning does not. Anthropic specifically warns that unstable key ordering in tool_use JSON, common in Swift and Go, breaks matches. Sort tools and keys before sending, and load any dynamic tools last. (Sources: Anthropic, Flexera)
3. Changing settings in the middle of a session
Switching models starts over, because caches are tied to the model. Other settings also count. OpenAI lists tools, parallel_tool_calls, text.format, reasoning.effort and text.verbosity. Anthropic lists tool definitions, the system prompt, web search or citations toggles, speed mode, tool_choice and thinking or effort settings at various levels. A router that bounces between models on each step can lose most of the benefit. (Sources: OpenAI, Anthropic)
4. Falling just under the minimum
Short prompts simply are not cached, and Anthropic says no error is returned. The floor varies widely: 512 tokens on current Claude 5.5 models, 1,024 on GPT-5.6 and later, and 4,096 on Gemini 4 Argon. On older Claude models such as Opus 4.6 and Haiku 4.5, it is also 4,096. A 3,000-token prompt can cache on one provider and not on another. Google also notes that on Gemini 4 multi-turn chats, the full previous request is cached only if the new request adds at least 2,050 tokens. (Sources: Anthropic, OpenAI, Google)
5. Letting the cache expire between requests
Claude's default entry lasts 5 minutes, refreshed free on each read, and that clock starts when a request begins, so long streaming replies eat into it. If traffic is sparse you pay the write premium again and again. The math on the longer option is simple: a 5-minute Claude write or a 30-minute OpenAI write pays for itself after 1 read, while a 1-hour Claude write or a 12-hour OpenAI write needs 2 reads at 0.05x pricing. A prefix that is never read again costs you 25% more than no caching, or 100% to 150% more on the long lifetimes. That break-even is our arithmetic from published multipliers. (Sources: Anthropic, OpenAI)
6. Rewriting history to save tokens
Compaction, summaries and edits to earlier turns change the prefix. OpenAI lists compaction or summarization that rewrites earlier context among the things that stop reuse. Summarizing can still be the right call in a very long session, but do it rarely and on purpose, not on every turn, because each rewrite triggers a new write of the whole context. (Sources: OpenAI, Digital Applied)
How to check whether it is working
Log the usage fields on every call and chart your hit rate. A sudden drop usually means a deploy changed your prefix.
- Claude: cache_read_input_tokens, cache_creation_input_tokens and input_tokens. Add all three for total input. The input_tokens field only counts what comes after the last breakpoint, so a 100,000-token cached document plus a 50-token question shows input_tokens of 50.
- OpenAI: usage.input_tokens_details.cached_tokens and cache_write_tokens.
- Gemini: usage.total_cached_tokens in the Interactions API response.
- DeepSeek: prompt_cache_hit_tokens and prompt_cache_miss_tokens.
- If both Claude cache fields read 0, Anthropic says the prompt most likely fell below the minimum.
Caching also has a rate-limit angle. On most Claude models, cache reads do not count toward input tokens per minute, so Anthropic's example turns a 2 million limit at an 80% hit rate into 10 million total tokens per minute. OpenAI's FAQ says the opposite for its platform: cached input tokens still count. Discounts can also combine. Anthropic says caching stacks with the 50% Batch API discount, and OpenAI lists batch rates at 50% for cached input and cache writes. (Sources: Anthropic rate limits, OpenAI, OpenAI pricing, Anthropic pricing)
Before tuning anything, diff two consecutive rendered prompts, byte for byte. The first line that differs is where your cache ends. Move whatever is on that line below everything that stays the same.
01What is prompt caching?
It lets an AI provider reuse the processed start of a prompt that it has seen recently, so those repeated input tokens are billed at a steep discount and the first token often arrives sooner.
02Do cached prompts give stale answers?
No. Only the input processing is reused. The model generates a new response. Anthropic says caching has no effect on output generation, and OpenAI describes caching as reusing the processed prompt, not stored answers.
03How much does prompt caching save?
Reads can be 95% off input on some models, but real bills fall less because output is unaffected. One arXiv study measured 41% to 80% lower API costs across agent sessions.
04Can prompt caching cost more than no caching?
Yes. If a prefix is written at a premium and never read again, you pay 25% or more above the normal rate on models that charge for writes.
05How long does a cached prompt last?
It depends on the provider: 5 minutes by default on Claude with a paid 1-hour option, 30 minutes by default on GPT-5.6 and later with a 12-hour option, and best effort on DeepSeek.
06Do I need this if I only use ChatGPT or Claude in the app?
No. Prompt caching is an API billing feature. It matters if you build on the API or run tools billed by the token.
07Is my cached data visible to other companies?
OpenAI and Anthropic both say caches are not shared across organizations.
If you are not building on an API and just want to compare models, you can skip the token math. LumiChats gives you 40+ AI models under one subscription, which makes it easy to find the right model for a job before you write a line of integration code.
