TL;DR
- Prompt caching reuses computation for an exact input prefix; it does not reuse an old answer.
- On current GPT-5.6 pricing, one write followed by one read saved 32.5% in testing, while changing the cache key every request cost 25% more than uncached input.
- TTL is model-specific: the tested GPT-5.6 models used 30 minutes, while the tested Claude models supported 5-minute and 1-hour entries.
- Edited files do not read stale cache entries because changed content no longer matches the prefix.
- Cache savings come from repeatable request structure, not from enabling the feature alone.
Prompt caching makes repeated context cheaper. It does not make every model call cheaper.
That distinction matters for coding agents because their requests contain large repeated prefixes: system instructions, repository guidance, tool definitions, file contents, and prior conversation. A cache hit discounts the repeated input. A miss processes it again. A write with no later read can cost more than leaving caching off.
What prompt caching stores
An LLM request has two broad phases: processing the input and generating the output. Prompt caching reduces repeated work in the input-processing phase.
It does not store and replay the model’s answer. The model still generates a new response for each request.
Request 1: [stable instructions + tools + files] + [question 1]
└──────────── cache write ────────────┘
Request 2: [same stable prefix] + [question 2]
└── discounted cache read ──┘The key word is same. OpenAI requires an exact prefix match. Amazon Bedrock describes cache checkpoints as contiguous prompt prefixes that must remain static between requests. Images, tools, schemas, and messages all contribute to the rendered prefix.
This makes prompt caching different from a response cache. A response cache returns an old answer for an identical full request. A prompt cache reuses earlier computation and still lets the model answer a new question.
The economics start with reuse
Cache pricing has three relevant token types:
| Token type | Meaning |
|---|---|
| Ordinary input | Content processed at the normal input rate |
| Cache write | Content stored as a reusable prefix |
| Cache read | Matching prefix reused on a later request |
The exact rates depend on the provider and model. For GPT-5.6, OpenAI documents cache writes at 1.25 times ordinary input and cache reads at a 90% discount. Amazon Bedrock documents the same pricing shape for GPT-5.6 on its Responses API.
That produces a simple break-even rule. One write with no reuse costs 25% more than ordinary input. The first later read repays that premium and moves the pair into savings.
I measured the following normalized input economics in a 186-request GPT-5.6 matrix:
| Pattern | Normalized input savings |
|---|---|
| One write + one read | 32.5% |
| One write + three reads | 61.2% |
| One write + five concurrent reads | 70.8% |
| One write + twenty concurrent reads | 84.4% |
| New cache key on every request | 25% more expensive |
Caching is therefore not free money. Reusable prefixes are the asset. Cache writes are the investment required to create them.
TTL is model-specific
The original version of this article treated five minutes as the standard cache lifetime. That framing is no longer accurate.
Current APIs expose different retention contracts:
| Tested model family | Requested retention | Observed result |
|---|---|---|
| GPT-5.6 Luna and Terra | 30-minute minimum | Hit at 10 minutes; rewrite at 31 minutes |
| Claude Sonnet 5 and Opus 4.8 | 5 minutes | Rewrite at 6 minutes |
| Claude Sonnet 5 and Opus 4.8 | 1 hour | Hit at 31 minutes; rewrite at 61 minutes |
Each delayed test used an independent cache entry, preventing an earlier hit from refreshing a later probe. The results show the cache state at those times, not the exact instant of eviction.
The provider documentation also differs in semantics. GPT-5.6 uses a 30-minute minimum lifetime through prompt_cache_options.ttl; OpenAI may retain the entry longer, and the user cannot request another value. Claude exposes 5-minute and 1-hour choices through cache controls on supported models. Anthropic recommends the longer duration for prompts reused less frequently than every five minutes.
TTL controls how long matching work remains reusable. It does not determine whether changed content is stale.
Changed content creates a new prefix
Prompt caching uses prefix identity. If a file changes inside the cached prefix, the new request no longer matches the old entry. The service processes the changed prefix and, when eligible, writes a new cache entry. The old entry can remain until its TTL ends, but the changed request does not read it.
The operational risk is the opposite: accidental changes can destroy reuse.
A timestamp in the system prompt, a reordered tool list, a different JSON schema serialization, or a rewritten earlier message can turn a warm agent loop into repeated cache writes. The model still receives current content; the bill loses the discount.
What coding agents need to keep stable
Coding-agent requests commonly contain:
- System and developer instructions
- Repository guidance and project context
- Tool names, descriptions, and schemas
- Referenced source files
- Conversation history
- Tool calls and tool results
The order matters. Amazon Bedrock and Anthropic process cacheable Claude content as tools → system → messages. Changing tools can invalidate everything after them.
In live testing, an unchanged tool catalog produced an 8,556-token cache read. Reordering tools, changing one description, or changing one schema forced a full new write. Changing only a tool result after an explicit breakpoint preserved the same 8,556-token hit.
The cache-friendly request shape is:
[stable tools]
[stable instructions]
[stable shared context]
[cache boundary]
[changing conversation]
[tool results]
[latest request]Multi-turn agents should append new turns rather than rewrite earlier messages. OpenAI’s Prompt Caching 101 example calls out both requirements: keep tool definitions and their order identical, and append new messages to preserve the cached history prefix.
Cost savings are more reliable than latency claims
Providers advertise substantial latency reductions for long cached prompts. The direction is credible, but the result depends on prompt size, network routing, request admission, model scheduling, and output generation.
My short-prompt measurements did not support a general speedup claim. Terra improved from 3.935 seconds cold to 1.868 seconds warm on one 128,000-token test, while the comparable Luna calls were effectively unchanged.
A study covering more than 500 agent sessions measured 45–80% cost reductions and 13–31% time-to-first-token improvements across providers. It also found that naive full-context caching could make latency worse when dynamic tool content triggered writes without future reuse.
Cache-token accounting is the stronger first metric. Log ordinary input, cache writes, and cache reads. Measure time to first token separately with enough samples to report percentiles.
Beyond coding
The same pattern applies wherever many requests share a large, stable prefix:
- Document review with a fixed policy or contract corpus
- Research sessions grounded in the same source set
- Support agents carrying the same product rules
- Multi-turn workflows with stable instructions and tools
- Long-running analysis against an unchanged reference document
The workload needs repetition within the retention window. A one-shot request receives no benefit from a cache entry that is never read.
So what
Prompt caching changes the economics of repeated context, but the API cannot create repetition on its own.
The practical sequence is:
- Put stable content before dynamic content.
- Keep tools, schemas, and serialization deterministic.
- Place explicit boundaries after content that will be reused.
- Append conversation turns instead of rewriting history.
- Track cache writes as a cost, not only cache reads as a discount.
The deeper implementation problem sits in the agent harness: the component that assembles instructions, tools, context, state, and history into each request.
The open question is dynamic tool discovery. A fixed tool catalog preserves a stable prefix but consumes context and can make tool selection harder. A changing catalog improves relevance but fragments the cache. The best design will need to optimize both, not choose one blindly.