Your chatbot repeats the same 4,000-token system prompt on every turn. Your RAG pipeline re-sends the same retrieved context for every follow-up question. Your agent re-sends the same 20 tool definitions on every step. Every one of those tokens is billed, every single time, and most of them have not changed since the conversation started.
Prompt caching is the boring, obvious fix. Mark the stable prefix once, pay a small write surcharge, then read it back for 10% of the input price on every subsequent request that matches. On a 5,000-token system prompt over 100 turns, the math goes from $1.50 to $0.18 on Claude Sonnet 4.5. That is a real saving you can ship this week.
This article shows the mechanism, the gotchas, and a working Python implementation for three common shapes: multi-turn chat, RAG retrieval, and a tool-using agent.
Prerequisites
- Python 3.10+
pip install anthropic(SDK is at 0.116.x as of writing)- An
ANTHROPIC_API_KEYin your environment - Optional:
pip install openaifor the comparison section
How Pricing Actually Works
The trick that catches most people: there is no separate "cache mode." Caching is just a price modifier on input tokens. Every input token falls into one of three buckets, and the response tells you which.
| Bucket | Anthropic (5-min TTL) | Anthropic (1-hour TTL) | OpenAI (auto) |
|---|---|---|---|
| Cache write | 1.25x base input | 2.0x base input | free (no surcharge) |
| Cache read | 0.1x base input | 0.1x base input | 0.5x base input |
| Regular input | 1.0x base input | 1.0x base input | 1.0x base input |
Source: Anthropic prompt caching pricing and OpenAI prompt caching guide.
The break-even point is the second request. If you only ever send a prompt once, the 1.25x write surcharge makes you pay more than the no-cache baseline. From the second matching request onward you are ahead. The longer the prefix, the more you save per request.
For Claude Sonnet 4.5 today, the numbers look like this (per million tokens):
- Base input: $3.00
- 5-min cache write: $3.75
- 1-hour cache write: $6.00
- Cache read: $0.30
- Output: $15.00
For a 5,000-token cached prefix over 100 requests, total cost on Sonnet 4.5:
- No cache: 5,000 x 100 / 1,000,000 x $3 = $1.50
- With cache: (5,000 write at $3.75) + (5,000 x 99 read at $0.30) = $0.0188 + $0.1485 = $0.1673
That is an 89% reduction. The exact number depends on how many cache reads you get, but the order of magnitude is what matters.
The Two Flavors of Anthropic Caching
Anthropic ships both automatic and explicit caching. Pick based on how dynamic your prompt is.
Automatic Caching (Multi-Turn Chat)
Drop a single cache_control at the top level of the request. The API moves the cache breakpoint to the last cacheable block on each call, so the cache grows with your conversation. This is the right choice when the only thing changing is the message list.
import anthropic
client = anthropic.Anthropic()
def chat(messages: list[dict]) -> str:
response = client.messages.create(
model="claude-sonnet-4-5",
max_tokens=1024,
# One line. The system prompt and earlier messages get cached automatically.
cache_control={"type": "ephemeral"},
system="You are a customer support agent for an Indonesian fintech. "
"Reply in Bahasa Indonesia unless the user writes in English. "
"Be concise. Never invent account balances.", # ~50 tokens
messages=messages,
)
print(response.usage) # Check the cache_creation_input_tokens and cache_read_input_tokens
return response.content[0].text
Run the function twice with the same system prompt and a growing message list. On the second call, the usage block will show cache_read_input_tokens covering the system prompt and any earlier messages that match the prior request.
Explicit Caching (Stable Sections, Varying Sections)
When parts of your prompt change at different rates, automatic caching breaks. RAG is the textbook case: the system prompt and tool definitions are stable, the retrieved context changes per query, and the user message changes per turn. Three different cadences.
Mark each section with its own cache_control block:
import anthropic
client = anthropic.Anthropic()
SYSTEM = [
{
"type": "text",
"text": "You answer questions about a Postgres schema. Be terse. Use SQL when needed.",
"cache_control": {"type": "ephemeral"}, # Breakpoint 1
}
]
TOOLS = [
{
"name": "query_database",
"description": "Run a read-only SQL query against the warehouse.",
"input_schema": {
"type": "object",
"properties": {
"sql": {"type": "string", "description": "A SELECT statement"},
},
"required": ["sql"],
},
# Tools cache as a block on their own.
"cache_control": {"type": "ephemeral"}, # Breakpoint 2
}
]
def ask(question: str, retrieved_chunks: list[str]) -> str:
context = "\n\n".join(retrieved_chunks)
response = client.messages.create(
model="claude-sonnet-4-5",
max_tokens=1024,
system=SYSTEM,
tools=TOOLS,
messages=[{
"role": "user",
"content": (
f"Use the context below to answer.\n\n"
f"<context>\n{context}\n</context>\n\n"
f"Question: {question}"
),
"cache_control": {"type": "ephemeral"}, # Breakpoint 3
}],
)
return response.content[0].text
Three breakpoints, three independent cache entries. The system prompt and tools stay cached across every request. The retrieved context gets a new entry when the chunks change. The user message itself caches so the next turn can read it back.
The Lookback Gotcha
Anthropic's cache uses a 20-block lookback window. If your breakpoint moves further than 20 blocks past the last write, the cache miss happens silently. You pay the full input price and probably never notice unless you check the usage object.
This bites multi-turn conversations. Turn 1 writes at block 10. Turn 2 adds 5 blocks and writes at block 15. Turn 3 adds 30 blocks. The breakpoint is now at block 45. The lookback walks back 20 positions, finds nothing, and you pay for a fresh write at 1.25x on the entire prefix.
The fix is to add a second, intermediate breakpoint early in the conversation so an entry is available closer to where the next one will be. Up to 4 breakpoints per request are allowed.
# In a long-running agent, place a "waypoint" breakpoint.
# This entry stays alive even if the main breakpoint moves past it.
{
"role": "user",
"content": "[checkpoint: still on the same task, summarize current state in one line]",
"cache_control": {"type": "ephemeral"},
}
When to Choose 5-Minute vs 1-Hour TTL
The default is 5 minutes. It works for chat and for any synchronous request flow. Choose 1-hour when your traffic is bursty but the prefix stays valid for longer.
cache_control={"type": "ephemeral", "ttl": "1h"}
The 1-hour write costs 2x base (double the 5-minute write), but the reads are the same 0.1x. If you actually hold the cache alive for the full hour, the per-request cost drops to about 10% of base. If you evict the cache every 6 minutes by changing the prefix, the 1-hour write premium is wasted money.
A simple rule: if your application can naturally re-read the same prefix every few minutes, 5-minute is fine. If you build a batch job that reuses a system prompt across 1,000 requests over an hour, 1-hour wins. If you do not know yet, start with 5-minute and measure.
Prewarming the Cache (Latency Win)
Caches only become available after the first response begins. If you have a known-cold-start path (a daily report, a scheduled job, the first user of the day), prewarm it with a zero-token request:
def prewarm_cache() -> None:
"""Call at startup or on a cron. Costs only the cache write surcharge."""
client.messages.create(
model="claude-sonnet-4-5",
max_tokens=0,
system=[
{
"type": "text",
"text": "You are a senior staff engineer reviewing pull requests...",
"cache_control": {"type": "ephemeral"},
}
],
messages=[{"role": "user", "content": "warmup"}],
)
# stop_reason will be "max_tokens" and content will be [].
# The cache entry is now in place for the next real request.
The first real request after this gets a cache hit, which also cuts time-to-first-token. Anthropic reports up to 85% latency reduction on cache reads for long prompts. That matters for streaming UIs.
Minimum Token Thresholds
Caching is not free to set up. Each platform enforces a minimum:
- Anthropic Claude: 1,024 tokens for Claude Sonnet 4.5, 2,048 tokens for Opus 4.x
- OpenAI gpt-4o and newer: 1,024 tokens
- Google Gemini: a few thousand tokens (varies by model)
If your prefix is below the threshold, the API silently skips caching and you pay the regular input price. There is no error. Always check cache_read_input_tokens in the response to confirm caching is actually happening.
How OpenAI and Google Compare
OpenAI's prompt caching is fully automatic. No cache_control parameter, no breakpoints. Requests with matching prefixes (hashed over the first 256 tokens) get cached for 5 to 10 minutes. The pricing is simpler: 0.5x for cache reads, no write surcharge. The catch is less control. You cannot pin a section, you cannot extend the TTL, and you do not get fine-grained usage reporting in the same way.
Google Gemini's implicit caching works similarly: no explicit marker, automatic prefix matching, free reads, and a much longer default TTL. Explicit caching is also available via a cachedContent resource if you need control.
If you are starting a new project and do not need precise control, OpenAI's automatic path is the lowest-friction option. If you have a complex prompt with sections that change at different rates, Anthropic's explicit breakpoints give you the control.
Verifying Your Cache Is Hitting
The single most important habit: log the usage object on every request. Caching failures are silent.
def log_cache_stats(usage):
cached = usage.cache_read_input_tokens or 0
written = usage.cache_creation_input_tokens or 0
fresh = usage.input_tokens or 0
total = cached + written + fresh
if total == 0:
return
hit_rate = cached / total
print(f"cache hit rate: {hit_rate:.1%} ({cached} read, {written} written, {fresh} fresh)")
if hit_rate < 0.5 and total > 2000:
print("WARNING: low cache hit rate, check your breakpoint placement")
A healthy setup with explicit breakpoints should hit above 80% on the cached prefix after the first request. If you see 0%, the most common causes are: breakpoint placed on a block that changes every request (timestamp in the prefix), prefix hash differs because of tool ordering, or the prefix is below the minimum token threshold.
A Working RAG Example
Putting the pieces together for a RAG pipeline:
import anthropic
client = anthropic.Anthropic()
# Stable across all queries: system instructions + tool definitions.
SYSTEM_PROMPT = [
{
"type": "text",
"text": (
"You answer questions about company policies using the provided context. "
"Quote the source chunk when possible. If the answer is not in the context, "
"say 'not found in the provided context' and do not guess."
),
"cache_control": {"type": "ephemeral"},
}
]
TOOL_DEFINITIONS = [
{
"name": "search_policies",
"description": "Search the policy database for relevant chunks.",
"input_schema": {
"type": "object",
"properties": {"query": {"type": "string"}},
"required": ["query"],
},
"cache_control": {"type": "ephemeral"},
}
]
# Per-query: retrieved context + user question.
def answer(question: str, chunks: list[str]) -> str:
context_text = "\n\n".join(f"[{i+1}] {c}" for i, c in enumerate(chunks))
response = client.messages.create(
model="claude-sonnet-4-5",
max_tokens=800,
system=SYSTEM_PROMPT,
tools=TOOL_DEFINITIONS,
messages=[{
"role": "user",
"content": (
f"Context:\n{context_text}\n\nQuestion: {question}"
),
# This block changes per query, but it caches the previous turn's
# context so the next follow-up question can read it back.
"cache_control": {"type": "ephemeral"},
}],
)
log_cache_stats(response.usage)
return response.content[0].text
Run answer("What is the leave policy?", [chunk1, chunk2, chunk3]) once. The system prompt and tool definitions cache. Run it again with the same system prompt and tools but different chunks. The system and tools come from cache. Run it again with the same chunks and a follow-up question. The chunks come from cache.
Common Mistakes
Putting the breakpoint on the user message that includes a timestamp. The hash includes the timestamp, so the prefix never matches. Move the breakpoint to the last stable block instead.
Adding more breakpoints than you need. Four is the maximum, but each breakpoint you add does not increase your cost. It only gives you more control. Use them when sections genuinely change at different rates, not "just in case."
Forgetting that the cache is per-region and per-workspace. A cache created in your staging environment will not be available in production. If you run the same workload in multiple regions, each region has its own cache.
Assuming the cache survives a model change. Switching from Sonnet to Opus evicts the cache. The new model has its own cache namespace.
Ignoring output token costs. Caching only helps with input. If your prompt is small but your outputs are large (long-form generation, agentic loops with verbose tool calls), the input savings are a rounding error on the bill. Profile first.
Where to Go Next
- Anthropic prompt caching docs for the full reference, including the cache invalidation rules and ZDR eligibility
- OpenAI prompt caching for the automatic, no-code-change approach
- The Anthropic pricing page for current per-model rates (these change, do not hardcode them)
If you want to combine caching with the tool-heavy agent pattern from earlier articles, add cache_control to the system prompt and the tools array. The tool definitions rarely change, so they cache well. The message history and tool results are the parts that move.