How to Measure LLM Prompt Cache Savings

Calculate prompt cache savings from token classes and provider rates, then check whether they reduce cost per completed task.

Price the same input tokens twice to estimate prompt cache savings. First use their observed read, write and ordinary rates. Then price all the tokens at the ordinary input rate. The difference estimates what caching saved for the same input. Then compare cost per completed task before and after the change. That second measure includes changes in prompt length, output and retries.

Normalize token fields before multiplying

OpenAI documents cache-read counts under input or prompt token details, depending on the endpoint. When the endpoint reports cache reads and separately priced cache writes as part of total input, subtract both counts before pricing ordinary input. Anthropic exposes separate counters for regular input, cache creation and cache reads. Add the classes to obtain total input before you construct the no-cache estimate. Apply these mappings per endpoint and preserve the raw counters for audit. OpenAI prompt caching and Anthropic prompt caching

Keep output tokens separate. If the provider prices cache writes differently by lifetime, retain each write class and its rate. Store the model, service tier, currency and effective price date. A correct token count multiplied by the wrong tier rate still produces the wrong answer.

Work through one cache lifetime

Suppose a feature sends the same 20,000 cached-prefix tokens four times before expiry. For this hypothetical model, ordinary input costs $2 per million tokens, cache writes $2.50 and cache reads $0.20. Exclude unchanged suffix and output charges from this calculation.

Input class Tokens Estimated charge
First request, cache write 20,000 $0.050
Three subsequent reads 60,000 $0.012
Observed prefix input total 80,000 $0.062
Same input at ordinary rate 80,000 $0.160

The estimated prefix saving is $0.098. With only the first request, there is a $0.010 premium against ordinary input. With one read, the cumulative cost is $0.054 compared with $0.080 for two ordinary inputs, so the write premium has been recovered. Compute break-even separately for each prefix and write lifetime when traffic concentrates in a few large customers.

Compare the metrics by feature and model

A request hit rate gives the same weight to a small and a large prefix. Report cached token share and dollar savings beside it. Track cache writes per read to find low reuse after a model migration or a prompt-template edit. Group by feature, served model and customer tier before comparing periods. A fleet-wide average can improve while the feature with the worst margins becomes more expensive.

Compare cost per completed task for an unchanged workload mix or a controlled rollout. A longer shared prefix can produce a larger calculated saving than before and still increase cost per task. Include the effect of any retries or validation calls the change adds. Keep the comparison window long enough to cover peak and low-traffic periods, since a burst of requests can reuse a prefix that otherwise expires before its next use.

When some customers share a prompt template, assign reads and writes to the requests that incurred them using provider usage fields. Tag benchmark or pre-warm calls with the feature that owns them so they enter the feature's cost even when they produce no customer-visible response. An experiment can separately estimate how cost would change if you moved the write charge into a shared pool.

Reconcile before using savings in margin reports

Group usage by provider account and the same billing period the provider uses. Check for missing terminal usage, rate changes, contract discounts and model fallback. Report unknown usage separately from known cache misses. Test your calculation with one ordinary request, one write and one read, and confirm that every input token belongs to a single billing class. Check that the recomputed amount ties out after aggregation and rounding.

Alectura’s Cost page uses provider-reported cache-read input tokens for prompt-cache analysis. Alectura tracks estimated spend at list prices. Compare the estimate with your provider charges before using it to assess margins. Alectura platform reference and customer-level cost reconciliation

Source documents (2)

Trace a request through Alectura.

Bring a sample prompt and a customer policy. See what reaches the model and which decisions appear in the request record.

Try the quickstart

Keep reading

Set Up Customer-Owned LLM API Keys in AI SaaS ↗Which AI outputs need a label under EU AI Act Article 50? ↗When to Buffer Streaming AI Output for Guardrails ↗