04 / Policy · Analysis
Prompt caching cuts the input rate for reused system prompts
When requests repeat the same system prompt, the lower rate applies to tokens reused from the cache.

aiunplugged published an explainer on DEV Community that put cached input rates at 5% to 50% of standard input rates.
Without caching, the article said, the same system prompt is billed at the standard rate each time it is sent.
The article identified repeated input at the start of a request as eligible for caching. Providers bill by the token, a unit of input the model reads.
Instructions eligible for caching
The article described prompt caching as a feature of the application programming interface (API), which software uses to call the model.
It gave the system prompt as an example of input that can recur at the start of a request. A system prompt gives the model instructions that can carry across requests.
How tokens add up across requests
The article used the example of a 20,000-token system prompt sent 100 times a day. Without caching, that adds up to 2 million tokens a day billed at the standard input rate.
Under the pricing described in the article, the lower rate applies to the tokens the API reuses from the cache when that prompt is sent again.