What Prompt Caching Is

Most of what your application sends a model is the same every time. The same instructions, the same examples, the same long document that every question is asked about — sent again with each request, and read again from the beginning on arrival. You pay for that reading every time.

Prompt caching is the arrangement that stops the re-reading. Prompt caching reuses work when requests share the same prompt prefix. You mark the part of the request that does not change; the platform keeps what it worked out about that part, and the next request that begins the same way starts from there instead of from nothing.

This guide builds the measurement by hand. You write one file, run it once, and read three sets of numbers: the same question sent with no cache, then with the mark that writes the cache, then again while the cache is still warm. The numbers on this page are what that file printed on the day the stamp shows. By the end you will know what to mark, what it saves, and the three mistakes that quietly turn caching off.

By the end of this guide you will

• Turn on prompt caching by adding two lines to a request, and prove from the response's own numbers that it worked.
• Read cached_tokens and cache_write_tokens and say which request paid for a write and which read one for a tenth of the price.
• Lay out a request so the largest possible part of it is cacheable.
• Recognise the three things that quietly empty the cache: a changing line above the mark, a changed tool or model, and a block below the minimum length.

Before you start

Four things. Check each before you write any code.

• Node.js 22 or newer. Run node --version. It prints a version such as v24.14.1, which is what the run on this page used. If it prints an error, install Node.js from the official site.
• An OpenAI API key. Create one on the API keys page of your dashboard and copy it somewhere safe. It never goes into the code.
• A folder with the SDK installed. Run mkdir caching-demo, then cd caching-demo, then npm init -y, then npm install openai@7.15.0 — the version this run used.
• The key in your environment, in the same terminal window you will run the file from. On Windows PowerShell: $env:OPENAI_API_KEY = "your key here". On macOS or Linux: export OPENAI_API_KEY="your key here". The SDK reads that variable itself, which is why the code never mentions the key.

The model here is gpt-5.6-luna. GPT-5.6 Luna, model id gpt-5.6-luna, is the GPT-5.6 model OpenAI describes as optimized for cost-sensitive workloads. Any GPT-5.6-or-later model behaves the same way; the numbers you compare against are Luna's.

Run the file once. The second example writes the cache and the third reads it. Run the whole file again within half an hour and the second example finds the entry already there, so it reads instead of writing — the cache working exactly as it should, and not the output Step 2 describes. "The one rule" says why half an hour.

Step 1 — Send the request with no cache, and look at the numbers

Create a file named caching.mjs and put this in it:

import OpenAI from 'openai';

const client = new OpenAI(); // reads OPENAI_API_KEY from the environment; the key is never in the code

// A "document" long enough to be worth caching: one sentence, repeated. A real application would
// send a real document; the numbers behave the same way.
const document = 'The quarterly report covers revenue, costs, and headcount for each region. '.repeat(160);
const question = 'What does the report cover? Answer in one sentence.';

// The four numbers that tell the whole story, and nothing else from `usage`.
const usageOf = (response) => {
  const { input_tokens, input_tokens_details, output_tokens } = response.usage;
  return { input_tokens, cached_tokens: input_tokens_details.cached_tokens, cache_write_tokens: input_tokens_details.cache_write_tokens, output_tokens };
};

const noCache = await client.responses.create({
  model: 'gpt-5.6-luna',
  prompt_cache_options: { mode: 'explicit' }, // only breakpoints we place count — and here we place none
  input: [
    { role: 'developer', content: [{ type: 'input_text', text: `Answer from this document only:\n${document}` }] },
    { role: 'user', content: question },
  ],
});

console.log(usageOf(noCache));

{
  input_tokens: 2428,
  cached_tokens: 0,
  cache_write_tokens: 0,
  output_tokens: 18
}

Run it with node caching.mjs. Four numbers come back. Read them one at a time; the rest of the guide is these four numbers changing.

• input_tokens: 2428 — everything the model read: the document, the question, and what the platform puts around them. A token is the unit of both counting and billing; an ordinary English word is one token or a little more.
• cached_tokens: 0 — nothing was read from a cache. Expected: this request marked nothing.
• cache_write_tokens: 0— and nothing was written to one. This is the line that proves the request really did opt out. Setting prompt_cache_options.mode to explicit uses only developer-selected breakpoints, each marked by adding prompt_cache_breakpoint to a supported content block inside an input message; when no explicit breakpoints are placed, the request does not use prompt caching or create cache writes. That is why the file setsmode: 'explicit' and marks nothing: it makes "no caching" a request you wrote, not a default you hope for.
• output_tokens: 18 — the answer's own length, which caching never touches.

Two details in the code matter later. The document is one sentence repeated 160 times, which is enough to clear the minimum this guide meets in Step 2. And it sits in a developer message as an input_text block, not in a top-level instructionsfield, because that is where a mark is allowed: each request can create up to four cache writes, and top-level instructions cannot contain an explicit breakpoint: reusable developer instructions go in an input_text block inside a developer message.

Run the file a second time if you like. The same 2,428 tokens are read again and charged again, though the document has not changed by a character. That is the problem.

Step 2 — Add the mark, and watch the cache be written

Add this below what you wrote in Step 1, in the same file:

const cacheWrite = await client.responses.create({
  model: 'gpt-5.6-luna',
  prompt_cache_options: { mode: 'explicit' },
  input: [
    {
      role: 'developer',
      content: [
        {
          type: 'input_text',
          text: `Answer from this document only:\n${document}`,
          prompt_cache_breakpoint: { mode: 'explicit' }, // the one new line: cache everything up to and including this block
        },
      ],
    },
    { role: 'user', content: question },
  ],
});

console.log(usageOf(cacheWrite));

{
  input_tokens: 2428,
  cached_tokens: 0,
  cache_write_tokens: 2410,
  output_tokens: 18
}

Run node caching.mjs again. Step 1's numbers print first, then these. One line is new and one has changed:

• cache_write_tokens: 2410 — the platform read the marked block and kept what it worked out. Compare with input_tokens: 2428: nearly the whole request is now in the cache, all but the eighteen-odd tokens of the question that sits below the mark. What was kept is not the text. Prompt caching preserves the model's intermediate state for a reusable prefix — the unchanged tokens at the beginning of a prompt — and reuses it when a later request has the same prefix, while still processing any new input.
• cached_tokens: 0 — nothing was read from a cache, because there was nothing there yet. This request did not save you anything; it cost a little more.For GPT-5.6 and later, cache writes cost 1.25 times the standard uncached input-token rate and subsequent reads cost 0.1 times that rate; writing a prefix once and fully reusing it once costs 1.35 times its ordinary input cost, against 2 times for processing it twice without caching.

The only change to the code is the prompt_cache_breakpoint line on the document's block. That is the whole interface: a mark that says "everything up to and including this is the stable part". Nothing else moved.

Step 3 — Send it again, and watch the cache be read

Add this below Step 2's code:

// The same request again, sent within the cache's lifetime of the one above.
const cacheRead = await client.responses.create({
  model: 'gpt-5.6-luna',
  prompt_cache_options: { mode: 'explicit' },
  input: [
    {
      role: 'developer',
      content: [
        {
          type: 'input_text',
          text: `Answer from this document only:\n${document}`,
          prompt_cache_breakpoint: { mode: 'explicit' },
        },
      ],
    },
    { role: 'user', content: question },
  ],
});

console.log(usageOf(cacheRead));

{
  input_tokens: 2428,
  cached_tokens: 2410,
  cache_write_tokens: 0,
  output_tokens: 18
}

Run node caching.mjs a third time — the third example goes out seconds after the second, well inside the cache's life. This is the number the whole guide is for:

• cached_tokens: 2410 — the same 2,410 tokens, read from the cache instead of processed again, at a tenth of the rate. The read also renews the entry.
• cache_write_tokens: 0 — nothing new was written; the prefix was already there, unchanged to the character.
• input_tokens: 2428 — unchanged, and worth understanding: this figure is what the model read, not what you were charged at full price. Of those 2,428 tokens, 2,410 were cache reads and only the remainder was ordinary input.

Now put the three runs side by side and the arithmetic is plain. The first request paid full price for 2,428 tokens. The second paid 1.25× for 2,410 of them, and bought an entry. The third paid 0.1× for those 2,410. Every request after that, while the entry lives, pays the same tenth. Across ten requests, one write and nine full reads cost 2.15 times the ordinary input cost, compared with 10 times without caching. The three benefits are avoiding recalculation of a prefix the model has already processed, paying the reduced cached-input rate for reused tokens — discounted up to 90% — and reducing the time spent processing input before the response starts. The saving is not a small percentage; on an application that re-sends a long stable prefix, it is most of the input bill.

The one rule

The stable part goes first, the changing part goes last, and the mark goes between them. Cache reuse requires the entire rendered prefix to match; if content or a relevant setting changes before a breakpoint, the prefix after that change cannot match the existing cache entry. A prefix is matched from the very beginning, character for character — not searched for familiar passages in the middle.

So the order in which you assemble a request decides how much of it can be cached:

• Everything that is the same on every request — instructions, examples, the long document, the tool definitions — goes above the mark.
• Everything that changes — the user's question, a timestamp, today's data — goes below it.
• One character changed above the mark makes a different prefix, and the next request pays in full.

Two limits belong to the same rule. A block must be long enough to cache: a prompt prefix must meet the model's minimum cacheable token length before it can be cached, and tokens in the OpenAI-provided hidden system content do not count toward that minimum. The minimum cacheable prompt length is 1,024 tokens for GPT-5.6 and later, and varies by request settings for earlier models.And an entry does not live forever: prompt_cache_options.ttl controls the minimum cache lifetime; the only supported value, 30m, is also the default, and a cached prefix remains eligible for reuse for 30 minutes after its most recent write or reuse.

If you would rather not place the mark yourself, the platform will place one. In implicit mode OpenAI places a breakpoint at the end of the latest eligible message, and eligible messages are user messages, the last tool response in a consecutive group, and the last developer message in the initial consecutive group. That is the default, and it suits a growing conversation, where the newest message is exactly the boundary you want. It suits a fixed document with a changing question less well, which is the case this guide measured — hencemode: 'explicit' and a mark you control.

When to use it, and when not

Terms that came up

• Prompt — everything you send in one request: the instructions, the document, the tools and the question.
• Token — the unit of counting and billing; an ordinary English word is one token or a little more.
• Prefix — the beginning of the request up to a given point. The cache matches prefixes, from the first character.
• Cache breakpoint — the mark that says where the stable prefix ends; in this guide, prompt_cache_breakpoint on a content block.
• Explicit mode — prompt_cache_options.mode: 'explicit': only the marks you place count, and with none, no caching happens.
• Implicit mode — the default, where the platform places one breakpoint at the end of the latest eligible message.
• Cache write — the first request with a given prefix, which stores it at 1.25× the ordinary input rate; cache_write_tokens counts it.
• Cache read — a later request that matches, charged 0.1× for the matched tokens; cached_tokens counts it.
• TTL — how long an entry stays eligible for reuse; 30 minutes here, renewed by each reuse.
• Developer message — the role that carries your application's instructions; a mark may sit on its blocks, unlike top-level instructions.
• usage — the object on the response carrying the four numbers this guide reads.

In short

Prompt caching stores what the model worked out about the unchanging beginning of your request, and reuses it on the next request that begins the same way, at a tenth of the price. Mark the end of that stable part, keep everything that changes below the mark, and make sure the part above it clears the model's minimum length. Then read cached_tokens on the second request: if it is not close to your prefix's size, something above the mark is moving.

What changed recently

No source of this guide has changed since the last check.