What Prompt Caching Is

Most of what an application sends to a model is the same every time. The instructions, the examples, the background document a question is asked about — all of it is re-sent on every request, and all of it is re-processed on arrival. Prompt caching is the arrangement that stops the re-processing. The system checks if a prompt prefix, up to a specified cache breakpoint, is already cached from a recent query.

The word that matters there is *prefix*. A cache hit is not a search for familiar passages anywhere in the request: it is a match from the very beginning, and it stops at the point the request marks. The cache covers the whole prompt in order — tools, then system, then messages — up to and including the block the request marks. So the order in which an application assembles a request decides what it can cache. Anything placed before the marked block is cached with it; anything that changes early, such as a timestamp at the top of a system prompt, invalidates everything after it.

That gives a plain rule for laying out a request: put what stays the same first, and what changes last. The stable instructions and the long document go above the mark; the user's actual question goes below it.

One boundary is worth knowing before an application relies on this. Prompt caching uses workspace-level isolation. A cache does not follow a prompt across workspace boundaries, so an application split across workspaces gets a separate cache in each rather than one shared between them.