Prompt caching

Many providers cache the beginning of a prompt: when the next request starts with the same tokens, those tokens are read from the cache at a lower price.

Automatic caching

Most providers cache long prefixes on their own. To benefit:

• put what does not change (system prompt, instructions, documents, tool definitions) at the beginning and what changes at the end;
• send repeated requests within a few minutes of each other: caches expire quickly.

Explicit cache breakpoints

Some models cache only the parts you mark. Add cache_control to a text content part; the router passes it to the provider as is:

{
  "role": "system",
  "content": [
    { "type": "text", "text": "You are a support agent. Here is the product manual: …", "cache_control": { "type": "ephemeral" } }
  ]
}

Writing to such a cache may cost more than ordinary input; reading costs less. A breakpoint helps only for prompts you repeat.

See what was cached

• usage.prompt_tokens_details.cached_tokens: input tokens read from the cache.
• GET /api/v1/models gives the cache read price per token in pricing.input_cache_read.
• The cabinet logs show cached tokens of every call.

usage.cost already accounts for the cache discount.