Docs

Prompt Caching

An agent loop re-sends its whole history on every step, so a long session pays for the same tokens again and again. NanoCodana marks the stable parts of the prompt as cacheable automatically — on Anthropic models, re-read input is billed at a fraction of the full price, cutting input cost by up to ~90% on long sessions.

It's on by default and needs zero configuration. When the model is an Anthropic provider instance (or a Claude model routed through OpenRouter), the agent places cacheControl breakpoints on the system prompt + tool definitions and on the tail of the conversation, so each loop step re-reads the growing history from cache instead of re-paying full price. You can watch it work: result.totalUsage reports cached input tokens separately.

const agent = NodeAgent({
  model: anthropic('claude-sonnet-5'),
  workingDirectory: process.cwd(),
  // promptCaching: true is the default — nothing to do.
})

// Opt out to manage cacheControl yourself:
const manual = NodeAgent({ model, promptCaching: false })

What gets detected#

Auto: Anthropic provider instances (anthropic('claude-…')) and Claude models via OpenRouter.
Not auto: gateway model strings (model: 'anthropic/claude-…') and Claude via Bedrock or Vertex — wrap those with the exported promptCachingMiddleware yourself if the route forwards Anthropic provider options.
Other providers: no-op. OpenAI, Google, and DeepSeek cache automatically server-side; there's nothing to mark.

manual wrapping (gateway / Bedrock / Vertex)

import { wrapLanguageModel } from 'ai'
import { promptCachingMiddleware } from '@nanocodana/core'

const model = wrapLanguageModel({ model: bedrockClaude, middleware: promptCachingMiddleware })

Prompts or tools that already carry a manual cache breakpoint (in the anthropic or openrouter namespace) are left untouched, with or without auto-caching — your explicit placement always wins.