Home › Entities › Cache-Aware Tool Registration

Cache-Aware Tool Registration

Description

Claude Code assembles its tool list for every API call using a partition-then-sort strategy that keeps the prompt cache stable when MCP servers connect, disconnect, or change their tool sets. The core invariant: built-in tools form a contiguous, alphabetically sorted prefix, and MCP tools form a separate, alphabetically sorted suffix. The two groups never interleave.

This matters because Anthropic's server-side cache policy (claude_code_system_cache_policy) places a cache breakpoint after the last prefix-matched built-in tool. If a naive flat sort were used, an MCP tool whose name sorted between two built-in tools would interleave into the built-in block, shifting downstream positions and invalidating the entire cached prefix. With partition-sort, MCP tools can only appear after all built-ins, so adding or removing an MCP tool only affects the suffix -- the expensive built-in prefix stays byte-identical and cache-hot.

The mechanism is implemented in two places that must stay in sync:

  1. assembleToolPool() in src/tools.ts -- the primary assembly function used by both the REPL and runAgent. It calls getTools() for built-ins and filterToolsByDenyRules() for MCP tools, then applies the partition-sort via [...builtInTools].sort(byName).concat(allowedMcpTools.sort(byName)). Deduplication (lodash uniqBy('name')) preserves insertion order, so built-ins win on name conflicts.

  2. mergeAndFilterTools() in src/utils/toolPool.ts -- used by the useMergedTools React hook in the REPL path. It re-partitions the merged pool using lodash partition with the isMcpTool predicate, then applies the same [...builtIn.sort(byName), ...mcp.sort(byName)] pattern. The comment explicitly says "Partition-sort for prompt-cache stability (same as assembleToolPool)".

Both functions use (a, b) => a.name.localeCompare(b.name) as the comparator and avoid Array.toSorted() for Node 18 compatibility.

Supporting mechanisms

Three additional systems reinforce the cache stability that partition-sort enables:

Prompt cache break detection

src/services/api/promptCacheBreakDetection.ts monitors for cache breaks across API calls by hashing all request components (system prompt, tool schemas, model, betas, cache_control, etc.) and comparing them turn-over-turn. When a cache break is detected (cache read tokens drop >5% and >2000 tokens), it logs a detailed tengu_prompt_cache_break event with field-level attribution, including per-tool schema hashes to identify exactly which tool's description changed. This telemetry is what drives architectural decisions about cache stability.

Key claims

Relations

Sources