Home › Entities › StreamingToolExecutor

StreamingToolExecutor

Description

The concurrency system inside the query loop that makes Claude Code feel fast. It begins executing tools while the API is still generating a response — the moment a complete tool_use block arrives in the stream, execution starts, even if the model is still generating additional tool calls. Results are buffered and yielded in request order despite out-of-order execution.

File: src/services/tools/StreamingToolExecutor.ts (532 lines)

Concurrency model

Every tool has an isConcurrencySafe flag evaluated per invocation (Tool.ts:402):

Category Tools Concurrent?
Read-only FileReadTool, GlobTool, GrepTool, WebFetchTool, WebSearchTool Yes — always true
Dynamic BashTool Depends — parses the command for write operations (isReadOnly() at BashTool.tsx:434-441)
State-mutating FileEditTool, FileWriteTool, ExitPlanModeTool No — always false
Default Any tool without explicit override No (Tool.ts:759)

The concurrency check (StreamingToolExecutor.ts:129-135):

canExecuteTool(isConcurrencySafe: boolean): boolean {
  const executingTools = this.tools.filter(t => t.status === 'executing')
  return (
    executingTools.length === 0 ||
    (isConcurrencySafe && executingTools.every(t => t.isConcurrencySafe))
  )
}

Rules: read-only tools run in parallel with each other. State-mutating tools run alone. Mixed batches block — a write tool waits for reads to finish, and reads wait for a write to finish.

Max concurrency: 10 (default, configurable via CLAUDE_CODE_MAX_TOOL_USE_CONCURRENCY env var). Implemented with Promise.race() in src/utils/generators.ts:32-72 to maintain exactly N concurrent executions.

Tool partitioning

src/services/tools/toolOrchestration.ts:85-116 — the partitioning algorithm converts tool_use blocks into batches of consecutive same-safety tools:

Input:  [Read, Read, Bash(write), Grep, Bash(write), Read]
Output: [{safe, [Read,Read]}, {unsafe, [Bash]}, {safe, [Grep]}, {unsafe, [Bash]}, {safe, [Read]}]

Safe batches run via runToolsConcurrently() (all tools in parallel up to concurrency cap). Unsafe batches run via runToolsSerially() (one at a time).

Streaming integration

The executor integrates with the query loop across three phases:

Phase 1: Tool registration during streaming (query.ts:838-844) As tool_use blocks arrive in the API stream, each is immediately registered via addTool(). The executor evaluates concurrency safety and calls processQueue() — if conditions allow, the tool starts executing while the API continues generating.

Phase 2: Result collection during streaming (query.ts:847-862) Between API messages, getCompletedResults() is called non-blockingly. It walks the tools array in request order, yielding completed results sequentially. Progress messages are yielded immediately regardless of ordering.

Phase 3: Remaining results after streaming (query.ts:1380-1408) After the API stream completes, getRemainingResults() consumes all remaining tool executions. It loops: process queue → yield completed results → Promise.race() on executing tools + progress promise → repeat until all tools are done.

Result ordering

Execute out-of-order, yield in-order.

Each tracked tool has a status: queued → executing → completed → yielded.

getCompletedResults() (lines 412-440) iterates tools in request order. For each tool: - Yield progress messages immediately (always) - If completed: yield results, mark as yielded - If executing and non-concurrent: stop — don't yield later tools yet

This means if tools [A, B, C] are submitted and B finishes first, B's result is held until A completes and yields. This preserves conversational coherence.

Error handling

Bash-only cascade

Only Bash tool errors cancel sibling tools (lines 355-364). Rationale: Bash commands often have implicit dependency chains (mkdir fails → subsequent commands are pointless). Read-only tools are independent — one WebFetch failure shouldn't cancel a concurrent GrepTool.

When a Bash tool errors: 1. siblingAbortController.abort('sibling_error') fires 2. All running tool processes receive the abort signal and are killed 3. Pending queued tools receive synthetic error messages 4. The parent abort controller is NOT fired — the turn continues

Sibling abort chain

toolUseContext.abortController (parent — query-level)
  └── siblingAbortController (StreamingToolExecutor — Bash errors only)
        └── toolAbortController (per-tool — process-level)

This isolation means a Bash error kills sibling tools but doesn't end the conversation turn.

Synthetic error messages

Three abort reasons generate synthetic tool_result error blocks (lines 153-205): 1. sibling_error — another tool (Bash) errored 2. user_interrupted — user pressed ESC or typed a new message 3. streaming_fallback — API stream fallback triggered; discard pending results

Streaming fallback handling

When the API stream fails and falls back to a new response (query.ts:712-740): 1. Yield tombstone markers for orphaned assistant messages 2. Call streamingToolExecutor.discard() — marks executor as discarded, prevents results from yielding 3. Create a fresh executor for the fallback response

This prevents tool_result messages with stale tool_use_ids from the failed attempt from leaking into the fallback response.

Interrupt behavior

Tools define how they respond to user interrupts (Tool.ts:416): - 'block' (default) — tool continues running; user must wait - 'cancel' — tool is aborted and receives a synthetic error

When the user types a new message while tools are executing (abort reason 'interrupt'), each tool's interrupt behavior determines whether it's cancelled or allowed to finish.

Progress messages

Progress messages are stored separately from results and yielded immediately (lines 367-374, 418-422). They bypass the ordering guarantee — users see real-time "Tool executing... 3s elapsed" feedback even if earlier tools haven't completed yet.

Progress availability wakes up the getRemainingResults() loop via a progressAvailableResolve promise, preventing unnecessary blocking.

Context modifiers

Sequential (state-mutating) tools can return context modifiers that update the ToolUseContext between executions (lines 388-395). This is not supported for concurrent tools — the comment notes this is a known limitation, but no concurrent tools currently use context modifiers.

What depends on it

Design trade-offs

Decision Trade-off
Start tools during streaming Significant latency savings, but tool results may be wasted if the API stream fails/falls back
In-order yielding despite out-of-order execution Coherent conversation, but fast tools blocked behind slow ones
Bash-only error cascade Prevents spurious cancellation of independent tools, but non-Bash dependency chains won't cascade
Max 10 concurrent tools Prevents resource exhaustion, but could bottleneck on I/O-heavy turns with many reads
Per-invocation concurrency check (not per-tool-type) BashTool can run concurrently for read-only commands, but requires command parsing overhead
Separate siblingAbortController Bash errors don't end the turn, but the turn continues with partial results
Context modifiers sequential-only Simple implementation, but limits future concurrent tools that might need context mutation

Key claims

Relations

Sources

src-20260409-6913a0b93c8b, src-20260409-037a8abb6277, source code at src/services/tools/StreamingToolExecutor.ts, src/services/tools/toolOrchestration.ts, src/utils/generators.ts