Streaming Tool Execution 40% Speedup
- Entity ID:
ent-20260419-576401eeb8a1 - Type:
concept - Scope:
shared - Status:
active - Aliases: streaming tool parallelism, 40% speed advantage
Description
Performance pattern where StreamingToolExecutor launches tools while the LLM is still generating. Typical 5-tool turn completes in ~18s instead of ~30s. Implementation watches for tool_use blocks during streaming because stop_reason==='tool_use' is documented as unreliable (line 554-555 comment).
Key claims
- stop_reason==='tool_use' is unreliable
- Streaming tools yield ~40% speedup
- StreamingToolExecutor uses sibling abort + progress-available signal for concurrent reads
Relations
- Streaming Tool Execution 40% Speedup --[derived_from]--> StreamingToolExecutor