Openai Agent Tool List Change Prompt Cache Proof 2026

Date: 2026-08-12 (UTC), run timestamp 2026-08-12T00:41:30Z
Script: scripts/tool-list-cache-probe.py
Artifact: data/lab-runs/openai-agent-tool-list-change-prompt-cache-proof-2026.openai.json
Endpoint: https://api.openai.com/v1/responses
Model: gpt-5.6-luna
Purpose: measure what happens to OpenAI's prompt cache when an agent's tools array changes between requests, and whether tool_choice.type = "allowed_tools" avoids the damage
Prefix: fixed instructions string plus 20 synthetic function-tool definitions for a fictional retailer's fulfilment desk. All tool names, descriptions and policy text are invented.
Budget guard: every call passed through scripts/proof_budget.py (check_budget before, record_usage after)
Safety boundary: no confidential, customer, credential or private data
Cost of the run: 17 calls, 29,338 total tokens on gpt-5.6-luna

Command

python3 scripts/tool-list-cache-probe.py

Design

Every call uses the same model, the same instructions, the same single user message and the same prompt_cache_key (effloow-toolchange-probe-v1). The only variable is the tools array, or tool_choice.

ID	Variant	Change vs baseline
A	baseline	20 tools, untouched
E	allowed_tools	20 tools untouched; `tool_choice: {type: "allowed_tools", mode: "auto", tools: [3 of them]}`
B	append_tail	21 tools: one new tool appended after all 20 originals
C	remove_middle	19 tools: tool #6 (`escalate_to_finance_queue`) deleted
D	reorder	same 20 tools, positions 3 and 15 swapped
F	edit_description	same 20 tools, one word changed in tool #4's description ("Open" → "Raise")

Baseline re-checks (A4–A7) are interleaved after each mutation so a genuine prefix break can be distinguished from ordinary cache eviction. Calls are ~2 seconds apart.

Raw output

[A1_baseline_cold] tools=20 input=1705 cached=0
[A2_baseline_repeat] tools=20 input=1705 cached=1702
[A3_baseline_repeat] tools=20 input=1705 cached=1702
[E1_allowed_tools] tools=20 input=1705 cached=1702
[E2_allowed_tools] tools=20 input=1705 cached=1702
[A4_baseline_recheck] tools=20 input=1705 cached=1702
[B1_append_tail] tools=21 input=1787 cached=0
[B2_append_tail] tools=21 input=1787 cached=1784
[A5_baseline_recheck] tools=20 input=1705 cached=1702
[C1_remove_middle] tools=19 input=1621 cached=0
[C2_remove_middle] tools=19 input=1621 cached=1618
[A6_baseline_recheck] tools=20 input=1705 cached=1702
[D1_reorder] tools=20 input=1705 cached=0
[D2_reorder] tools=20 input=1705 cached=1702
[F1_edit_description] tools=20 input=1705 cached=0
[F2_edit_description] tools=20 input=1705 cached=1702
[A7_baseline_recheck] tools=20 input=1705 cached=1702

Saved data/lab-runs/openai-agent-tool-list-change-prompt-cache-proof-2026.openai.json

The cold call A1 reported cache_write_tokens: 1702 alongside cached_tokens: 0.

What the run shows

The baseline prefix caches reliably: 1,702 of 1,705 input tokens read from cache on every repeat. The 3-token gap is the uncached tail; OpenAI matches cached prefixes in blocks, not token by token.
Every one of the four tool-array mutations returned cached_tokens: 0. Not a reduced hit, not a partial prefix match. Zero.
Appending a tool after all 20 originals (B) was as destructive as deleting one from the middle (C). The shared leading text bought nothing.
Reordering (D) changed no tool text at all and still zeroed the cache. Input tokens were identical (1,705), so the payload was the same size and the same content in a different order.
Editing one word inside one description (F) zeroed the cache.
allowed_tools (E) preserved the cache completely (1,702 cached), because the tools array itself was untouched.
Every mutation's second call (B2/C2/D2/F2) hit its own cache, so the misses are genuine prefix breaks rather than a broken request or a dead cache.
Baseline re-checks A4–A7 returned 1,702 cached every time, so no mutation evicted the baseline prefix. The baseline and each variant coexist under the same prompt_cache_key.
allowed_tools does not reduce input tokens: E1/E2 billed 1,705 input, identical to baseline. It protects the cache, not the context window.

Limitations

One model (gpt-5.6-luna), one endpoint (Responses API), one account, one day. Not a benchmark and not a cross-model claim.
One prefix size (1,705 input tokens, 20 tools). Behaviour at 50 or 200 tools is [DATA NOT AVAILABLE] from this run.
Implicit caching only. This run did not use prompt_cache_options: {"mode": "explicit"} or prompt_cache_breakpoint; whether an explicit breakpoint placed before the tools block changes the outcome is [DATA NOT AVAILABLE] and worth a follow-up run.
Multi-turn conversations were not tested. Each call is a single user message.
Anthropic's mid-conversation-tool-changes-2026-07-01 beta claims tools can change between turns without cache invalidation on some Claude models. Effloow has no Anthropic API credential configured, so that claim is source-verified only and was NOT tested here.
Prices used in the article come from OpenAI's published pricing page, not from an invoice. Effloow did not measure billed cost.