Prompt Tooling Sunset Migration Scanner 2026

• Slug: prompt-tooling-sunset-migration-scanner-2026
• Date: 2026-08-10
• Runtime: Claude (text), local shell
• Evidence level: openai-api-backed-lab (live read-only OpenAI API probes + executed static scanner)
• Artifact: data/lab-runs/prompt-tooling-sunset-migration-scanner-2026.openai.json
• Sandbox: /tmp/effloow-sunset-poc (disposable)

What this run establishes

1. Whether the retiring endpoints announce themselves at runtime (HTTP Deprecation / Sunset headers, RFC 8594).
2. Whether the OpenAI reusable-prompts REST surface is still reachable on an ordinary API key today.
3. That a dated, source-cited static scanner finds these references in a repo, with its real output and its real false positives.

No prompt objects, evals, or agents were created. No model tokens were spent. All OpenAI calls are GET.

Environment

• macOS, Python 3 stdlib only (urllib.request), no SDKs.
• Credential: OPENAI_API_KEY from the repo .env, never printed or logged.
• Anthropic probes were sent unauthenticated — Effloow holds no ANTHROPIC_API_KEY.

Probe 1 — collection endpoints

python3 probe.py

Probe 2 — route existence, and why the Anthropic half is inconclusive

python3 probe2.py

The control returned a byte-identical error to the three real endpoints. Anthropic answers unauthenticated requests with the same 404 regardless of whether the route exists, so this probe proves nothing about the Anthropic endpoints' status. Reported as inconclusive. Their live state before 2026-08-17 is [DATA NOT AVAILABLE] to Effloow — testing it would need an Anthropic key we do not hold.

Probe 3 — OpenAI control, routed vs unrouted

python3 probe3.py

OpenAI distinguishes the two cases: a routed path with a bad id returns a descriptive JSON error, an unrouted path returns an empty 404. /v1/prompts matches the unrouted signature exactly, on both the collection and the item path.

Finding: on this account, the reusable-prompts REST surface is already unreachable — 112 days before the published 2026-11-30 shutdown date — while /v1/evals is fully live and the deprecation page still lists both as shutting down on the same day.

Limitation: this is one API key on one account. The absence could be account-tier gating, a staged rollout, or a surface that was only ever exposed through the SDK's prompt: {id: ...} parameter rather than a public REST collection. Effloow cannot distinguish those from outside, and does not claim OpenAI removed the API early or globally.

Probe finding that holds across all three runs

Not one response carried a Deprecation or Sunset header. RFC 8594 defines Sunset for exactly this purpose. Neither vendor emits it on the surfaces they have publicly scheduled for removal. A gateway, proxy, or APM watching response headers sees nothing. The deadline is only visible in a docs page a human has to remember to read.

Scanner run

Fixture repo: 5 files (fixture/), four containing retiring surfaces and one clean control (src/safe_client.py, inline prompt, no reusable prompt object).

python3 sunset_scan.py fixture --today 2026-08-10

Result: 11 findings, sorted by days-remaining. Both Anthropic hits surfaced at the top with 7 days left. src/safe_client.py was correctly not flagged.

CI gate:

python3 sunset_scan.py fixture --today 2026-08-10 --fail-within-days 30
# FAIL: at least one deadline is within 30 days.
# exit=1

Real false positive

Line 9 of fixture/src/evals.py is def poll_run(eval_id, run_id):. The scanner flagged the function parameter name eval_id, not an API surface. The \beval_id\b rule cannot tell a local variable from a config key without parsing each language's AST. Of the 11 findings, this is 1 — roughly 9%. Kept in the published output rather than tuned away, because narrowing the pattern to config-only contexts would miss genuine eval_id references in YAML. Triage the list; do not treat it as a defect count.

Other known limits

• Line-based regex. A reference split across lines, or a prompt id built by string concatenation at runtime, is missed.
• pmpt_ / eval_ id patterns assume the current prefix convention.
• Scans text files only (extension allowlist). A prompt id stored in a database or a secrets manager is invisible.
• The scanner reads the deadline from its own rule table. It does not re-fetch the vendor pages, so the dates need review if a vendor moves them.

Sources read directly for this run

• https://developers.openai.com/api/docs/deprecations
• https://platform.claude.com/docs/en/release-notes/api (July 17, 2026 entry)

Full source

sunset_scan.py

#!/usr/bin/env python3
"""Prompt-tooling sunset scanner.

Flags source references to prompt/eval tooling that two vendors are retiring.
Rules are dated and cite the vendor page that sets the deadline.
Static only: reads files, calls nothing, needs no API key.
"""
import argparse, json, pathlib, re, sys, datetime

RULES = [
    dict(id="ANTHROPIC-PROMPT-TOOLS", deadline="2026-08-17", vendor="Anthropic",
         severity="critical",
         pattern=r"/v1/experimental/(generate_prompt|improve_prompt|templatize_prompt)",
         what="Experimental prompt tools API",
         action="Endpoint returns an error after removal. Inline the generated prompt or move the step off-platform.",
         source="https://platform.claude.com/docs/en/release-notes/api"),
    dict(id="ANTHROPIC-WORKBENCH", deadline="2026-08-17", vendor="Anthropic",
         severity="high",
         pattern=r"platform\.claude\.com/workbench",
         what="Legacy Workbench link",
         action="Saved prompts, variables and evals do not carry over to the new playground. Export from the banner or Organizational Settings.",
         source="https://platform.claude.com/docs/en/release-notes/api"),
    dict(id="OPENAI-PROMPTS-API", deadline="2026-11-30", vendor="OpenAI",
         severity="critical",
         pattern=r"(api\.openai\.com/v1/prompts|[\"']prompt[\"']\s*:\s*\{[^}]*\bid\b|\bpmpt_[A-Za-z0-9]{6,})",
         what="Reusable prompt object / v1/prompts API",
         action="Move the prompt text into your own repo and pass it inline.",
         source="https://developers.openai.com/api/docs/deprecations"),
    dict(id="OPENAI-EVALS", deadline="2026-11-30", vendor="OpenAI",
         severity="high",
         pattern=r"(api\.openai\.com/v1/evals|\beval_id\b|\beval_[A-Za-z0-9]{5,})",
         what="Evals dashboard and API",
         action="Read-only from 2026-10-31. Export results and move the graders (OpenAI points to Promptfoo).",
         source="https://developers.openai.com/api/docs/deprecations"),
    dict(id="OPENAI-AGENT-BUILDER", deadline="2026-11-30", vendor="OpenAI",
         severity="high",
         pattern=r"agent_builder|agentbuilder",
         what="Agent Builder artifact",
         action="Rebuild on the Agents SDK or ChatGPT Workspace Agents.",
         source="https://developers.openai.com/api/docs/deprecations"),
]
SKIP_DIRS = {".git", "node_modules", "vendor", "dist", "build", ".venv", "__pycache__"}
EXTS = {".py",".ts",".tsx",".js",".jsx",".json",".yaml",".yml",".toml",".env",".md",".rb",".go",".java",".php",".sh"}

def scan(root, today):
    root = pathlib.Path(root)
    findings = []
    for p in sorted(root.rglob("*")):
        if not p.is_file() or any(d in p.parts for d in SKIP_DIRS): continue
        if p.suffix not in EXTS: continue
        try: lines = p.read_text(encoding="utf-8", errors="replace").splitlines()
        except OSError: continue
        for n, line in enumerate(lines, 1):
            for rule in RULES:
                if re.search(rule["pattern"], line):
                    days = (datetime.date.fromisoformat(rule["deadline"]) - today).days
                    findings.append(dict(rule_id=rule["id"], severity=rule["severity"],
                        vendor=rule["vendor"], deadline=rule["deadline"], days_left=days,
                        what=rule["what"], file=str(p.relative_to(root)), line=n,
                        excerpt=line.strip()[:110], action=rule["action"], source=rule["source"]))
    order = {"critical": 0, "high": 1}
    findings.sort(key=lambda f: (f["days_left"], order.get(f["severity"], 9), f["file"], f["line"]))
    return findings

def main():
    ap = argparse.ArgumentParser()
    ap.add_argument("path"); ap.add_argument("--json", action="store_true")
    ap.add_argument("--today", default=datetime.date.today().isoformat())
    ap.add_argument("--fail-within-days", type=int, default=None,
                    help="Exit 1 if any finding is due within N days (CI gate).")
    a = ap.parse_args()
    today = datetime.date.fromisoformat(a.today)
    f = scan(a.path, today)
    if a.json:
        print(json.dumps({"scanned_at": a.today, "finding_count": len(f), "findings": f}, indent=2))
    else:
        if not f:
            print("No retiring prompt-tooling references found."); return 0
        print(f"Prompt-tooling sunset scan — {a.today}\n{len(f)} reference(s) found\n")
        cur = None
        for x in f:
            hdr = f'{x["deadline"]}  ({x["days_left"]} days left)  {x["vendor"]}'
            if hdr != cur:
                cur = hdr; print(f"=== {hdr} ===")
            print(f'  [{x["severity"].upper():8}] {x["file"]}:{x["line"]}  {x["what"]}')
            print(f'             {x["excerpt"]}')
            print(f'             -> {x["action"]}')
        print()
    if a.fail_within_days is not None and any(x["days_left"] <= a.fail_within_days for x in f):
        print(f"FAIL: at least one deadline is within {a.fail_within_days} days.", file=sys.stderr)
        return 1
    return 0

if __name__ == "__main__":
    sys.exit(main())

probe3.py (routed-vs-unrouted control, the decisive probe)

#!/usr/bin/env python3
"""Control probes: what does a definitely-unrouted OpenAI path return vs a live one?"""
import json, os, urllib.request, urllib.error, datetime, pathlib
def key():
    for line in pathlib.Path(".env").read_text().splitlines():
        if line.startswith("OPENAI_API_KEY="):
            return line.split("=",1)[1].strip().strip('"').strip("'")
K = os.environ.get("OPENAI_API_KEY") or key()
def probe(url):
    req = urllib.request.Request(url); req.add_header("Authorization", f"Bearer {K}")
    rec = {"url": url}
    try:
        with urllib.request.urlopen(req, timeout=30) as r:
            rec.update(status=r.status, body_head=r.read(300).decode("utf-8","replace"))
    except urllib.error.HTTPError as e:
        rec.update(status=e.code, body_head=e.read(300).decode("utf-8","replace"))
    return rec
urls = [
  "https://api.openai.com/v1/definitely_not_a_route_effloow",   # control: unrouted
  "https://api.openai.com/v1/prompts",                          # under test
  "https://api.openai.com/v1/prompts/pmpt_effloow_nonexistent_probe",
  "https://api.openai.com/v1/evals",                            # under test (alive)
  "https://api.openai.com/v1/vector_stores/vs_effloow_nonexistent", # control: routed, bad id
]
res=[probe(u) for u in urls]
pathlib.Path("probe3-result.json").write_text(json.dumps(
  {"probed_at_utc": datetime.datetime.now(datetime.timezone.utc).isoformat(), "results": res}, indent=2))
for r in res: print(r["status"], "|", r["url"], "|", repr(r["body_head"][:180]))

Fixture repo

fixture/config/pipeline.yaml
fixture/src/anthropic_tools.py
fixture/src/evals.py
fixture/src/prompts.ts
fixture/src/safe_client.py