Effloow Lab OpenAI API Run: openai-apps-sdk-internal-mcp-app-readiness-2026

• Date: 2026-06-09T00:32:14.683960+00:00
• Model: gpt-5.5-2026-04-23
• Purpose: apps-sdk-synthetic-readiness-check
• Request ID: req_28448bc4f20d4e488c4a5ba7e520e3e0
• Usage: {"input_tokens": 270, "input_tokens_details": {"cached_tokens": 0}, "output_tokens": 1400, "output_tokens_details": {"reasoning_tokens": 192}, "total_tokens": 1670}
• Prompt SHA-256: 686d5fcf72f926baea103cde1541987284ed27f0228c1530890cd31729682b5a
• Safety boundary: no confidential, customer, credential, or private data should be included in this run.

Prompt Excerpt

Evaluate this synthetic, non-confidential readiness plan for a ChatGPT Apps SDK internal MCP app. Do not invent facts about OpenAI availability, pricing, app approval, or real workspace access. Return: (1) likely readiness gaps, (2) tests to run before publishing, (3) governance questions for an admin, and (4) what evidence must remain DATA NOT AVAILABLE until a real ChatGPT developer-mode test is performed.

Synthetic plan:
- App purpose: create and update support tickets from ChatGPT for a SaaS operations team.
- MCP endpoint: HTTPS /mcp hosted by the vendor.
- Tools: search_tickets(read), get_ticket(read), create_ticket(write), update_ticket_status(write).
- Auth: OAuth 2.1 with PKCE, per-user scopes tickets:read and tickets:write.
- UI: embedded ticket detail component.
- Intended rollout: internal pilot for Business or Enterprise workspace admins.
- Current evidence: source review only; no real ChatGPT Developer Mode connection, no OAuth tenant test, no tool scan, no user test, no app submission.

Output

## Verified observations from the synthetic plan

Based only on the provided plan:

- The app is intended to let users create and update SaaS support tickets from ChatGPT.
- The MCP endpoint is planned as `HTTPS /mcp` hosted by the vendor.
- Four tools are planned:
  - `search_tickets` — read
  - `get_ticket` — read
  - `create_ticket` — write
  - `update_ticket_status` — write
- Authentication is planned as OAuth 2.1 with PKCE and per-user scopes:
  - `tickets:read`
  - `tickets:write`
- The app includes an embedded ticket detail component.
- The target rollout is an internal pilot for Business or Enterprise workspace admins.
- Current evidence is limited to source review.
- No real ChatGPT Developer Mode connection, OAuth tenant test, tool scan, user test, or app submission has been performed.

---

# 1. Likely readiness gaps

## A. Integration readiness gaps

1. **No real ChatGPT Developer Mode connection has been tested**
   - The plan has not yet shown that ChatGPT can connect to the `/mcp` endpoint.
   - Tool discovery, invocation, auth handoff, and embedded UI behavior remain unverified.

2. **No tool scan or schema validation evidence**
   - It is not yet verified that the declared tools are discoverable, correctly described, and usable by ChatGPT.
   - Input and output schemas need validation against the actual MCP implementation.

3. **No end-to-end MCP session evidence**
   - Source review alone does not prove that a complete user flow works:
     - User asks to search tickets.
     - ChatGPT calls `search_tickets`.
     - User opens a ticket.
     - ChatGPT calls `get_ticket`.
     - User requests a status change.
     - ChatGPT calls `update_ticket_status`.
     - UI reflects the result.

4. **Embedded component unverified**
   - The ticket detail component has not been tested inside ChatGPT.
   - Rendering, data hydration, state updates, navigation behavior, and error handling are unproven.

---

## B. Authentication and authorization gaps

1. **OAuth tenant flow untested**
   - The plan states OAuth 2.1 with PKCE, but no real tenant test has been performed.
   - Unknowns include:
     - Redirect URI correctness.
     - Token exchange behavior.
     - Refresh behavior, if applicable.
     - Scope consent behavior.
     - Error states for denied consent or expired sessions.

2. **Scope separation needs proof**
   - The plan has separate `tickets:read` and `tickets:write` scopes, which is good design, but enforcement is not proven.
   - Need to verify that users with only `tickets:read` cannot call `create_ticket` or `update_ticket_status`.

3. **Per-user authorization not yet proven**
   - The plan says per-user scopes, but source review does not prove that tool calls execute under the correct user identity.
   - Need evidence that user A cannot access or modify user B’s tickets unless allowed by the SaaS authorization model.

4. **Workspace/admin installation behavior unknown**
   - Intended rollout is to Business or Enterprise workspace admins, but no real admin install or developer-mode test has been performed.
   - Availability, admin controls, approval flow, and workspace behavior must remain unverified until tested.

---

## C. Write-action safety gaps

1. **No confirmation policy described for writes**
   - `create_ticket` and `update_ticket_status` are write tools.
   - The plan should define when ChatGPT must ask for confirmation before creating or changing a ticket.

2. **No idempotency strategy described**
   - `create_ticket` could accidentally create duplicate tickets if a request is retried.
   - The plan should specify idempotency keys or duplicate detection behavior.

3. **No rollback or correction process described**
   - If ChatGPT updates the wrong ticket status, the plan does not say how the user or admin can detect and revert it.

4. **No audit event requirements described**
   - Write actions should produce auditable records:
     - Who initiated the action.
     - Which tool was called.
    
...[truncated]

Limitations

This API run is a bounded lab check. It is not a production benchmark, user study, or proof that an external product works in a real customer environment.