fabro/test/twin/openai/docs/compatibility-matrix.md
Bryan Helmkamp 5b7eabee8f Add twin test mode for OpenAI E2E tests
Integrate twin-openai (fake OpenAI server) into the workspace and wire
it into the e2e_test macro so OpenAI tests can run without real API
credentials. The twin server starts in-process via OnceLock on first use
and provides per-test isolation through bearer-token namespacing.

Changes:
- Add Twin as default TestMode, replacing Off (gating now via #[ignore])
- Extend #[e2e_test] macro with `twin` requirement for twin-only,
  live-only, and dual-mode (twin + live) test gating
- Add e2e_openai!() macro returning (base_url, api_key)
- Convert openai_complete and openai_gpt_5_3_codex_complete to dual-mode
- Add new openai_server_error twin-only test with scripted 500 error
- Standardize axum 0.8 as workspace dependency across all crates
- Relax twin-openai ResponsesRequest to accept unknown fields via flatten

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 20:36:48 -04:00

60 lines
1.3 KiB
Markdown

# Compatibility Matrix
Supported endpoints:
- `GET /healthz`
- `POST /v1/responses`
- `POST /v1/chat/completions`
- `POST /__admin/scenarios`
- `POST /__admin/reset`
- `GET /__admin/requests`
State isolation:
- `/v1/*` request state is scoped by bearer token
- admin routes may include the same bearer token to target that namespace
- admin routes without auth operate on the global namespace
Supported `/v1/responses` fields:
- bearer auth
- `stream`
- `metadata`
- `stop`
- `previous_response_id`
- `reasoning`
- `text.format.type = text | json_object | json_schema`
- image inputs in `input[*].content[*].type = input_image`
- scripted tool calls and continuation input items
Supported `/v1/chat/completions` fields:
- bearer auth
- `stream`
- `tools`
- `tool_choice`
- `response_format.type = text | json_object | json_schema`
- `stop`
- reasoning-bearing assistant content
Structured output subset:
- object roots
- primitive property types: `string`, `integer`, `number`, `boolean`
- nested object properties
Unsupported schema constructs fail explicitly:
- arrays
- `anyOf`
- `oneOf`
Failure scripting:
- application errors with explicit status and OpenAI-shaped body
- optional `Retry-After`
- delay before headers
- hang before first byte
- inter-event stream delay
- close stream after N chunks
- malformed/truncated SSE ending