scriptoriumadmin/fabro
1
0
Fork 0
mirror of https://github.com/fabro-sh/fabro.git synced 2026-09-06 08:18:58 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 3
fabro/test/twin/openai/docs/compatibility-matrix.md
Bryan Helmkamp 5b7eabee8f Add twin test mode for OpenAI E2E tests 
Integrate twin-openai (fake OpenAI server) into the workspace and wire
it into the e2e_test macro so OpenAI tests can run without real API
credentials. The twin server starts in-process via OnceLock on first use
and provides per-test isolation through bearer-token namespacing.

Changes:
- Add Twin as default TestMode, replacing Off (gating now via #[ignore])
- Extend #[e2e_test] macro with `twin` requirement for twin-only,
  live-only, and dual-mode (twin + live) test gating
- Add e2e_openai!() macro returning (base_url, api_key)
- Convert openai_complete and openai_gpt_5_3_codex_complete to dual-mode
- Add new openai_server_error twin-only test with scripted 500 error
- Standardize axum 0.8 as workspace dependency across all crates
- Relax twin-openai ResponsesRequest to accept unknown fields via flatten

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 20:36:48 -04:00

1.3 KiB
Raw Permalink Blame History

Compatibility Matrix

Supported endpoints:

  • GET /healthz
  • POST /v1/responses
  • POST /v1/chat/completions
  • POST /__admin/scenarios
  • POST /__admin/reset
  • GET /__admin/requests

State isolation:

  • /v1/* request state is scoped by bearer token
  • admin routes may include the same bearer token to target that namespace
  • admin routes without auth operate on the global namespace

Supported /v1/responses fields:

  • bearer auth
  • stream
  • metadata
  • stop
  • previous_response_id
  • reasoning
  • text.format.type = text | json_object | json_schema
  • image inputs in input[*].content[*].type = input_image
  • scripted tool calls and continuation input items

Supported /v1/chat/completions fields:

  • bearer auth
  • stream
  • tools
  • tool_choice
  • response_format.type = text | json_object | json_schema
  • stop
  • reasoning-bearing assistant content

Structured output subset:

  • object roots
  • primitive property types: string, integer, number, boolean
  • nested object properties

Unsupported schema constructs fail explicitly:

  • arrays
  • anyOf
  • oneOf

Failure scripting:

  • application errors with explicit status and OpenAI-shaped body
  • optional Retry-After
  • delay before headers
  • hang before first byte
  • inter-event stream delay
  • close stream after N chunks
  • malformed/truncated SSE ending