litellm/tests/integration/_support
Alexander Chernov d1cde1002e
fix(chatgpt): forward text and max_output_tokens to the Codex backend (#38086)
* fix(chatgpt): keep text and accept string input for the Codex backend

Two request-side defects stop chatgpt/* serving structured output or a string
input.

The allowlist at the end of transform_responses_api_request omits "text". The
chat-to-responses bridge translates response_format into text.format, so the
allowlist discards strict schemas silently and the backend answers with prose
instead of JSON. The backend does honour text.format when it receives it.

Separately the backend rejects a bare string input with
{"detail": "Input must be a list"}, while the Responses API itself accepts
either shape, so the string needs wrapping before it is sent.

Both are request-shaping only, so neither depends on the streaming work in
#31332 or #34095, though those two block the same call and have to be in place
to observe this one end to end.

The "text" case was reported in #24356 and closed by the stale bot without a
fix.

Signed-off-by: Alexander Chernov <alexander@chernov.it>

* fix(chatgpt): annotate the input coercion for the type-discipline gate

The LIT002 budget counts mutable-collection construction, and wrapping the
string input builds a list and a dict, which put the total two over its
ceiling. Both constructions are the wire payload this backend requires, so the
list cannot be replaced with an immutable equivalent; annotate the line with a
reason instead, matching the convention used elsewhere in the tree.

LIT005 freezes reasonless suppressions at zero, so the reason is required
rather than decorative.

Signed-off-by: Alexander Chernov <alexander@chernov.it>

* refactor(chatgpt): bind the coerced input instead of rebinding the parameter

Greptile flagged the parameter rebinding and the explanatory comments against
the repo conventions in CLAUDE.md, which bans rebinding a function parameter
(LIT011) and keeps comments to suppressions, TODOs and genuinely complex logic

Bind a Final local for the coerced input rather than reassigning `input`, and
drop both prose comments; the rationale lives in the commit and the PR body
rather than duplicated at the call site. The mutable-ok suppression stays,
since it is the allowed kind and the list is still constructed

Behaviour is unchanged: 24 tests pass, and the same 4 fail with only the
provider file reverted to base

Signed-off-by: Alexander Chernov <alexander@chernov.it>

* fix(chatgpt): forward max_output_tokens to the Codex backend and return incomplete non-stream responses

The Codex backend accepts and enforces max_output_tokens, so the allowlist now keeps it
instead of dropping it. A structured-output request could otherwise stream until the
backend gave up, ignoring the caller's max_tokens.

A capped request ends with response.incomplete, so the non-stream SSE aggregator now
treats that event as terminal and returns the partial response instead of an error.

---------

Signed-off-by: Alexander Chernov <alexander@chernov.it>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-11-06 06:14:32 -08:00
..
__init__.py test: add CircleCI integration contract foundation 2026-09-14 03:30:52 -07:00
a2a_denial_probe.py fix(a2a): log authenticated pre-dispatch agent rejections (#49492) 2026-10-31 15:53:01 -07:00
actors.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
agentic_probe.py fix(responses): honor caller stream flag when provider forces SSE (internal copy of #34095) (#41235) 2026-10-06 03:30:47 +00:00
anthropic_refusal.py fix(caching): stop counting an unrun server tool call mark, and follow #49192 and #35602 in their integration tests (#49468) 2026-10-31 12:27:17 -07:00
anthropic_sse.py fix(router): retry a /v1/messages stream the provider drops before the first content chunk (#44276) 2026-10-05 22:53:39 +00:00
anthropic_thinking.py test: expect the list-position index on streamed thinking blocks in the integration suite (#46618) 2026-10-15 05:52:56 +00:00
asgi.py test(integration): fix four flaky integration tests (#49554) 2026-11-02 09:23:17 -08:00
async_server.py fix(bedrock): forward Nova Sonic realtime tool calls with their arguments (#46214) 2026-10-16 01:10:53 +00:00
aws_control_plane.py fix(bedrock): list the account's invocable models behind bedrock/* when check_provider_endpoint is on (#43981) 2026-10-09 14:14:17 -07:00
bedrock_discovery.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
bedrock_realtime_peer.py fix(bedrock)!: block realtime replay after an input-first start and reject malformed Nova Sonic session settings (#46936) 2026-10-17 00:29:50 +00:00
bedrock_runtime_peer.py test(ci): fix six CircleCI test regressions on main (#44429) 2026-10-03 20:55:01 -07:00
browser_state.py test: add extension and browser integration contracts 2026-09-16 13:15:53 -07:00
budget.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
claude_code.py test(anthropic): native /v1/messages reasoning integration tests built on a captured Claude Code request (#43361) 2026-10-01 09:12:53 -07:00
client.py fix(caching): read the cache key's model group from the router's metadata for the call (#48917) 2026-10-29 05:35:23 -07:00
codex_vendor.py fix(chatgpt): forward text and max_output_tokens to the Codex backend (#38086) 2026-11-06 06:14:32 -08:00
copilot_session_prometheus_worker.py fix(prometheus): count provider-side 401s in LLM failure and deployment metrics (#46668) 2026-10-26 17:57:06 -07:00
custom_video_provider.py feat(videos): serve custom_provider_map BaseVideoConfig providers on /v1/videos (#49785) 2026-11-02 21:28:46 -08:00
daily_activity.py test(usage): assert per-user entity spend and metadata (#50251) 2026-11-05 20:16:40 +00:00
daily_spend_rows.py fix(ui): group cost optimization cache leakage by model group (#43008) 2026-09-24 23:51:51 -07:00
database.py test(integration): isolate every multi-worker integration suite by default (#46027) 2026-10-12 03:46:06 -07:00
database_relay.py test(integration): hand every harness server a port from one per-worker block (#46761) 2026-10-15 16:56:40 -07:00
deepseek_tokens.py fix(token_counter): count DeepSeek V4 text with DeepSeek's own tokenizer (#47353) 2026-10-20 03:43:35 +00:00
device_login.py test(copilot): route direct-token fixtures through authenticated peers (#50307) 2026-11-05 17:42:46 -08:00
forward_proxy.py feat(strands_decider): call a Strands Decider on an AgentCore runtime ARN (#45678) 2026-10-09 22:51:19 -07:00
generation.py fix(tests): resolve the integration support package without run.py's PYTHONPATH 2026-09-15 21:10:08 -07:00
hook_count_probe.py fix(mcp): bill both calls of a streamed chat completions MCP auto-execute (#48899) 2026-10-30 07:34:10 -07:00
hook_counts.py fix(logging)!: fire one CustomLogger failure hook per async call outside the async flag list (#47124) 2026-10-20 15:19:14 -07:00
idle_gateway.py fix(http): restore the 15s aiohttp keep-alive default in the Python and Rust clients (#46355) 2026-10-14 10:50:11 -07:00
interception_vendor.py fix(spend): bill web search interception follow-ups once under the outer request (#49762) 2026-11-02 20:08:45 -08:00
mail.py feat(proxy): email alerts at configured percentages of a team member budget (#42665) 2026-09-26 02:42:45 +00:00
manifest.py fix(caching): never store or serve a chat completion with no choices (#44709) 2026-10-06 20:07:30 +00:00
mcp.py test(mcp): cancel disconnected HTTP tool handlers (#49436) 2026-10-31 16:41:49 -07:00
mcp_e2e_upstream_server.py test: migrate proxy security, schema migration and MCP legacy tests (#45937) 2026-10-11 16:17:24 -07:00
mcp_grants.py fix(mcp): resolve team-granted toolsets for non-admin keys and dashboard sessions (#43908) 2026-10-01 14:15:06 -07:00
mcp_stdio_peer.py test(integration): add MCP gateway coverage wave 1 with a dedicated mcp shard and proxy coverage artifact (#42711) 2026-09-23 07:48:46 -07:00
oauth_server.py test(integration): credential canary slots for MCP and pass-through credentials (#43308) 2026-09-28 17:10:51 -07:00
openai_wire.py test: fix shared-provider discovery, Codex catalog size and generated master key mismatches in CircleCI suites (#44905) 2026-10-07 00:11:25 +00:00
ordering.py fix(ci): isolate integration workers and repair test fixtures (#45863) 2026-10-11 01:56:02 -07:00
otlp_sink.py test(integration): hand every harness server a port from one per-worker block (#46761) 2026-10-15 16:56:40 -07:00
pdf_document.py fix(token_counter): price a base64 PDF document per page instead of as one image (#45301) 2026-10-08 21:20:46 -07:00
ports.py test(integration): hand every harness server a port from one per-worker block (#46761) 2026-10-15 16:56:40 -07:00
prisma_clients.py test: migrate tests/proxy_behavior into integration and unit tiers (#45939) 2026-10-11 08:28:42 -07:00
process.py fix(proxy): keep every startup line JSON when json_logs is on (#47035) 2026-10-27 14:49:07 -07:00
prometheus_series.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
prompt_cache_breakpoint.py fix(responses): map input_audio blocks in the chat-to-Responses bridge (#45224) 2026-10-07 22:25:35 -07:00
provider.py fix(ci): isolate integration workers and repair test fixtures (#45863) 2026-10-11 01:56:02 -07:00
proxy.py perf(ci): isolate and shard integration suites (#44146) 2026-10-10 11:15:19 -07:00
redaction_probe.py fix(logging): deliver completed batch success callbacks consistently (#47192) 2026-10-18 09:19:04 -07:00
redis_breaker_worker.py fix(caching): report Redis circuit-breaker metrics through the Prometheus services logger (#46484) 2026-10-14 14:29:54 -07:00
redis_process.py test(integration): hand every harness server a port from one per-worker block (#46761) 2026-10-15 16:56:40 -07:00
responses_replay.py fix(responses): replay the original Responses item order, phase, and boundaries on the next chat turn (#48912) 2026-10-29 00:56:02 -07:00
responses_stream.py fix(responses): keep the provider code and message on top-level stream error events (#46438) 2026-10-14 16:16:46 -07:00
responses_vendor.py fix(responses): drop bridge-minted reasoning items from OpenAI replays (#44132) 2026-10-02 19:39:48 -07:00
responses_ws_peer.py fix(responses): write one spend-log row per native WebSocket turn (#46768) 2026-10-15 19:05:22 -07:00
riva.py fix(nvidia_riva): send the api key exactly once by calling riva over grpc directly (#50140) 2026-11-04 18:07:31 -08:00
routing.py test(integration): isolate every multi-worker integration suite by default (#46027) 2026-10-12 03:46:06 -07:00
runtime.py test(integration): wait for every worker to report the websocket session cap instead of sleeping (#47412) 2026-10-19 15:35:29 -07:00
safeguards_rig.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
sagemaker_embedding.py feat(sagemaker)!: inference components and OpenAI-compatible embeddings (#44985) 2026-10-12 22:25:52 +00:00
scim.py fix(scim)!: delete a group through the team delete's bulk path instead of one member at a time (#47544) 2026-10-21 00:59:23 +00:00
sdk_models.py test(integration): build OpenAI and Anthropic SDK models before tests start threads (#46797) 2026-10-15 18:28:32 -07:00
sharding.py perf(ci): use native timing splits with bounded integration concurrency (#45837) 2026-10-10 13:35:11 -07:00
sigv4.py fix(bedrock): list the account's invocable models behind bedrock/* when check_provider_endpoint is on (#43981) 2026-10-09 14:14:17 -07:00
timing.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
tls.py fix(azure_storage): name Data Lake objects without base64 padding or slashes (#43914) 2026-09-30 18:17:13 -07:00
tool_rows.py feat(tool-policies): show the user who owns the key that discovered a tool (#43892) 2026-10-01 20:34:57 +00:00
tool_search.py fix(responses bridge): round-trip tool_search_call and tool_search_output items between turns (#46444) 2026-10-14 21:40:14 -07:00
upstream.py test(integration): stop GET /__observations from draining the upstream log (#47256) 2026-10-18 09:58:05 +00:00
vertex.py fix(vertex_ai): forward system and tools to partner model count_tokens (#43900) 2026-10-03 16:36:14 -07:00
wire.py test(integration): stop the wire peer from dropping requests whose bytes arrive late (#49889) 2026-11-05 09:24:05 -08:00
worker_readiness.py test(integration): cover responses endpoint contracts (#44733) 2026-10-14 06:59:15 +00:00
workers.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00