scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-21 05:15:06 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 161
litellm/tests/integration/providers
History
devin-ai-integration[bot] 5b042c0db6
fix(rust): add the dangerous-tool-use beta when an anthropic /v1/messages request carries safeguards (#46102) 
* fix(rust): add the dangerous-tool-use beta when an anthropic /v1/messages request carries safeguards

* test(rust): check the dangerous-tool-use beta reaches the wire with safeguards

* fix(rust): derive the dangerous-tool-use beta only for hosts with a beta map column

DeepSeek and the other hosts without a column in anthropic_beta_headers_config.json
keep forwarding safeguards and client betas as sent. Adds the wire-level audit tests.

* fix(rust): add the safeguards beta on Azure AI and Claude Platform hosts

Azure AI and Claude Platform Messages transforms now add the
dangerous-tool-use beta for a non-null safeguards field, through the
Azure AI and Anthropic beta-map columns. The safeguards wire and chaos
tests get unique file names so they do not collide with the Python
fix's tests, the Python-route test asserts the fixed behavior behind a
BUG skip, and the DeepSeek test checks that no beta is added

* fix(rust): keep Claude Platform out of the safeguards beta until a route can prove it

Claude Platform has no beta-map column and no QA account, so it goes back to sending
safeguards with no derived beta, like DeepSeek. Also release held chaos-test requests
before the thread pool waits on them, and mark the no-column test as rstest.

* fix(rust): derive the safeguards beta for Claude Platform and keep the null-cell test off Databricks

Claude Platform on AWS answers 400 to safeguards without dangerous-tool-use-2026-09-03 and 200 with it, so its Messages transform derives the beta through the Anthropic column again. The null-cell case now uses only Bedrock Converse, since Databricks support for the beta is changing in a sibling PR.

* fix(rust): drop a null safeguards key instead of sending it without its beta

Anthropic rejects any safeguards key, null included, when the
dangerous-tool-use beta is missing, so an explicit "safeguards": null on the
Rust gateway still answered 400. The shared Messages transform now drops a
null safeguards key on every host, as the proxy already drops None params
before calling Rust, so it answers 200 with no beta.

The scripted peer in the integration wire tests now rejects the key on
presence, matching the real API.

* test: cover Bedrock Mantle in the null safeguards case and match the vendor's non-list wording

The Rust null-safeguards regression test now also runs for bedrock_mantle, whose beta is keyed on the field being present. The scripted Anthropic peer answers a non-list safeguards value with the real API's "valid array" wording.

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-10-17 16:48:05 +00:00
..
_anthropic_safeguards_beta_support.py fix(rust): add the dangerous-tool-use beta when an anthropic /v1/messages request carries safeguards (#46102) 2026-10-17 16:48:05 +00:00
_api_base_trailing_slash_support.py fix(llms): sanitize trailing slashes in provider get_complete_url (#42887) 2026-10-15 02:59:46 -07:00
_assistants_route_errors_support.py test(integration): hand every harness server a port from one per-worker block (#46761) 2026-10-15 16:56:40 -07:00
_cache_control_marks_support.py fix(responses): propagate cache_control through responses->chat completion bridge (#32511) 2026-10-14 15:44:34 -07:00
_coralbricks.py test(integration): hand every harness server a port from one per-worker block (#46761) 2026-10-15 16:56:40 -07:00
_count_tokens_system_lift.py fix(anthropic): count a leading system run through count_tokens' system parameter (#45463) 2026-10-08 22:34:25 -07:00
_dashscope_video_support.py feat(dashscope): add Wan 3.0, Wan 2.7 and HappyHorse video generation via /v1/videos (#43706) 2026-10-14 10:10:07 -07:00
_gemini_long_decode_support.py feat(gemini)!: auto-continue gemini-4-argon long decodes with continuationToken (#46240) 2026-10-15 22:54:23 +00:00
_gemini_tts_voice_support.py perf(ci): isolate and shard integration suites (#44146) 2026-10-10 11:15:19 -07:00
_idle_upstream_support.py fix(http): restore the 15s aiohttp keep-alive default in the Python and Rust clients (#46355) 2026-10-14 10:50:11 -07:00
_mantle_gpt_prompt_cache_support.py fix(bedrock_mantle): send OpenAI explicit prompt cache breakpoints for GPT-5.6 and newer (#38729) 2026-10-08 02:07:22 -07:00
_responses_bridge_prompt_cache_breakpoint.py fix(responses): keep prompt_cache_breakpoint markers in the chat to responses bridge (#44119) 2026-10-07 16:25:36 -07:00
_sagemaker_chat_container.py fix(sagemaker): keep AWS credentials out of the sagemaker_chat request body (#45724) 2026-10-10 15:30:54 -07:00
_tool_search_support.py fix(anthropic): keep tool_search_tool_result so the search call replays with its result (#46440) 2026-10-14 18:52:32 -07:00
_vertex_batch_rows.py test(integration): hand every harness server a port from one per-worker block (#46761) 2026-10-15 16:56:40 -07:00
fine_tuning_proxy_config.yaml test: migrate SDK and proxy-booting legacy CircleCI tests to their tiers (#45758) 2026-10-11 00:17:21 -07:00
openai_fine_tuning.jsonl test: migrate SDK and proxy-booting legacy CircleCI tests to their tiers (#45758) 2026-10-11 00:17:21 -07:00
test_a2a_sdk_upstream_wire.py test: migrate legacy unit dirs and unrun folders into tests/unit, tests/integration and tests/e2e (#45940) 2026-10-11 06:24:57 -07:00
test_anthropic_count_tokens_system_lift_wire.py test(integration): hand every harness server a port from one per-worker block (#46761) 2026-10-15 16:56:40 -07:00
test_anthropic_empty_user_turn_wire.py fix(anthropic): keep trailing user messages with None or empty list content (#41397) 2026-10-16 07:41:05 +00:00
test_anthropic_passthrough_wire.py test(integration): cover pass_through endpoint contracts (#44690) 2026-10-14 07:38:17 +00:00
test_anthropic_refusal_stop_details_chaos.py test(integration): hand every harness server a port from one per-worker block (#46761) 2026-10-15 16:56:40 -07:00
test_anthropic_refusal_stop_details_wire.py fix(anthropic): keep refusal stop_reason and stop_details on the logged response (#46104) 2026-10-14 14:42:49 -07:00
test_anthropic_skills_create_wire.py test(integration): cover anthropic_messages endpoint contracts (#44704) 2026-10-14 06:57:54 +00:00
test_anthropic_system_cache_control_wire.py test(integration): assert /v1/responses usage reports Anthropic system cache write then read (#42855) 2026-09-23 18:52:06 -07:00
test_anthropic_thinking_signature_logging_wire.py fix(anthropic): stop repeating streamed thinking text in the signature chunk (#44127) 2026-10-02 19:38:41 -07:00
test_anthropic_thinking_signature_stream_wire.py test: expect the list-position index on streamed thinking blocks in the integration suite (#46618) 2026-10-15 05:52:56 +00:00
test_anthropic_tool_search_result_chaos.py fix(anthropic): keep tool_search_tool_result so the search call replays with its result (#46440) 2026-10-14 18:52:32 -07:00
test_anthropic_tool_search_result_matrix_wire.py fix(anthropic): count a replayed server tool call's cache mark the way modify_params sends it (#46591) 2026-10-15 14:46:43 -07:00
test_anthropic_tool_search_result_modify_params_wire.py fix(anthropic): count a replayed server tool call's cache mark the way modify_params sends it (#46591) 2026-10-15 14:46:43 -07:00
test_anthropic_tool_search_result_passthrough_wire.py fix(anthropic): keep tool_search_tool_result so the search call replays with its result (#46440) 2026-10-14 18:52:32 -07:00
test_anthropic_tool_search_result_wire.py fix(anthropic): keep tool_search_tool_result so the search call replays with its result (#46440) 2026-10-14 18:52:32 -07:00
test_api_base_trailing_slash_chaos.py fix(llms): sanitize trailing slashes in provider get_complete_url (#42887) 2026-10-15 02:59:46 -07:00
test_api_base_trailing_slash_owned_proxy.py fix(llms): sanitize trailing slashes in provider get_complete_url (#42887) 2026-10-15 02:59:46 -07:00
test_api_base_trailing_slash_wire.py fix(llms): sanitize trailing slashes in provider get_complete_url (#42887) 2026-10-15 02:59:46 -07:00
test_assemblyai_passthrough.py test: migrate SDK and proxy-booting legacy CircleCI tests to their tiers (#45758) 2026-10-11 00:17:21 -07:00
test_assistants_evals_wire.py test(integration): pin assistants and evals endpoint contracts (#44734) 2026-10-14 07:08:35 +00:00
test_assistants_route_errors_chaos.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_assistants_route_errors_wire.py test(integration): hand every harness server a port from one per-worker block (#46761) 2026-10-15 16:56:40 -07:00
test_audit_any_sweep_harness_endpoint.py refactor(types): replace Any with proven types in 9 files (#44389) 2026-10-03 21:03:16 +00:00
test_audit_any_sweep_vector_store.py refactor(types): replace Any with proven types in 9 files (#44389) 2026-10-03 21:03:16 +00:00
test_azure_ai_chat_wire.py test(integration): regression tests for August provider translation and streaming bugs (#42621) 2026-09-22 20:52:00 -07:00
test_azure_ai_claude_messages_wire.py test(integration): cover anthropic_messages endpoint contracts (#44704) 2026-10-14 06:57:54 +00:00
test_azure_ai_flux2_image_wire.py test(integration): regression tests for August provider translation and streaming bugs (#42621) 2026-09-22 20:52:00 -07:00
test_azure_ai_rerank_auth_wire.py test(integration): regression tests for August provider translation and streaming bugs (#42621) 2026-09-22 20:52:00 -07:00
test_azure_embedding_wire.py test(integration): cover embeddings endpoint contracts (#44660) 2026-10-14 03:38:12 +00:00
test_azure_reasoning_effort_none_wire.py fix(providers): route Azure Responses past dated api versions, count Grok reasoning tokens on azure_ai, refresh provider sets after register_model, fix gpt-6.1-sol Azure rows (#46377) 2026-10-16 15:23:46 -07:00
test_azure_responses_api_version_wire.py fix(providers): route Azure Responses past dated api versions, count Grok reasoning tokens on azure_ai, refresh provider sets after register_model, fix gpt-6.1-sol Azure rows (#46377) 2026-10-16 15:23:46 -07:00
test_bedrock_anthropic_beta_header_wire.py refactor(types): replace Any with proven types in 157 files (#44798) 2026-10-06 15:31:54 +00:00
test_bedrock_auth_wire.py test(integration): regression tests for August provider translation and streaming bugs (#42621) 2026-09-22 20:52:00 -07:00
test_bedrock_batch_blank_s3_env_wire.py test(e2e): assert only litellm-owned batch behavior and move the blank S3 env pin to an integration test (#43321) 2026-09-26 11:17:03 -07:00
test_bedrock_batch_files_wire.py test(integration): regression tests for August provider translation and streaming bugs (#42621) 2026-09-22 20:52:00 -07:00
test_bedrock_batch_retrieve_status_wire.py fix(bedrock): map Scheduled batch jobs to in_progress on retrieve (#45157) 2026-10-07 18:08:51 -07:00
test_bedrock_claude_thinking_wire.py fix(anthropic): drop unknown legacy thinking keys under drop_params when rebuilding adaptive thinking (#46741) 2026-10-15 23:09:30 +00:00
test_bedrock_converse_client_metadata_wire.py test(integration): regression tests for July provider translation, routing and streaming bugs (#42693) 2026-09-23 09:51:06 -07:00
test_bedrock_converse_config_blocks_wire.py test(integration): regression tests for August provider translation and streaming bugs (#42621) 2026-09-22 20:52:00 -07:00
test_bedrock_converse_lookaround_regex_chaos.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_bedrock_converse_lookaround_regex_wire.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_bedrock_converse_missing_content_wire.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_bedrock_converse_redacted_thinking_wire.py test: expect the list-position index on streamed thinking blocks in the integration suite (#46618) 2026-10-15 05:52:56 +00:00
test_bedrock_converse_stream_event_frames_wire.py test(integration): update stale CircleCI integration expectations (#46570) 2026-10-14 18:21:59 -07:00
test_bedrock_deepseek_reasoning_wire.py test(integration): regression tests for July provider translation, routing and streaming bugs (#42693) 2026-09-23 09:51:06 -07:00
test_bedrock_embedding_wire.py test(integration): regression tests for August provider translation and streaming bugs (#42621) 2026-09-22 20:52:00 -07:00
test_bedrock_gpt5_reasoning_wire.py test(integration): regression tests for August provider translation and streaming bugs (#42621) 2026-09-22 20:52:00 -07:00
test_bedrock_gpt_responses_native_wire.py feat(bedrock): serve gpt-5.6+ chat completions natively by default, with chat_completions/ opt-in for gpt-oss and grok (#44307) 2026-10-02 19:55:54 -07:00
test_bedrock_invoke_cache_usage_wire.py test(integration): regression tests for July provider translation, routing and streaming bugs (#42693) 2026-09-23 09:51:06 -07:00
test_bedrock_invoke_dangerous_tool_use_beta_wire.py fix(bedrock): drop dangerous-tool-use beta without safeguards (#44471) 2026-10-16 21:00:37 +00:00
test_bedrock_kimi_k3_cache_point_wire.py test(integration): hand every harness server a port from one per-worker block (#46761) 2026-10-15 16:56:40 -07:00
test_bedrock_knowledge_base_user_context_wire.py test(integration): regression tests for July provider translation, routing and streaming bugs (#42693) 2026-09-23 09:51:06 -07:00
test_bedrock_mantle_claude_chat_chaos.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_bedrock_mantle_claude_chat_wire.py test(integration): assert the Messages API health probe on Mantle Claude (#44612) 2026-10-05 15:28:02 -07:00
test_bedrock_mantle_codex_input_wire.py test(integration): regression tests for July provider translation, routing and streaming bugs (#42693) 2026-09-23 09:51:06 -07:00
test_bedrock_mantle_context_overflow_fallbacks_wire.py test(integration): send post-kill requests on a fresh connection in two worker-kill tests (#47080) 2026-10-16 23:27:51 -07:00
test_bedrock_mantle_context_overflow_wire.py test(integration): hand every harness server a port from one per-worker block (#46761) 2026-10-15 16:56:40 -07:00
test_bedrock_mantle_count_tokens_wire.py fix(proxy)!: answer 400 for malformed tools on count_tokens instead of crashing (#46583) 2026-10-16 14:57:53 -07:00
test_bedrock_mantle_gpt_prompt_cache_breakpoint_wire.py fix(bedrock_mantle): send OpenAI explicit prompt cache breakpoints for GPT-5.6 and newer (#38729) 2026-10-08 02:07:22 -07:00
test_bedrock_mantle_output_limit_wire.py test(integration): update stale CircleCI integration expectations (#46570) 2026-10-14 18:21:59 -07:00
test_bedrock_mantle_responses_wire.py test(integration): regression tests for July provider translation, routing and streaming bugs (#42693) 2026-09-23 09:51:06 -07:00
test_bedrock_mantle_wire.py test(integration): regression tests for July provider translation, routing and streaming bugs (#42693) 2026-09-23 09:51:06 -07:00
test_bedrock_marengo_embed_3_wire.py test(integration): regression tests for July provider translation, routing and streaming bugs (#42693) 2026-09-23 09:51:06 -07:00
test_bedrock_model_discovery_chaos.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_bedrock_model_discovery_wire.py fix(bedrock): list the account's invocable models behind bedrock/* when check_provider_endpoint is on (#43981) 2026-10-09 14:14:17 -07:00
test_bedrock_native_structured_output_wire.py fix(bedrock): use native structured output on Haiku 5.5 and Sonnet 5.5, and fix Fable 5 and Sonnet 4.6 schema 400s (#46870) 2026-10-16 20:57:23 -07:00
test_bedrock_nova_sonic_realtime_tool_calls.py fix(cost): price exact cost-map names before price-free capability rules, and fix SCIM and MCP integration tests on CircleCI (#47037) 2026-10-17 06:46:52 +00:00
test_bedrock_passthrough_aws_errors.py fix(bedrock-passthrough)!: return AWS-shaped errors so AWS SDKs read the code and message (#46522) 2026-10-15 22:05:46 -07:00
test_bedrock_passthrough_request_metadata_wire.py feat(bedrock): apply bedrock_request_metadata_fields on the /bedrock pass-through routes (#47040) 2026-10-17 14:01:50 +00:00
test_bedrock_passthrough_stream_wire.py test(integration): cover pass_through endpoint contracts (#44690) 2026-10-14 07:38:17 +00:00
test_bedrock_rerank_wire.py test(integration): regression tests for August provider translation and streaming bugs (#42621) 2026-09-22 20:52:00 -07:00
test_bedrock_role_configuration.py test(integration): regression tests for July provider translation, routing and streaming bugs (#42693) 2026-09-23 09:51:06 -07:00
test_bedrock_runtime_chat_completions_chaos.py test(integration): hand every harness server a port from one per-worker block (#46761) 2026-10-15 16:56:40 -07:00
test_bedrock_runtime_chat_completions_sad_wire.py test(integration): fix seven CircleCI integration tests that race or assume a worker spread (#47001) 2026-10-16 22:45:29 +00:00
test_bedrock_runtime_chat_completions_wire.py fix(bedrock): drop stop sequences for GPT 5.6 and newer instead of forwarding them to a 400 (#45664) 2026-10-09 15:00:24 -07:00
test_bedrock_stream_timeout_chaos.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_bedrock_stream_timeout_wire.py fix(responses): stop SDK retries nesting under router retries, and unbreak CircleCI integration tests (#44791) 2026-10-06 15:57:33 +00:00
test_bedrock_thinking_tokens_wire.py test(integration): regression tests for July provider translation, routing and streaming bugs (#42693) 2026-09-23 09:51:06 -07:00
test_cache_control_tool_call_marks_owned_proxy.py test(integration): hand every harness server a port from one per-worker block (#46761) 2026-10-15 16:56:40 -07:00
test_cache_control_tool_call_marks_wire.py test(integration): follow #46101 in the breakpoint cache mark census expectation (#46509) 2026-10-14 20:55:43 -07:00
test_chat_completions_tool_search_chaos.py test(integration): hand every harness server a port from one per-worker block (#46761) 2026-10-15 16:56:40 -07:00
test_chat_completions_tool_search_history_wire.py fix(anthropic)!: translate the OpenAI tool_search tool on /v1/chat/completions for Claude models (#46389) 2026-10-15 04:47:05 +00:00
test_chat_completions_tool_search_wire.py fix(anthropic)!: translate the OpenAI tool_search tool on /v1/chat/completions for Claude models (#46389) 2026-10-15 04:47:05 +00:00
test_chatgpt_responses_caller_stream_chaos.py test(integration): hand every harness server a port from one per-worker block (#46761) 2026-10-15 16:56:40 -07:00
test_chatgpt_responses_caller_stream_wire.py fix(responses): honor caller stream flag when provider forces SSE (internal copy of #34095) (#41235) 2026-10-06 03:30:47 +00:00
test_chatgpt_responses_top_level_error_wire.py fix(responses): keep the provider code and message on top-level stream error events (#46438) 2026-10-14 16:16:46 -07:00
test_cloudflare_clef_images_chaos.py feat(decisions)!: send images to Cloudflare Clef through its images field (#45734) 2026-10-14 12:14:48 +00:00
test_cloudflare_clef_images_wire.py feat(decisions)!: send images to Cloudflare Clef through its images field (#45734) 2026-10-14 12:14:48 +00:00
test_cloudflare_clef_omni_media_chaos.py test(integration): hand every harness server a port from one per-worker block (#46761) 2026-10-15 16:56:40 -07:00
test_cloudflare_clef_omni_media_wire.py feat(decisions): support Clef-omni audio and video (#45811) 2026-10-12 20:29:49 +00:00
test_container_file_routes_wire.py refactor(types): replace Any with proven types in 157 files (#44798) 2026-10-06 15:31:54 +00:00
test_coralbricks_provider_chaos.py feat(providers): add CoralBricks as a native provider (#44700) 2026-10-12 21:56:59 +00:00
test_coralbricks_provider_env_wire.py feat(providers): add CoralBricks as a native provider (#44700) 2026-10-12 21:56:59 +00:00
test_coralbricks_provider_wire.py test(integration): hand every harness server a port from one per-worker block (#46761) 2026-10-15 16:56:40 -07:00
test_count_tokens_error_shapes_wire.py fix(proxy)!: count endpoints return errors in their source API shape instead of silent local counts (#46815) 2026-10-16 00:11:39 -07:00
test_cursor_responses_wire.py fix(cache): keep cache hits free of unset fields, and follow intentional behavior changes in CircleCI suites (#46359) 2026-10-14 08:56:05 -07:00
test_custom_prompt_template_chaos.py fix(prompt-templates): treat initial_prompt_value and final_prompt_value as optional (#39762) 2026-10-16 10:30:59 +00:00
test_custom_prompt_template_wire.py fix(prompt-templates): treat initial_prompt_value and final_prompt_value as optional (#39762) 2026-10-16 10:30:59 +00:00
test_dashscope_chat_wire.py fix(proxy): keep provider refusal, logprobs and system_fingerprint in chat completions (#44788) 2026-10-13 13:17:40 -07:00
test_dashscope_video_chaos.py feat(dashscope): add Wan 3.0, Wan 2.7 and HappyHorse video generation via /v1/videos (#43706) 2026-10-14 10:10:07 -07:00
test_dashscope_video_wire.py feat(dashscope): add Wan 3.0, Wan 2.7 and HappyHorse video generation via /v1/videos (#43706) 2026-10-14 10:10:07 -07:00
test_databricks_agent_wire.py feat(agents): support Databricks Agent Platform agents in the agent gateway (#45681) 2026-10-15 12:42:06 -07:00
test_databricks_chat_wire.py test: cover response contracts across translation endpoints (#46632) 2026-10-15 00:05:09 -07:00
test_databricks_decisions_wire.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_databricks_oauth_wire.py test(integration): regression tests for August provider translation and streaming bugs (#42621) 2026-09-22 20:52:00 -07:00
test_decision_model_guardrail.py fix(guardrails): resolve decision models in caller team scope (#46085) 2026-10-15 11:43:36 -07:00
test_decisions_box_deployment_flag_chaos.py test(integration): hand every harness server a port from one per-worker block (#46761) 2026-10-15 16:56:40 -07:00
test_decisions_box_deployment_flag_wire.py fix(decisions): read a deployment's own supports_box_questions flag before the cost map (#46696) 2026-10-15 12:34:36 -07:00
test_decisions_chaos.py test(integration): hand every harness server a port from one per-worker block (#46761) 2026-10-15 16:56:40 -07:00
test_decisions_openai_format_wire.py fix(decisions): keep box answers from OpenAI-format providers instead of turning them into refusals (#46633) 2026-10-15 14:01:59 +00:00
test_decisions_wire.py feat(azure-ai): support Entra ID auth for Microsoft-Decision-1 on /v1/decisions and /v1/systemone (#46539) 2026-10-17 09:18:34 +00:00
test_deepseek_vision_wire.py test(integration): regression tests for August provider translation and streaming bugs (#42621) 2026-09-22 20:52:00 -07:00
test_device_code_login_guard_boot.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_device_code_login_guard_wire.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_embedding_cache_partial_hit_wire.py test(integration): cover embeddings endpoint contracts (#44660) 2026-10-14 03:38:12 +00:00
test_embedding_token_array_wire.py test(integration): cover embeddings endpoint contracts (#44660) 2026-10-14 03:38:12 +00:00
test_fake_openai_upstream_routes.py test: migrate local_testing SDK and provider tests out of build_and_test (#45760) 2026-10-10 23:14:25 +00:00
test_fal_ai_chat_wire.py fix(fal_ai): honour global api_base for image generation and reject non-string reasoning_effort with 400 (#42512) 2026-09-22 13:13:54 -07:00
test_fal_ai_image_wire.py test(integration): cover images endpoint contracts (#44653) 2026-10-14 03:40:29 +00:00
test_fal_ai_passthrough_wire.py fix(fal_ai): align /fal_ai queue gate with the pricer and normalise resolution type (#42505) 2026-09-22 13:12:07 -07:00
test_fal_ai_video_wire.py fix(fal_ai): reuse the status client and headers on the video result probe (#42511) 2026-09-22 13:12:51 -07:00
test_files_batches_wire.py test(integration): cover files_batches endpoint contracts (#44703) 2026-10-14 06:57:41 +00:00
test_fireworks_ai_router_slug_wire.py feat(fireworks_ai): route and list the auto, auto-instant and firerouter routers (#43641) 2026-09-28 19:08:16 -07:00
test_fireworks_ai_session_affinity_wire.py test(integration): regression tests for July provider translation, routing and streaming bugs (#42693) 2026-09-23 09:51:06 -07:00
test_gemini_agents_wire.py test(integration): cover google_genai endpoint contracts (#44680) 2026-10-14 04:01:35 +00:00
test_gemini_anyof_null_branches_wire.py fix: preserve non-null anyOf branches when stripping null types for Vertex (#41683) 2026-10-16 08:00:48 +00:00
test_gemini_chat_video_metadata_wire.py feat(gemini): accept file content blocks with video_metadata on multimodal embeddings (#45305) 2026-10-08 13:20:03 -07:00
test_gemini_embedding_file_block_chaos.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_gemini_embedding_file_block_wire.py feat(gemini): accept file content blocks with video_metadata on multimodal embeddings (#45305) 2026-10-08 13:20:03 -07:00
test_gemini_image_config_and_usage_wire.py refactor(types): replace Any with proven types in 157 files (#44798) 2026-10-06 15:31:54 +00:00
test_gemini_interactions_wire.py test(integration): cover google_genai endpoint contracts (#44680) 2026-10-14 04:01:35 +00:00
test_gemini_long_decode_chaos.py feat(gemini)!: auto-continue gemini-4-argon long decodes with continuationToken (#46240) 2026-10-15 22:54:23 +00:00
test_gemini_long_decode_wire.py feat(gemini)!: auto-continue gemini-4-argon long decodes with continuationToken (#46240) 2026-10-15 22:54:23 +00:00
test_gemini_omitted_temperature.py test(translation): share rich fixtures across Claude, GPT and Gemini providers (#46585) 2026-10-14 19:53:09 -07:00
test_gemini_thinking_replay_chaos.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_gemini_thinking_replay_wire.py test: make 77 legacy live tests offline in litellm_utils, router_unit and responses dirs (#45298) 2026-10-08 02:37:55 -07:00
test_gemini_tool_result_part_reference_chaos.py fix(vertex_ai): prevent Gemini from misinterpreting $ref in tool results (#38284) 2026-10-15 06:26:47 -07:00
test_gemini_tool_result_part_reference_wire.py fix(vertex_ai): prevent Gemini from misinterpreting $ref in tool results (#38284) 2026-10-15 06:26:47 -07:00
test_gemini_tts_voice_mapping_owned_proxy.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_gemini_tts_voice_mapping_wire.py test(integration): hand every harness server a port from one per-worker block (#46761) 2026-10-15 16:56:40 -07:00
test_google_genai_adapter_wire.py fix(vertex_ai): route Model Garden models through the generateContent adapter (#41836) 2026-10-15 08:06:49 -07:00
test_google_genai_native_wire.py test(integration): cover google_genai endpoint contracts (#44680) 2026-10-14 04:01:35 +00:00
test_google_genai_proxy_sdk_wire.py test: migrate SDK and proxy-booting legacy CircleCI tests to their tiers (#45758) 2026-10-11 00:17:21 -07:00
test_google_genai_vertex_routes_wire.py fix(vertex_ai): route Model Garden models through the generateContent adapter (#41836) 2026-10-15 08:06:49 -07:00
test_hosted_vllm_reasoning_content_chaos.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_hosted_vllm_reasoning_content_wire.py test: cover response contracts across translation endpoints (#46632) 2026-10-15 00:05:09 -07:00
test_idle_upstream_keepalive_edges_wire.py test(integration): update stale CircleCI integration expectations (#46570) 2026-10-14 18:21:59 -07:00
test_idle_upstream_keepalive_override_wire.py fix(http): restore the 15s aiohttp keep-alive default in the Python and Rust clients (#46355) 2026-10-14 10:50:11 -07:00
test_idle_upstream_reconnect_chaos_wire.py test(integration): hand every harness server a port from one per-worker block (#46761) 2026-10-15 16:56:40 -07:00
test_idle_upstream_reconnect_routes_wire.py fix(http): restore the 15s aiohttp keep-alive default in the Python and Rust clients (#46355) 2026-10-14 10:50:11 -07:00
test_idle_upstream_reconnect_rust_messages_wire.py fix(http): restore the 15s aiohttp keep-alive default in the Python and Rust clients (#46355) 2026-10-14 10:50:11 -07:00
test_idle_upstream_reconnect_wire.py fix(http): restore the 15s aiohttp keep-alive default in the Python and Rust clients (#46355) 2026-10-14 10:50:11 -07:00
test_image_gen_drop_params_wire.py test(integration): cover images endpoint contracts (#44653) 2026-10-14 03:40:29 +00:00
test_inferenceapis_context_window_fallbacks_wire.py feat(providers): add Inference APIs as OpenAI-compatible provider (#41669) 2026-10-15 08:17:36 -07:00
test_inferenceapis_wire.py feat(providers): add Inference APIs as OpenAI-compatible provider (#41669) 2026-10-15 08:17:36 -07:00
test_interactions_openai_bridge_wire.py refactor(types): replace Any with proven types in 157 files (#44798) 2026-10-06 15:31:54 +00:00
test_internal_params_wire.py fix(params): validate stream_chunk_size once, before any provider call (#43222) 2026-09-26 23:01:20 +00:00
test_lm_studio_chat_wire.py test: migrate local_testing SDK and provider tests out of build_and_test (#45760) 2026-10-10 23:14:25 +00:00
test_malformed_tools_chaos.py fix(proxy)!: answer 400 for malformed tools on count_tokens instead of crashing (#46583) 2026-10-16 14:57:53 -07:00
test_malformed_tools_wire.py fix(proxy)!: answer 400 for malformed tools on count_tokens instead of crashing (#46583) 2026-10-16 14:57:53 -07:00
test_moderations.py test: migrate SDK and proxy-booting legacy CircleCI tests to their tiers (#45758) 2026-10-11 00:17:21 -07:00
test_moderations_wire.py test(integration): cover small_inference endpoint contracts (#44654) 2026-10-14 04:06:15 +00:00
test_nvidia_nim_ranking_wire.py test(integration): regression tests for July provider translation, routing and streaming bugs (#42693) 2026-09-23 09:51:06 -07:00
test_ocr_multipart_wire.py test(integration): cover small_inference endpoint contracts (#44654) 2026-10-14 04:06:15 +00:00
test_ocr_router_wire.py test: move 81 legacy live tests in pass-through, spend, batches, audio, search, guardrails, image and ocr dirs offline (#45288) 2026-10-08 02:38:12 -07:00
test_ollama_prompt_tools_chaos.py fix(prompt_templates): do not mutate messages in-place in function_call_prompt (#41676) 2026-10-15 14:37:11 -07:00
test_ollama_prompt_tools_wire.py fix(prompt_templates): do not mutate messages in-place in function_call_prompt (#41676) 2026-10-15 14:37:11 -07:00
test_openai_chat_prompt_cache_breakpoint_wire.py fix(openai): carry cache_control to prompt_cache_breakpoint on chat completions to gpt-5.6 (#46101) 2026-10-13 15:45:49 -07:00
test_openai_chat_wire.py test(integration): cover chat_completions endpoint contracts (#44688) 2026-10-14 03:45:38 +00:00
test_openai_completion_wire.py test(integration): cover chat_completions endpoint contracts (#44688) 2026-10-14 03:45:38 +00:00
test_openai_count_tokens_tools_wire.py fix(count_tokens): return a 400 naming Anthropic blocks the OpenAI and Gemini count can't convert (#46772) 2026-10-16 03:19:55 +00:00
test_openai_dialect_prompt_cache_breakpoint_owned_proxy.py test(integration): fix seven CircleCI integration tests that race or assume a worker spread (#47001) 2026-10-16 22:45:29 +00:00
test_openai_embedding_timeout_wire.py test: migrate local_testing SDK and provider tests out of build_and_test (#45760) 2026-10-10 23:14:25 +00:00
test_openai_embedding_wire.py test(integration): cover embeddings endpoint contracts (#44660) 2026-10-14 03:38:12 +00:00
test_openai_error_handling.py test: migrate SDK and proxy-booting legacy CircleCI tests to their tiers (#45758) 2026-10-11 00:17:21 -07:00
test_openai_fine_tuning.py test: migrate SDK and proxy-booting legacy CircleCI tests to their tiers (#45758) 2026-10-11 00:17:21 -07:00
test_openai_image_edit_wire.py test(integration): cover images endpoint contracts (#44653) 2026-10-14 03:40:29 +00:00
test_openai_image_variation_wire.py test(integration): cover images endpoint contracts (#44653) 2026-10-14 03:40:29 +00:00
test_openai_like_response_headers_wire.py fix(prometheus): set the remaining rate-limit gauges from decimal headers again (#47039) 2026-10-17 05:20:00 +00:00
test_openai_passthrough_files_wire.py test: move 81 legacy live tests in pass-through, spend, batches, audio, search, guardrails, image and ocr dirs offline (#45288) 2026-10-08 02:38:12 -07:00
test_openai_passthrough_multipart_wire.py test(integration): cover pass_through endpoint contracts (#44690) 2026-10-14 07:38:17 +00:00
test_openai_realtime_transcription_url.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_openai_responses_prompt_cache_breakpoint_wire.py fix(openai): carry cache_control to prompt_cache_breakpoint on /v1/responses to gpt-5.6 (#46105) 2026-10-13 15:23:07 -07:00
test_openai_responses_tool_search_wire.py fix(responses): map Anthropic tool search to OpenAI server tool search (#46441) 2026-10-15 05:13:56 +00:00
test_openai_responses_websocket_wire.py fix(responses): write one spend-log row per native WebSocket turn (#46768) 2026-10-15 19:05:22 -07:00
test_openai_stream_text_usage_wire.py refactor(types): replace Any with proven types in 7 files (#43704) 2026-09-29 06:12:58 -07:00
test_provider_lookup_status.py test(integration): pin assistants and evals endpoint contracts (#44734) 2026-10-14 07:08:35 +00:00
test_rag_ingest_wire.py test(integration): cover vector_stores endpoint contracts (#44739) 2026-10-14 03:59:08 +00:00
test_realtime_transcription_guardrail_session_update.py fix(realtime)!: keep turn_detection.type when a transcript guardrail disables auto-response (#44939) 2026-10-16 15:24:40 +00:00
test_realtime_unguarded_voice_turns.py fix(realtime)!: keep turn_detection.type when a transcript guardrail disables auto-response (#44939) 2026-10-16 15:24:40 +00:00
test_realtime_webrtc_http.py test(integration): cover realtime endpoint contracts (#44689) 2026-10-14 03:49:34 +00:00
test_request_boundary.py fix(tests): resolve the integration support package without run.py's PYTHONPATH 2026-09-15 21:10:08 -07:00
test_rerank_latency_headers_wire.py test(integration): cover small_inference endpoint contracts (#44654) 2026-10-14 04:06:15 +00:00
test_responses_bridge_incomplete.py fix(responses): return the first message item as content for structured output on the chat bridge (#46508) 2026-10-14 16:58:00 -07:00
test_responses_bridge_input_audio_wire.py fix(responses): map input_audio blocks in the chat-to-Responses bridge (#45224) 2026-10-07 22:25:35 -07:00
test_responses_bridge_item_marks_guardrail_wire.py fix(responses): propagate cache_control through responses->chat completion bridge (#32511) 2026-10-14 15:44:34 -07:00
test_responses_bridge_message_cache_control_wire.py fix(responses): propagate cache_control through responses->chat completion bridge (#32511) 2026-10-14 15:44:34 -07:00
test_responses_bridge_message_cache_control_wire_chaos.py test(integration): hand every harness server a port from one per-worker block (#46761) 2026-10-15 16:56:40 -07:00
test_responses_bridge_namespace_tools.py test(integration): regression tests for July provider translation, routing and streaming bugs (#42693) 2026-09-23 09:51:06 -07:00
test_responses_bridge_prompt_cache_breakpoint.py fix(responses): keep prompt_cache_breakpoint markers in the chat to responses bridge (#44119) 2026-10-07 16:25:36 -07:00
test_responses_bridge_prompt_cache_breakpoint_chaos.py test(integration): hand every harness server a port from one per-worker block (#46761) 2026-10-15 16:56:40 -07:00
test_responses_bridge_prompt_cache_breakpoint_wire.py fix(anthropic): send detail on input_image parts bridged from /v1/messages to Responses (#46720) 2026-10-15 22:33:09 +00:00
test_responses_bridge_prompt_cache_breakpoint_wire_chaos.py test(integration): hand every harness server a port from one per-worker block (#46761) 2026-10-15 16:56:40 -07:00
test_responses_bridge_stream_output_items_wire.py fix(ollama): one stream id, 400 on non-text tool content, separated tool results, no empty message item (#45177) 2026-10-07 19:08:26 -07:00
test_responses_bridge_tool_search_chaos.py test(integration): fix seven CircleCI integration tests that race or assume a worker spread (#47001) 2026-10-16 22:45:29 +00:00
test_responses_bridge_tool_search_replay.py fix(responses bridge): round-trip tool_search_call and tool_search_output items between turns (#46444) 2026-10-14 21:40:14 -07:00
test_responses_bridge_tool_search_replay_wire.py fix(responses): translate OpenAI tool_search to Anthropic tool search for Claude models (#45882) 2026-10-14 20:42:22 -07:00
test_responses_bridge_tool_search_undeferred_wire.py fix(responses): translate OpenAI tool_search to Anthropic tool search for Claude models (#45882) 2026-10-14 20:42:22 -07:00
test_responses_bridge_tool_search_wire.py fix(responses bridge): round-trip tool_search_call and tool_search_output items between turns (#46444) 2026-10-14 21:40:14 -07:00
test_responses_client_header_forwarding_wire.py test(integration): regression tests for July provider translation, routing and streaming bugs (#42693) 2026-09-23 09:51:06 -07:00
test_responses_compact_wire.py fix(proxy): check previous_response_id ownership on /v1/responses/compact (#46408) 2026-10-14 17:57:34 +00:00
test_responses_core_wire.py test(integration): cover responses endpoint contracts (#44733) 2026-10-14 06:59:15 +00:00
test_responses_error_status_wire.py test: move 81 legacy live tests in pass-through, spend, batches, audio, search, guardrails, image and ocr dirs offline (#45288) 2026-10-08 02:38:12 -07:00
test_responses_lifecycle_wire.py test(integration): cover responses endpoint contracts (#44733) 2026-10-14 06:59:15 +00:00
test_responses_minted_reasoning_replay_chaos.py test(integration): hand every harness server a port from one per-worker block (#46761) 2026-10-15 16:56:40 -07:00
test_responses_minted_reasoning_replay_wire.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_responses_reasoning_item_wire.py refactor(types): replace Any with proven types in 157 files (#44798) 2026-10-06 15:31:54 +00:00
test_responses_websocket_session_limit.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_rust_anthropic_safeguards_beta_chaos.py fix(rust): add the dangerous-tool-use beta when an anthropic /v1/messages request carries safeguards (#46102) 2026-10-17 16:48:05 +00:00
test_rust_anthropic_safeguards_beta_wire.py fix(rust): add the dangerous-tool-use beta when an anthropic /v1/messages request carries safeguards (#46102) 2026-10-17 16:48:05 +00:00
test_s3_wire.py test: fix stale and flaky tests across CircleCI, GHA and Buildkite (#45530) 2026-10-09 11:40:14 -07:00
test_sagemaker_chat_credential_sources.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_sagemaker_chat_wire.py fix(sagemaker): keep AWS credentials out of the sagemaker_chat request body (#45724) 2026-10-10 15:30:54 -07:00
test_sagemaker_embedding_chaos.py test(integration): hand every harness server a port from one per-worker block (#46761) 2026-10-15 16:56:40 -07:00
test_sagemaker_embedding_wire.py feat(sagemaker)!: inference components and OpenAI-compatible embeddings (#44985) 2026-10-12 22:25:52 +00:00
test_search_wire.py fix(search): bill an unpriced fallback search tool at its own price, not the failed primary's (#46734) 2026-10-15 12:20:33 -07:00
test_strands_decider_agentcore_wire.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_tencent_chat_wire.py fix(proxy): keep provider refusal, logprobs and system_fingerprint in chat completions (#44788) 2026-10-13 13:17:40 -07:00
test_validated_reply_boundaries_under_faults.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_vector_store_create_wire.py test(integration): cover vector_stores endpoint contracts (#44739) 2026-10-14 03:59:08 +00:00
test_vector_store_search_wire.py test(integration): cover vector_stores endpoint contracts (#44739) 2026-10-14 03:59:08 +00:00
test_vertex_anthropic_inline_tools_beta_wire.py test(integration): remove six setup causes of CircleCI integration reds (#46490) 2026-10-14 12:00:35 -07:00
test_vertex_anthropic_structured_output_wire.py fix(bedrock): use native structured output on Haiku 5.5 and Sonnet 5.5, and fix Fable 5 and Sonnet 4.6 schema 400s (#46870) 2026-10-16 20:57:23 -07:00
test_vertex_batch_output_info_wire.py fix(vertex_ai): keep batch output_file_id null until Vertex reports outputInfo (#43030) 2026-09-24 14:05:31 -07:00
test_vertex_batch_row_line_errors_chaos.py test(integration): hand every harness server a port from one per-worker block (#46761) 2026-10-15 16:56:40 -07:00
test_vertex_batch_row_line_errors_owned_proxy.py test(integration): hand every harness server a port from one per-worker block (#46761) 2026-10-15 16:56:40 -07:00
test_vertex_batch_row_line_errors_wire.py test(integration): hand every harness server a port from one per-worker block (#46761) 2026-10-15 16:56:40 -07:00
test_vertex_credentials_source_wire.py fix(vertex): report an unreadable credentials file as a read failure (#40173) 2026-10-15 02:03:56 -07:00
test_vertex_embedding_batch_file_block_wire.py feat(gemini): accept file content blocks with video_metadata on multimodal embeddings (#45305) 2026-10-08 13:20:03 -07:00
test_vertex_embedding_file_block_wire.py feat(gemini): accept file content blocks with video_metadata on multimodal embeddings (#45305) 2026-10-08 13:20:03 -07:00
test_vertex_embedding_wire.py test(integration): cover embeddings endpoint contracts (#44660) 2026-10-14 03:38:12 +00:00
test_vertex_gemini_count_tokens_wire.py fix(count_tokens): apply context_management edits in the Gemini /v1/messages/count_tokens count (#46843) 2026-10-16 17:53:06 +00:00
test_vertex_gemini_fragmented_stream_wire.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_vertex_gemini_function_call_wire.py refactor(types): replace Any with proven types in 7 files (#43704) 2026-09-29 06:12:58 -07:00
test_vertex_partner_count_tokens_wire.py test(integration): hand every harness server a port from one per-worker block (#46761) 2026-10-15 16:56:40 -07:00
test_vertex_partner_messages_wire.py test(integration): cover anthropic_messages endpoint contracts (#44704) 2026-10-14 06:57:54 +00:00
test_vertex_passthrough_wire.py fix(anthropic): keep tool_search_tool_result so the search call replays with its result (#46440) 2026-10-14 18:52:32 -07:00
test_vertex_realtime_multi_region_host.py fix(vertex_ai): dial the multi-region Live API host for realtime sessions (#45166) 2026-10-07 17:39:38 -07:00
test_vertex_timestamps_utc_wire.py fix(vertex_ai): parse Vertex timestamps as UTC so created_at is not shifted by the host timezone (#45550) 2026-10-10 15:18:15 -07:00
test_voyage_mongodb_host_wire.py test(integration): hand every harness server a port from one per-worker block (#46761) 2026-10-15 16:56:40 -07:00
test_xai_batch_output_file_wire.py fix(xai): name a completed xAI batch's results file and stop looking it up on xAI's files API (#45559) 2026-10-14 22:05:43 -07:00
test_xai_server_tool_usage_wire.py refactor(types): replace Any with proven types in 157 files (#44798) 2026-10-06 15:31:54 +00:00
test_xai_web_search_wire.py test(integration): regression tests for August provider translation and streaming bugs (#42621) 2026-09-22 20:52:00 -07:00
test_xiaomi_mimo_wire.py fix(proxy): keep provider refusal, logprobs and system_fingerprint in chat completions (#44788) 2026-10-13 13:17:40 -07:00