scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-11-01 07:03:02 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 129
litellm/tests/integration/providers
History
yuneng-jiang fbd126e518
test: fix integration tests that fail on test order, races and intended changes (#49310) 
Seven integration tests fail on main for reasons that are not product bugs. Two are order or race bugs in the tests, one reads rows other tests wrote, one passed on an error body, and three still expect behavior that #48644, #39864 and #39540 changed on purpose
2026-10-30 13:25:47 -07:00
..
_anthropic_safeguards_beta_support.py fix(rust): add the dangerous-tool-use beta when an anthropic /v1/messages request carries safeguards (#46102) 2026-10-17 16:48:05 +00:00
_api_base_trailing_slash_support.py fix(llms): sanitize trailing slashes in provider get_complete_url (#42887) 2026-10-15 02:59:46 -07:00
_api_base_url_join_support.py test(llms): audit #47535 (#48786) 2026-10-28 03:38:21 -07:00
_assistants_route_errors_support.py test(integration): hand every harness server a port from one per-worker block (#46761) 2026-10-15 16:56:40 -07:00
_cache_control_marks_support.py fix(responses): propagate cache_control through responses->chat completion bridge (#32511) 2026-10-14 15:44:34 -07:00
_coralbricks.py test(integration): hand every harness server a port from one per-worker block (#46761) 2026-10-15 16:56:40 -07:00
_count_tokens_system_lift.py fix(responses)!: count input tokens the way inference sends the request (#47363) 2026-10-20 15:13:59 +00:00
_dashscope_video_support.py feat(dashscope): add Wan 3.0, Wan 2.7 and HappyHorse video generation via /v1/videos (#43706) 2026-10-14 10:10:07 -07:00
_gemini_long_decode_support.py feat(gemini)!: auto-continue gemini-4-argon long decodes with continuationToken (#46240) 2026-10-15 22:54:23 +00:00
_gemini_tts_voice_support.py perf(ci): isolate and shard integration suites (#44146) 2026-10-10 11:15:19 -07:00
_idle_upstream_support.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
_mantle_gpt_prompt_cache_support.py fix(bedrock_mantle): send OpenAI explicit prompt cache breakpoints for GPT-5.6 and newer (#38729) 2026-10-08 02:07:22 -07:00
_responses_bridge_prompt_cache_breakpoint.py fix(responses): keep prompt_cache_breakpoint markers in the chat to responses bridge (#44119) 2026-10-07 16:25:36 -07:00
_sagemaker_chat_container.py fix(sagemaker)!: parse every streamed record however SageMaker cuts the payload parts (#47495) 2026-10-20 09:36:49 +00:00
_tool_search_support.py fix(anthropic): keep tool_search_tool_result so the search call replays with its result (#46440) 2026-10-14 18:52:32 -07:00
_vertex_batch_rows.py test(integration): hand every harness server a port from one per-worker block (#46761) 2026-10-15 16:56:40 -07:00
fine_tuning_proxy_config.yaml test: migrate SDK and proxy-booting legacy CircleCI tests to their tiers (#45758) 2026-10-11 00:17:21 -07:00
openai_fine_tuning.jsonl test: migrate SDK and proxy-booting legacy CircleCI tests to their tiers (#45758) 2026-10-11 00:17:21 -07:00
test_a2a_sdk_upstream_wire.py test: migrate legacy unit dirs and unrun folders into tests/unit, tests/integration and tests/e2e (#45940) 2026-10-11 06:24:57 -07:00
test_anthropic_context_window_stop_wire.py fix(anthropic): map model_context_window_exceeded stop reason to length (#48874) 2026-10-28 13:48:37 -07:00
test_anthropic_count_tokens_system_lift_wire.py fix(responses)!: answer 400 before counting a lone Responses built-in tool on Claude (#48798) 2026-10-28 12:12:45 -07:00
test_anthropic_empty_user_turn_wire.py fix(anthropic): keep trailing user messages with None or empty list content (#41397) 2026-10-16 07:41:05 +00:00
test_anthropic_passthrough_wire.py test(integration): cover pass_through endpoint contracts (#44690) 2026-10-14 07:38:17 +00:00
test_anthropic_refusal_stop_details_chaos.py test(integration): hand every harness server a port from one per-worker block (#46761) 2026-10-15 16:56:40 -07:00
test_anthropic_refusal_stop_details_wire.py test: fix flaky team delete, refusal sink, loop-blocking, Moyai tamper and model-cost leak tests (#47135) 2026-10-20 18:01:36 +00:00
test_anthropic_skills_create_wire.py test(integration): cover anthropic_messages endpoint contracts (#44704) 2026-10-14 06:57:54 +00:00
test_anthropic_system_cache_control_wire.py test(integration): assert /v1/responses usage reports Anthropic system cache write then read (#42855) 2026-09-23 18:52:06 -07:00
test_anthropic_thinking_signature_logging_wire.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_anthropic_thinking_signature_stream_wire.py test: expect the list-position index on streamed thinking blocks in the integration suite (#46618) 2026-10-15 05:52:56 +00:00
test_anthropic_tool_search_result_chaos.py fix(anthropic): keep tool_search_tool_result so the search call replays with its result (#46440) 2026-10-14 18:52:32 -07:00
test_anthropic_tool_search_result_matrix_wire.py fix(anthropic): count a replayed server tool call's cache mark the way modify_params sends it (#46591) 2026-10-15 14:46:43 -07:00
test_anthropic_tool_search_result_modify_params_wire.py fix(anthropic): count a replayed server tool call's cache mark the way modify_params sends it (#46591) 2026-10-15 14:46:43 -07:00
test_anthropic_tool_search_result_passthrough_wire.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_anthropic_tool_search_result_wire.py fix(anthropic): keep tool_search_tool_result so the search call replays with its result (#46440) 2026-10-14 18:52:32 -07:00
test_api_base_trailing_slash_chaos.py fix(llms): sanitize trailing slashes in provider get_complete_url (#42887) 2026-10-15 02:59:46 -07:00
test_api_base_trailing_slash_owned_proxy.py fix(llms): sanitize trailing slashes in provider get_complete_url (#42887) 2026-10-15 02:59:46 -07:00
test_api_base_trailing_slash_wire.py fix(ollama): forward embedding credentials, headers and timeout (#48707) 2026-10-28 09:32:47 -07:00
test_api_base_url_join_chaos.py fix(llms): join api_base and route with one shared helper (#47535) 2026-10-21 06:59:32 +00:00
test_api_base_url_join_host_owned_proxy.py test(llms): audit #47535 (#48786) 2026-10-28 03:38:21 -07:00
test_api_base_url_join_wire.py test(llms): audit #47535 (#48786) 2026-10-28 03:38:21 -07:00
test_assemblyai_passthrough.py test: migrate SDK and proxy-booting legacy CircleCI tests to their tiers (#45758) 2026-10-11 00:17:21 -07:00
test_assistants_evals_wire.py test(integration): pin assistants and evals endpoint contracts (#44734) 2026-10-14 07:08:35 +00:00
test_assistants_route_errors_chaos.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_assistants_route_errors_wire.py test(integration): hand every harness server a port from one per-worker block (#46761) 2026-10-15 16:56:40 -07:00
test_audit_any_sweep_harness_endpoint.py refactor(types): replace Any with proven types in 9 files (#44389) 2026-10-03 21:03:16 +00:00
test_audit_any_sweep_vector_store.py refactor(types): replace Any with proven types in 9 files (#44389) 2026-10-03 21:03:16 +00:00
test_azure_ai_chat_wire.py test(integration): regression tests for August provider translation and streaming bugs (#42621) 2026-09-22 20:52:00 -07:00
test_azure_ai_claude_messages_wire.py test(integration): cover anthropic_messages endpoint contracts (#44704) 2026-10-14 06:57:54 +00:00
test_azure_ai_flux2_image_wire.py test(integration): regression tests for August provider translation and streaming bugs (#42621) 2026-09-22 20:52:00 -07:00
test_azure_ai_rerank_auth_wire.py test(integration): regression tests for August provider translation and streaming bugs (#42621) 2026-09-22 20:52:00 -07:00
test_azure_embedding_wire.py test(integration): cover embeddings endpoint contracts (#44660) 2026-10-14 03:38:12 +00:00
test_azure_entra_token_reuse_wire.py fix(azure): share one Entra token request per worker and audit #47348 (#48769) 2026-10-28 12:24:20 -07:00
test_azure_reasoning_effort_none_wire.py test(integration): repair seven stale integration tests on main (#47600) 2026-10-20 11:21:04 -07:00
test_azure_responses_api_version_wire.py fix(providers): route Azure Responses past dated api versions, count Grok reasoning tokens on azure_ai, refresh provider sets after register_model, fix gpt-6.1-sol Azure rows (#46377) 2026-10-16 15:23:46 -07:00
test_bedrock_anthropic_beta_header_wire.py refactor(types): replace Any with proven types in 157 files (#44798) 2026-10-06 15:31:54 +00:00
test_bedrock_auth_wire.py test(integration): regression tests for August provider translation and streaming bugs (#42621) 2026-09-22 20:52:00 -07:00
test_bedrock_batch_blank_s3_env_wire.py test(e2e): assert only litellm-owned batch behavior and move the blank S3 env pin to an integration test (#43321) 2026-09-26 11:17:03 -07:00
test_bedrock_batch_files_wire.py fix(bedrock): send native structured output first on Opus 4.8 and Sonnet 5 and fall back to the json tool when Bedrock rejects it (#47100) 2026-10-18 00:06:07 +00:00
test_bedrock_batch_retrieve_status_wire.py fix(bedrock): map Scheduled batch jobs to in_progress on retrieve (#45157) 2026-10-07 18:08:51 -07:00
test_bedrock_claude_thinking_wire.py fix(anthropic): drop unknown legacy thinking keys under drop_params when rebuilding adaptive thinking (#46741) 2026-10-15 23:09:30 +00:00
test_bedrock_converse_client_metadata_wire.py test(integration): regression tests for July provider translation, routing and streaming bugs (#42693) 2026-09-23 09:51:06 -07:00
test_bedrock_converse_config_blocks_wire.py test(integration): regression tests for August provider translation and streaming bugs (#42621) 2026-09-22 20:52:00 -07:00
test_bedrock_converse_lookaround_regex_chaos.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_bedrock_converse_lookaround_regex_wire.py test: update stale integration and guardrail e2e tests to intended behavior (#48665) 2026-10-28 01:11:25 +00:00
test_bedrock_converse_missing_content_wire.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_bedrock_converse_redacted_thinking_wire.py test(anthropic, bedrock): audit #47489 (#48782) 2026-10-28 04:24:50 -07:00
test_bedrock_converse_stream_event_frames_wire.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_bedrock_deepseek_reasoning_wire.py test(integration): regression tests for July provider translation, routing and streaming bugs (#42693) 2026-09-23 09:51:06 -07:00
test_bedrock_embedding_wire.py test(integration): regression tests for August provider translation and streaming bugs (#42621) 2026-09-22 20:52:00 -07:00
test_bedrock_gpt5_reasoning_wire.py test(integration): regression tests for August provider translation and streaming bugs (#42621) 2026-09-22 20:52:00 -07:00
test_bedrock_gpt_responses_native_wire.py feat(bedrock): serve gpt-5.6+ chat completions natively by default, with chat_completions/ opt-in for gpt-oss and grok (#44307) 2026-10-02 19:55:54 -07:00
test_bedrock_invoke_cache_usage_wire.py test(integration): regression tests for July provider translation, routing and streaming bugs (#42693) 2026-09-23 09:51:06 -07:00
test_bedrock_invoke_dangerous_tool_use_beta_wire.py fix(bedrock): drop dangerous-tool-use beta without safeguards (#44471) 2026-10-16 21:00:37 +00:00
test_bedrock_kimi_k3_cache_point_wire.py fix(proxy)!: persist sent model_info fields on POST /model/update (#46902) 2026-10-30 12:10:13 -07:00
test_bedrock_knowledge_base_user_context_wire.py test(integration): regression tests for July provider translation, routing and streaming bugs (#42693) 2026-09-23 09:51:06 -07:00
test_bedrock_mantle_claude_chat_chaos.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_bedrock_mantle_claude_chat_wire.py fix(streaming)!: send the include_usage chunk with an empty choices array (#47109) 2026-10-21 02:26:05 +00:00
test_bedrock_mantle_codex_input_wire.py test(integration): regression tests for July provider translation, routing and streaming bugs (#42693) 2026-09-23 09:51:06 -07:00
test_bedrock_mantle_context_overflow_fallbacks_wire.py fix(router): recover streamed context window errors before output (#49293) 2026-10-30 13:19:58 -07:00
test_bedrock_mantle_context_overflow_wire.py test(integration): hand every harness server a port from one per-worker block (#46761) 2026-10-15 16:56:40 -07:00
test_bedrock_mantle_count_tokens_wire.py fix(responses)!: answer 400 before counting a lone Responses built-in tool on Claude (#48798) 2026-10-28 12:12:45 -07:00
test_bedrock_mantle_gpt_prompt_cache_breakpoint_wire.py test(integration): run the bedrock mantle prompt cache and output limit wire tests on their own proxy (#47152) 2026-10-17 13:51:31 -07:00
test_bedrock_mantle_output_limit_wire.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_bedrock_mantle_responses_wire.py test(integration): regression tests for July provider translation, routing and streaming bugs (#42693) 2026-09-23 09:51:06 -07:00
test_bedrock_mantle_wire.py test(integration): regression tests for July provider translation, routing and streaming bugs (#42693) 2026-09-23 09:51:06 -07:00
test_bedrock_marengo_embed_3_wire.py test(integration): regression tests for July provider translation, routing and streaming bugs (#42693) 2026-09-23 09:51:06 -07:00
test_bedrock_model_discovery_chaos.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_bedrock_model_discovery_wire.py fix(bedrock): list the account's invocable models behind bedrock/* when check_provider_endpoint is on (#43981) 2026-10-09 14:14:17 -07:00
test_bedrock_native_guardrail_trace_spend.py feat(bedrock): price native guardrailConfig trace usage into spend and x-litellm-response-cost (#48278) 2026-10-27 12:10:45 -07:00
test_bedrock_native_structured_output_wire.py fix(bedrock): send native structured output first on Opus 4.8 and Sonnet 5 and fall back to the json tool when Bedrock rejects it (#47100) 2026-10-18 00:06:07 +00:00
test_bedrock_nova_sonic_realtime_tool_calls.py fix(cost): price exact cost-map names before price-free capability rules, and fix SCIM and MCP integration tests on CircleCI (#47037) 2026-10-17 06:46:52 +00:00
test_bedrock_passthrough_aws_errors.py fix(bedrock-passthrough)!: return AWS-shaped errors so AWS SDKs read the code and message (#46522) 2026-10-15 22:05:46 -07:00
test_bedrock_passthrough_guardrail_config_wire.py fix(bedrock)!: apply the deployment guardrailConfig on /bedrock pass-through routes (#47429) 2026-10-20 14:48:19 -07:00
test_bedrock_passthrough_request_metadata_wire.py feat(bedrock): apply bedrock_request_metadata_fields on the /bedrock pass-through routes (#47040) 2026-10-17 14:01:50 +00:00
test_bedrock_passthrough_stream_wire.py test(integration): cover pass_through endpoint contracts (#44690) 2026-10-14 07:38:17 +00:00
test_bedrock_rerank_wire.py test(integration): regression tests for August provider translation and streaming bugs (#42621) 2026-09-22 20:52:00 -07:00
test_bedrock_role_configuration.py test(integration): regression tests for July provider translation, routing and streaming bugs (#42693) 2026-09-23 09:51:06 -07:00
test_bedrock_runtime_chat_completions_chaos.py test(integration): hand every harness server a port from one per-worker block (#46761) 2026-10-15 16:56:40 -07:00
test_bedrock_runtime_chat_completions_sad_wire.py test(integration): fix seven CircleCI integration tests that race or assume a worker spread (#47001) 2026-10-16 22:45:29 +00:00
test_bedrock_runtime_chat_completions_wire.py fix(bedrock): drop stop sequences for GPT 5.6 and newer instead of forwarding them to a 400 (#45664) 2026-10-09 15:00:24 -07:00
test_bedrock_stream_timeout_chaos.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_bedrock_stream_timeout_wire.py fix(router)!: bound fake-streamed calls by the request timeout, not stream_timeout (#47162) 2026-10-17 22:08:02 +00:00
test_bedrock_thinking_tokens_wire.py test(integration): regression tests for July provider translation, routing and streaming bugs (#42693) 2026-09-23 09:51:06 -07:00
test_cache_control_tool_call_marks_owned_proxy.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_cache_control_tool_call_marks_wire.py fix(vertex_ai): build the cachedContents collection URL under a custom api_base (#44783) 2026-10-29 03:46:35 -07:00
test_chat_completions_tool_search_chaos.py test(integration): hand every harness server a port from one per-worker block (#46761) 2026-10-15 16:56:40 -07:00
test_chat_completions_tool_search_history_wire.py fix(anthropic)!: translate the OpenAI tool_search tool on /v1/chat/completions for Claude models (#46389) 2026-10-15 04:47:05 +00:00
test_chat_completions_tool_search_wire.py fix(anthropic)!: translate the OpenAI tool_search tool on /v1/chat/completions for Claude models (#46389) 2026-10-15 04:47:05 +00:00
test_chat_provider_annotations_wire.py fix(anthropic)!: send /v1/messages PDF documents to OpenAI-shaped providers as file parts (#47166) 2026-10-29 08:06:12 -07:00
test_chat_template_tool_count_wire.py fix(token_counter): count DeepSeek V4 text with DeepSeek's own tokenizer (#47353) 2026-10-20 03:43:35 +00:00
test_chatgpt_responses_caller_stream_chaos.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_chatgpt_responses_caller_stream_wire.py test: update stale integration and guardrail e2e tests to intended behavior (#48665) 2026-10-28 01:11:25 +00:00
test_chatgpt_responses_top_level_error_wire.py fix(responses): keep the provider code and message on top-level stream error events (#46438) 2026-10-14 16:16:46 -07:00
test_claude_thinking_content_replay_wire.py test(anthropic, bedrock): audit #47489 (#48782) 2026-10-28 04:24:50 -07:00
test_cloudflare_clairic_extraction_wire.py test(extraction): audit #47473 (#48898) 2026-10-28 17:12:07 -07:00
test_cloudflare_clef_images_chaos.py feat(decisions)!: send images to Cloudflare Clef through its images field (#45734) 2026-10-14 12:14:48 +00:00
test_cloudflare_clef_images_wire.py feat(decisions)!: send images to Cloudflare Clef through its images field (#45734) 2026-10-14 12:14:48 +00:00
test_cloudflare_clef_omni_media_chaos.py test(integration): hand every harness server a port from one per-worker block (#46761) 2026-10-15 16:56:40 -07:00
test_cloudflare_clef_omni_media_wire.py feat(decisions): support Clef-omni audio and video (#45811) 2026-10-12 20:29:49 +00:00
test_container_file_routes_wire.py refactor(types): replace Any with proven types in 157 files (#44798) 2026-10-06 15:31:54 +00:00
test_coralbricks_provider_chaos.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_coralbricks_provider_env_wire.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_coralbricks_provider_wire.py test(integration): hand every harness server a port from one per-worker block (#46761) 2026-10-15 16:56:40 -07:00
test_count_tokens_error_shapes_wire.py test(proxy): audit #46815 (#48768) 2026-10-28 03:55:25 -07:00
test_count_tokens_model_wire.py fix(count_tokens)!: audit #47441 and keep the framed estimate for Bedrock count models from the cost map (#48805) 2026-10-28 19:11:35 +00:00
test_cursor_responses_wire.py fix(proxy)!: log and bill Responses-style bodies sent to /cursor/chat/completions (#30277) 2026-10-30 00:18:15 -07:00
test_custom_prompt_template_chaos.py fix(prompt-templates): treat initial_prompt_value and final_prompt_value as optional (#39762) 2026-10-16 10:30:59 +00:00
test_custom_prompt_template_wire.py fix(prompt-templates): treat initial_prompt_value and final_prompt_value as optional (#39762) 2026-10-16 10:30:59 +00:00
test_dashscope_chat_wire.py fix(proxy): keep provider refusal, logprobs and system_fingerprint in chat completions (#44788) 2026-10-13 13:17:40 -07:00
test_dashscope_image_edit_and_speech_wire.py feat(dashscope): add image edits and text-to-speech (#48998) 2026-10-29 17:48:38 -07:00
test_dashscope_video_chaos.py feat(dashscope): add Wan 3.0, Wan 2.7 and HappyHorse video generation via /v1/videos (#43706) 2026-10-14 10:10:07 -07:00
test_dashscope_video_wire.py feat(dashscope): add Wan 3.0, Wan 2.7 and HappyHorse video generation via /v1/videos (#43706) 2026-10-14 10:10:07 -07:00
test_databricks_agent_wire.py feat(agents): support Databricks Agent Platform agents in the agent gateway (#45681) 2026-10-15 12:42:06 -07:00
test_databricks_chat_wire.py fix(databricks): stream Claude json_schema answers as text on every streaming path (#47424) 2026-10-20 02:41:55 +00:00
test_databricks_decisions_wire.py test: update integration tests left stale by recent product changes (#49143) 2026-10-29 18:06:28 -07:00
test_databricks_oauth_wire.py test(integration): regression tests for August provider translation and streaming bugs (#42621) 2026-09-22 20:52:00 -07:00
test_decision_model_guardrail.py test(integration): stop GET /__observations from draining the upstream log (#47256) 2026-10-18 09:58:05 +00:00
test_decisions_box_deployment_flag_chaos.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_decisions_box_deployment_flag_wire.py fix(decisions): read a deployment's own supports_box_questions flag before the cost map (#46696) 2026-10-15 12:34:36 -07:00
test_decisions_chaos.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_decisions_openai_format_wire.py test: update integration tests left stale by recent product changes (#49143) 2026-10-29 18:06:28 -07:00
test_decisions_wire.py feat(decisions): support Ollama decision models (#48871) 2026-10-28 12:35:26 -07:00
test_deepseek_vision_wire.py test(integration): stop GET /__observations from draining the upstream log (#47256) 2026-10-18 09:58:05 +00:00
test_device_code_login_guard_boot.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_device_code_login_guard_wire.py fix(github_copilot): treat a malformed api-key.json like a missing one (#44723) 2026-10-29 03:46:44 -07:00
test_embedding_cache_partial_hit_wire.py test(integration): cover embeddings endpoint contracts (#44660) 2026-10-14 03:38:12 +00:00
test_embedding_token_array_wire.py test(integration): cover embeddings endpoint contracts (#44660) 2026-10-14 03:38:12 +00:00
test_fake_openai_upstream_routes.py test: migrate local_testing SDK and provider tests out of build_and_test (#45760) 2026-10-10 23:14:25 +00:00
test_fal_ai_chat_wire.py fix(fal_ai): honour global api_base for image generation and reject non-string reasoning_effort with 400 (#42512) 2026-09-22 13:13:54 -07:00
test_fal_ai_image_wire.py test(integration): cover images endpoint contracts (#44653) 2026-10-14 03:40:29 +00:00
test_fal_ai_passthrough_wire.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_fal_ai_video_wire.py fix(fal_ai): reuse the status client and headers on the video result probe (#42511) 2026-09-22 13:12:51 -07:00
test_files_batches_wire.py test(integration): cover files_batches endpoint contracts (#44703) 2026-10-14 06:57:41 +00:00
test_fireworks_ai_router_slug_wire.py feat(fireworks_ai): route and list the auto, auto-instant and firerouter routers (#43641) 2026-09-28 19:08:16 -07:00
test_fireworks_ai_session_affinity_wire.py test(integration): regression tests for July provider translation, routing and streaming bugs (#42693) 2026-09-23 09:51:06 -07:00
test_friendliai_replay_wire.py fix(friendliai): drop litellm-internal message fields before sending (#45092) 2026-10-20 01:22:07 +00:00
test_gemini_agents_wire.py fix(gemini)!: accept Google's current Managed Agents fields on create and interactions (#48750) 2026-10-28 12:37:59 -07:00
test_gemini_anyof_null_branches_wire.py fix(vertex_ai)!: send integer enums to Gemini instead of dropping them (#44908) 2026-10-20 17:39:51 +00:00
test_gemini_chat_video_metadata_wire.py feat(gemini): accept file content blocks with video_metadata on multimodal embeddings (#45305) 2026-10-08 13:20:03 -07:00
test_gemini_context_caching_custom_api_base_wire.py fix(vertex_ai): build the cachedContents collection URL under a custom api_base (#44783) 2026-10-29 03:46:35 -07:00
test_gemini_default_output_budget_wire.py test(gemini): audit #47542 (#48758) 2026-10-28 02:42:45 -07:00
test_gemini_embedding_file_block_chaos.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_gemini_embedding_file_block_wire.py feat(gemini): accept file content blocks with video_metadata on multimodal embeddings (#45305) 2026-10-08 13:20:03 -07:00
test_gemini_grounding_annotations_wire.py test(gemini): cover every web source for a grounded passage (#48861) 2026-10-28 16:26:28 -07:00
test_gemini_image_config_and_usage_wire.py refactor(types): replace Any with proven types in 157 files (#44798) 2026-10-06 15:31:54 +00:00
test_gemini_interactions_wire.py test(integration): cover google_genai endpoint contracts (#44680) 2026-10-14 04:01:35 +00:00
test_gemini_long_decode_chaos.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_gemini_long_decode_wire.py test(gemini): audit #47542 (#48758) 2026-10-28 02:42:45 -07:00
test_gemini_omitted_temperature.py test(translation): share rich fixtures across Claude, GPT and Gemini providers (#46585) 2026-10-14 19:53:09 -07:00
test_gemini_thinking_replay_chaos.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_gemini_thinking_replay_wire.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_gemini_tool_result_part_reference_chaos.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_gemini_tool_result_part_reference_wire.py test: update stale integration and guardrail e2e tests to intended behavior (#48665) 2026-10-28 01:11:25 +00:00
test_gemini_tts_voice_mapping_owned_proxy.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_gemini_tts_voice_mapping_wire.py test: fix integration tests that fail on test order, races and intended changes (#49310) 2026-10-30 13:25:47 -07:00
test_google_genai_adapter_wire.py fix(vertex_ai): route Model Garden models through the generateContent adapter (#41836) 2026-10-15 08:06:49 -07:00
test_google_genai_native_wire.py test(integration): cover google_genai endpoint contracts (#44680) 2026-10-14 04:01:35 +00:00
test_google_genai_proxy_sdk_wire.py test: migrate SDK and proxy-booting legacy CircleCI tests to their tiers (#45758) 2026-10-11 00:17:21 -07:00
test_google_genai_vertex_routes_wire.py fix(vertex): match standalone 403 status tokens instead of any substring (#47134) 2026-10-17 19:17:18 +00:00
test_hosted_vllm_reasoning_content_chaos.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_hosted_vllm_reasoning_content_wire.py test: cover response contracts across translation endpoints (#46632) 2026-10-15 00:05:09 -07:00
test_hosted_vllm_stream_stall_wire.py test: fix integration tests that fail on test order, races and intended changes (#49310) 2026-10-30 13:25:47 -07:00
test_huggingface_rerank_truncate_wire.py fix(huggingface): stop forcing truncate=false on TEI rerank and pass truncate params through (#48706) 2026-10-28 09:27:26 -07:00
test_huggingface_tokenizer_downloads_wire.py test(token_counter): audit #47586 (#48800) 2026-10-28 04:30:19 -07:00
test_idle_upstream_guardrail_cut_wire.py feat(guardrails): add num_retries to generic_guardrail_api for checks lost after connect (#46965) 2026-10-17 12:22:41 -07:00
test_idle_upstream_keepalive_edges_wire.py test: update integration tests left stale by recent product changes (#49143) 2026-10-29 18:06:28 -07:00
test_idle_upstream_keepalive_override_wire.py test: update integration tests left stale by recent product changes (#49143) 2026-10-29 18:06:28 -07:00
test_idle_upstream_reconnect_chaos_wire.py test(integration): hand every harness server a port from one per-worker block (#46761) 2026-10-15 16:56:40 -07:00
test_idle_upstream_reconnect_routes_wire.py fix(http): restore the 15s aiohttp keep-alive default in the Python and Rust clients (#46355) 2026-10-14 10:50:11 -07:00
test_idle_upstream_reconnect_wire.py fix(http): restore the 15s aiohttp keep-alive default in the Python and Rust clients (#46355) 2026-10-14 10:50:11 -07:00
test_image_gen_drop_params_wire.py fix(images): honor max_retries and prevent Router retry amplification (#48705) 2026-10-28 10:46:47 -07:00
test_inferenceapis_context_window_fallbacks_wire.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_inferenceapis_wire.py test: update integration tests left stale by recent product changes (#49143) 2026-10-29 18:06:28 -07:00
test_interactions_openai_bridge_wire.py refactor(types): replace Any with proven types in 157 files (#44798) 2026-10-06 15:31:54 +00:00
test_internal_params_wire.py fix(params): validate stream_chunk_size once, before any provider call (#43222) 2026-09-26 23:01:20 +00:00
test_lm_studio_chat_wire.py test: migrate local_testing SDK and provider tests out of build_and_test (#45760) 2026-10-10 23:14:25 +00:00
test_malformed_tools_chaos.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_malformed_tools_wire.py fix(responses)!: count input tokens the way inference sends the request (#47363) 2026-10-20 15:13:59 +00:00
test_moderations.py test: migrate SDK and proxy-booting legacy CircleCI tests to their tiers (#45758) 2026-10-11 00:17:21 -07:00
test_moderations_wire.py test(integration): cover small_inference endpoint contracts (#44654) 2026-10-14 04:06:15 +00:00
test_nvidia_nim_ranking_wire.py test(integration): regression tests for July provider translation, routing and streaming bugs (#42693) 2026-09-23 09:51:06 -07:00
test_nvidia_riva_wire.py test(nvidia_riva): audit #41021 (#48801) 2026-10-28 05:24:08 -07:00
test_ocr_multipart_wire.py test(integration): cover small_inference endpoint contracts (#44654) 2026-10-14 04:06:15 +00:00
test_ocr_router_wire.py test: move 81 legacy live tests in pass-through, spend, batches, audio, search, guardrails, image and ocr dirs offline (#45288) 2026-10-08 02:38:12 -07:00
test_ollama_embedding_wire.py fix(ollama): forward embedding credentials, headers and timeout (#48707) 2026-10-28 09:32:47 -07:00
test_ollama_prompt_tools_chaos.py fix(prompt_templates): do not mutate messages in-place in function_call_prompt (#41676) 2026-10-15 14:37:11 -07:00
test_ollama_prompt_tools_wire.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_openai_chat_prompt_cache_breakpoint_wire.py fix(openai): carry cache_control to prompt_cache_breakpoint on chat completions to gpt-5.6 (#46101) 2026-10-13 15:45:49 -07:00
test_openai_chat_wire.py test(integration): cover chat_completions endpoint contracts (#44688) 2026-10-14 03:45:38 +00:00
test_openai_completion_wire.py test(integration): cover chat_completions endpoint contracts (#44688) 2026-10-14 03:45:38 +00:00
test_openai_count_tokens_tools_wire.py fix(responses)!: answer 400 before counting a lone Responses built-in tool on Claude (#48798) 2026-10-28 12:12:45 -07:00
test_openai_dialect_prompt_cache_breakpoint_owned_proxy.py fix(azure_ai)!: keep OpenAI prompt cache breakpoints on the Foundry chat completions path (#45299) 2026-10-29 23:49:55 -07:00
test_openai_embedding_timeout_wire.py test: migrate local_testing SDK and provider tests out of build_and_test (#45760) 2026-10-10 23:14:25 +00:00
test_openai_embedding_wire.py test(integration): cover embeddings endpoint contracts (#44660) 2026-10-14 03:38:12 +00:00
test_openai_error_handling.py test: migrate SDK and proxy-booting legacy CircleCI tests to their tiers (#45758) 2026-10-11 00:17:21 -07:00
test_openai_fine_tuning.py test: migrate SDK and proxy-booting legacy CircleCI tests to their tiers (#45758) 2026-10-11 00:17:21 -07:00
test_openai_image_edit_wire.py test(integration): cover images endpoint contracts (#44653) 2026-10-14 03:40:29 +00:00
test_openai_image_variation_wire.py test(integration): cover images endpoint contracts (#44653) 2026-10-14 03:40:29 +00:00
test_openai_like_response_headers_wire.py fix(prometheus): set the remaining rate-limit gauges from decimal headers again (#47039) 2026-10-17 05:20:00 +00:00
test_openai_passthrough_files_wire.py test: move 81 legacy live tests in pass-through, spend, batches, audio, search, guardrails, image and ocr dirs offline (#45288) 2026-10-08 02:38:12 -07:00
test_openai_passthrough_multipart_wire.py test(integration): cover pass_through endpoint contracts (#44690) 2026-10-14 07:38:17 +00:00
test_openai_realtime_transcription_url.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_openai_responses_prompt_cache_breakpoint_wire.py fix(openai): carry cache_control to prompt_cache_breakpoint on /v1/responses to gpt-5.6 (#46105) 2026-10-13 15:23:07 -07:00
test_openai_responses_tool_search_wire.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_openai_responses_websocket_wire.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_openai_stream_text_usage_wire.py refactor(types): replace Any with proven types in 7 files (#43704) 2026-09-29 06:12:58 -07:00
test_pdf_input_support_wire.py fix(anthropic)!: send /v1/messages PDF documents to OpenAI-shaped providers as file parts (#47166) 2026-10-29 08:06:12 -07:00
test_provider_lookup_status.py test(integration): stop GET /__observations from draining the upstream log (#47256) 2026-10-18 09:58:05 +00:00
test_rag_ingest_wire.py test(integration): cover vector_stores endpoint contracts (#44739) 2026-10-14 03:59:08 +00:00
test_realtime_transcription_guardrail_session_update.py fix(realtime)!: send the Gemini Live setup at the first content so session.update settings apply (#39864) 2026-10-30 01:31:39 -07:00
test_realtime_unguarded_voice_turns.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_realtime_webrtc_http.py test(integration): cover realtime endpoint contracts (#44689) 2026-10-14 03:49:34 +00:00
test_request_boundary.py test(integration): stop GET /__observations from draining the upstream log (#47256) 2026-10-18 09:58:05 +00:00
test_rerank_latency_headers_wire.py test(integration): cover small_inference endpoint contracts (#44654) 2026-10-14 04:06:15 +00:00
test_responses_bridge_history_replay_wire.py fix(responses): replay the original Responses item order, phase, and boundaries on the next chat turn (#48912) 2026-10-29 00:56:02 -07:00
test_responses_bridge_incomplete.py fix(responses): replay the original Responses item order, phase, and boundaries on the next chat turn (#48912) 2026-10-29 00:56:02 -07:00
test_responses_bridge_input_audio_wire.py fix(responses): map input_audio blocks in the chat-to-Responses bridge (#45224) 2026-10-07 22:25:35 -07:00
test_responses_bridge_item_marks_guardrail_wire.py fix(responses): propagate cache_control through responses->chat completion bridge (#32511) 2026-10-14 15:44:34 -07:00
test_responses_bridge_message_cache_control_wire.py feat(count_tokens)!: configurable token_count_framing and token_count_model, count source in the response (#46777) 2026-10-17 17:11:27 -07:00
test_responses_bridge_message_cache_control_wire_chaos.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_responses_bridge_namespace_tools.py test(integration): regression tests for July provider translation, routing and streaming bugs (#42693) 2026-09-23 09:51:06 -07:00
test_responses_bridge_prompt_cache_breakpoint.py fix(responses): keep prompt_cache_breakpoint markers in the chat to responses bridge (#44119) 2026-10-07 16:25:36 -07:00
test_responses_bridge_prompt_cache_breakpoint_chaos.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_responses_bridge_prompt_cache_breakpoint_wire.py fix(responses): drop a malformed prompt_cache_breakpoint under drop_params on the direct /v1/responses path (#45330) 2026-10-29 03:37:54 -07:00
test_responses_bridge_prompt_cache_breakpoint_wire_chaos.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_responses_bridge_stream_output_items_wire.py fix(ollama): one stream id, 400 on non-text tool content, separated tool results, no empty message item (#45177) 2026-10-07 19:08:26 -07:00
test_responses_bridge_tool_search_chaos.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_responses_bridge_tool_search_replay.py fix(responses bridge): round-trip tool_search_call and tool_search_output items between turns (#46444) 2026-10-14 21:40:14 -07:00
test_responses_bridge_tool_search_replay_wire.py fix(responses): translate OpenAI tool_search to Anthropic tool search for Claude models (#45882) 2026-10-14 20:42:22 -07:00
test_responses_bridge_tool_search_undeferred_wire.py test: update integration tests left stale by recent product changes (#49143) 2026-10-29 18:06:28 -07:00
test_responses_bridge_tool_search_wire.py test: update integration tests left stale by recent product changes (#49143) 2026-10-29 18:06:28 -07:00
test_responses_client_header_forwarding_wire.py test(integration): regression tests for July provider translation, routing and streaming bugs (#42693) 2026-09-23 09:51:06 -07:00
test_responses_codex_repeated_additional_tools_wire.py fix(responses): only merge repeated Codex tools when drop_params is on (#49112) 2026-10-29 12:55:29 -07:00
test_responses_compact_wire.py fix(responses): drop the null created_by OpenAI rejects when output items are replayed (#48783) 2026-10-28 21:11:49 -07:00
test_responses_compaction_items_wire.py fix(responses)!: round-trip Anthropic compaction blocks as compaction items (#46754) 2026-10-21 08:50:19 +00:00
test_responses_core_wire.py fix(responses)!: count input tokens the way inference sends the request (#47363) 2026-10-20 15:13:59 +00:00
test_responses_error_status_wire.py test: move 81 legacy live tests in pass-through, spend, batches, audio, search, guardrails, image and ocr dirs offline (#45288) 2026-10-08 02:38:12 -07:00
test_responses_lifecycle_wire.py fix(responses): resume background response streams with stream and starting_after (#48819) 2026-10-29 02:31:22 -07:00
test_responses_minted_reasoning_replay_chaos.py test(integration): hand every harness server a port from one per-worker block (#46761) 2026-10-15 16:56:40 -07:00
test_responses_minted_reasoning_replay_wire.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_responses_reasoning_item_wire.py refactor(types): replace Any with proven types in 157 files (#44798) 2026-10-06 15:31:54 +00:00
test_responses_websocket_session_limit.py test(integration): wait for every worker to report the websocket session cap instead of sleeping (#47412) 2026-10-19 15:35:29 -07:00
test_rust_anthropic_safeguards_beta_chaos.py test: update integration tests left stale by recent product changes (#49143) 2026-10-29 18:06:28 -07:00
test_rust_anthropic_safeguards_beta_wire.py test: update integration tests left stale by recent product changes (#49143) 2026-10-29 18:06:28 -07:00
test_s3_wire.py test: fix stale and flaky tests across CircleCI, GHA and Buildkite (#45530) 2026-10-09 11:40:14 -07:00
test_sagemaker_chat_credential_sources.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_sagemaker_chat_wire.py fix(router)!: keep a deployment's own model_id over the request body's (#49045) 2026-10-30 02:40:01 -07:00
test_sagemaker_embedding_chaos.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_sagemaker_embedding_wire.py fix(router)!: keep a deployment's own model_id over the request body's (#49045) 2026-10-30 02:40:01 -07:00
test_search_wire.py fix(tavily): return raw_content, score, favicon and the response envelope from Tavily search (#48684) 2026-10-27 18:52:06 -07:00
test_strands_decider_agentcore_wire.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_tencent_chat_wire.py fix(proxy): keep provider refusal, logprobs and system_fingerprint in chat completions (#44788) 2026-10-13 13:17:40 -07:00
test_tool_config_cache_point_wire.py fix(anthropic): apply tool_config cache-control points on chat and /v1/messages (#47394) 2026-10-19 23:26:41 +00:00
test_validated_reply_boundaries_under_faults.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_vector_store_create_wire.py test(integration): remove BUG skips for 3 tests that pass on main (#47364) 2026-10-19 21:39:35 -07:00
test_vector_store_search_wire.py test(integration): cover vector_stores endpoint contracts (#44739) 2026-10-14 03:59:08 +00:00
test_vector_store_team_model_provider_wire.py fix(proxy): route team key vector store calls to a team model of the store's provider (#46894) 2026-10-17 14:32:03 -05:00
test_vertex_anthropic_inline_tools_beta_wire.py test(integration): remove six setup causes of CircleCI integration reds (#46490) 2026-10-14 12:00:35 -07:00
test_vertex_anthropic_structured_output_wire.py test(anthropic): audit #47478 (#48803) 2026-10-28 04:34:21 -07:00
test_vertex_batch_output_info_wire.py fix(vertex_ai): keep batch output_file_id null until Vertex reports outputInfo (#43030) 2026-09-24 14:05:31 -07:00
test_vertex_batch_row_line_errors_chaos.py test(integration): hand every harness server a port from one per-worker block (#46761) 2026-10-15 16:56:40 -07:00
test_vertex_batch_row_line_errors_owned_proxy.py test(integration): hand every harness server a port from one per-worker block (#46761) 2026-10-15 16:56:40 -07:00
test_vertex_batch_row_line_errors_wire.py test(integration): hand every harness server a port from one per-worker block (#46761) 2026-10-15 16:56:40 -07:00
test_vertex_credentials_source_wire.py test: update integration tests left stale by recent product changes (#49143) 2026-10-29 18:06:28 -07:00
test_vertex_embedding_batch_file_block_wire.py feat(gemini): accept file content blocks with video_metadata on multimodal embeddings (#45305) 2026-10-08 13:20:03 -07:00
test_vertex_embedding_file_block_wire.py feat(gemini): accept file content blocks with video_metadata on multimodal embeddings (#45305) 2026-10-08 13:20:03 -07:00
test_vertex_embedding_wire.py test(integration): cover embeddings endpoint contracts (#44660) 2026-10-14 03:38:12 +00:00
test_vertex_gemini_count_tokens_wire.py fix(responses)!: count input tokens the way inference sends the request (#47363) 2026-10-20 15:13:59 +00:00
test_vertex_gemini_fragmented_stream_wire.py test: update stale integration and guardrail e2e tests to intended behavior (#48665) 2026-10-28 01:11:25 +00:00
test_vertex_gemini_function_call_wire.py test: update stale integration and guardrail e2e tests to intended behavior (#48665) 2026-10-28 01:11:25 +00:00
test_vertex_partner_count_tokens_wire.py fix(responses)!: count input tokens the way inference sends the request (#47363) 2026-10-20 15:13:59 +00:00
test_vertex_partner_labels_wire.py fix(vertex_ai): send Vertex Claude labels as X-Vertex-AI-Labels header instead of the request body (#48753) 2026-10-28 15:41:50 +00:00
test_vertex_partner_messages_wire.py test(integration): cover anthropic_messages endpoint contracts (#44704) 2026-10-14 06:57:54 +00:00
test_vertex_passthrough_wire.py feat(vertex_ai): let operators drop chosen anthropic-beta values on the Vertex AI pass-through routes (#47185) 2026-10-18 12:51:27 +00:00
test_vertex_realtime_multi_region_host.py fix(vertex_ai): dial the multi-region Live API host for realtime sessions (#45166) 2026-10-07 17:39:38 -07:00
test_vertex_timestamps_utc_wire.py fix(vertex_ai): parse Vertex timestamps as UTC so created_at is not shifted by the host timezone (#45550) 2026-10-10 15:18:15 -07:00
test_voyage_mongodb_host_wire.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_xai_batch_output_file_wire.py test(integration): stop GET /__observations from draining the upstream log (#47256) 2026-10-18 09:58:05 +00:00
test_xai_server_tool_usage_wire.py refactor(types): replace Any with proven types in 157 files (#44798) 2026-10-06 15:31:54 +00:00
test_xai_web_search_wire.py test(integration): regression tests for August provider translation and streaming bugs (#42621) 2026-09-22 20:52:00 -07:00
test_xiaomi_mimo_wire.py fix(proxy): keep provider refusal, logprobs and system_fingerprint in chat completions (#44788) 2026-10-13 13:17:40 -07:00