scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-30 06:43:27 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 249
litellm/tests/integration/providers
History
mateo-berri efc5e1e8d3 fix(anthropic): settle breakpoint ttls in linear time and check every breakpoint a client mark wins over 
Settling ttls scanned the message list once per mark, so thousands of marked system turns or 1h parts
took tens of seconds. Run ends and 1h reach are now computed in one pass each.

A breakpoint a client cache_control wins over is now still checked, so a ttl Claude lacks or a malformed
marker answers 400 without drop_params like every other breakpoint. A tool's breakpoint joined with its
function's keeps a 1h ttl the function asked for, and the replaced one is checked too.
2026-10-16 00:25:57 +00:00
..
_api_base_trailing_slash_support.py fix(llms): sanitize trailing slashes in provider get_complete_url (#42887) 2026-10-15 02:59:46 -07:00
_assistants_route_errors_support.py fix(proxy): keep the mapped status on assistants and threads route errors (#44994) 2026-10-09 16:55:15 -07:00
_cache_control_marks_support.py fix(responses): propagate cache_control through responses->chat completion bridge (#32511) 2026-10-14 15:44:34 -07:00
_claude_prompt_cache_breakpoint_support.py fix(anthropic): settle breakpoint ttls around moved system messages and Converse fallbacks 2026-10-15 23:19:20 +00:00
_coralbricks.py feat(providers): add CoralBricks as a native provider (#44700) 2026-10-12 21:56:59 +00:00
_count_tokens_system_lift.py fix(anthropic): count a leading system run through count_tokens' system parameter (#45463) 2026-10-08 22:34:25 -07:00
_dashscope_video_support.py feat(dashscope): add Wan 3.0, Wan 2.7 and HappyHorse video generation via /v1/videos (#43706) 2026-10-14 10:10:07 -07:00
_gemini_long_decode_support.py feat(gemini)!: auto-continue gemini-4-argon long decodes with continuationToken (#46240) 2026-10-15 22:54:23 +00:00
_gemini_tts_voice_support.py perf(ci): isolate and shard integration suites (#44146) 2026-10-10 11:15:19 -07:00
_idle_upstream_support.py fix(http): restore the 15s aiohttp keep-alive default in the Python and Rust clients (#46355) 2026-10-14 10:50:11 -07:00
_mantle_gpt_prompt_cache_support.py fix(bedrock_mantle): send OpenAI explicit prompt cache breakpoints for GPT-5.6 and newer (#38729) 2026-10-08 02:07:22 -07:00
_responses_bridge_prompt_cache_breakpoint.py fix(responses): keep prompt_cache_breakpoint markers in the chat to responses bridge (#44119) 2026-10-07 16:25:36 -07:00
_sagemaker_chat_container.py fix(sagemaker): keep AWS credentials out of the sagemaker_chat request body (#45724) 2026-10-10 15:30:54 -07:00
_tool_search_support.py fix(anthropic): keep tool_search_tool_result so the search call replays with its result (#46440) 2026-10-14 18:52:32 -07:00
_vertex_batch_rows.py fix(vertex_ai): return 400 with file line number for malformed batch JSONL row (#40097) 2026-10-15 03:21:48 -07:00
fine_tuning_proxy_config.yaml test: migrate SDK and proxy-booting legacy CircleCI tests to their tiers (#45758) 2026-10-11 00:17:21 -07:00
openai_fine_tuning.jsonl test: migrate SDK and proxy-booting legacy CircleCI tests to their tiers (#45758) 2026-10-11 00:17:21 -07:00
test_a2a_sdk_upstream_wire.py test: migrate legacy unit dirs and unrun folders into tests/unit, tests/integration and tests/e2e (#45940) 2026-10-11 06:24:57 -07:00
test_anthropic_count_tokens_system_lift_wire.py fix(proxy): raise on provider count_tokens errors instead of falling back to tiktoken (#46121) 2026-10-13 19:08:01 +00:00
test_anthropic_passthrough_wire.py test(integration): cover pass_through endpoint contracts (#44690) 2026-10-14 07:38:17 +00:00
test_anthropic_refusal_stop_details_chaos.py fix(anthropic): keep refusal stop_reason and stop_details on the logged response (#46104) 2026-10-14 14:42:49 -07:00
test_anthropic_refusal_stop_details_wire.py fix(anthropic): keep refusal stop_reason and stop_details on the logged response (#46104) 2026-10-14 14:42:49 -07:00
test_anthropic_skills_create_wire.py test(integration): cover anthropic_messages endpoint contracts (#44704) 2026-10-14 06:57:54 +00:00
test_anthropic_system_cache_control_wire.py test(integration): assert /v1/responses usage reports Anthropic system cache write then read (#42855) 2026-09-23 18:52:06 -07:00
test_anthropic_thinking_signature_logging_wire.py fix(anthropic): stop repeating streamed thinking text in the signature chunk (#44127) 2026-10-02 19:38:41 -07:00
test_anthropic_thinking_signature_stream_wire.py test: expect the list-position index on streamed thinking blocks in the integration suite (#46618) 2026-10-15 05:52:56 +00:00
test_anthropic_tool_search_result_chaos.py fix(anthropic): keep tool_search_tool_result so the search call replays with its result (#46440) 2026-10-14 18:52:32 -07:00
test_anthropic_tool_search_result_matrix_wire.py fix(anthropic): count a replayed server tool call's cache mark the way modify_params sends it (#46591) 2026-10-15 14:46:43 -07:00
test_anthropic_tool_search_result_modify_params_wire.py fix(anthropic): count a replayed server tool call's cache mark the way modify_params sends it (#46591) 2026-10-15 14:46:43 -07:00
test_anthropic_tool_search_result_passthrough_wire.py fix(anthropic): keep tool_search_tool_result so the search call replays with its result (#46440) 2026-10-14 18:52:32 -07:00
test_anthropic_tool_search_result_wire.py fix(anthropic): keep tool_search_tool_result so the search call replays with its result (#46440) 2026-10-14 18:52:32 -07:00
test_api_base_trailing_slash_chaos.py fix(llms): sanitize trailing slashes in provider get_complete_url (#42887) 2026-10-15 02:59:46 -07:00
test_api_base_trailing_slash_owned_proxy.py fix(llms): sanitize trailing slashes in provider get_complete_url (#42887) 2026-10-15 02:59:46 -07:00
test_api_base_trailing_slash_wire.py fix(llms): sanitize trailing slashes in provider get_complete_url (#42887) 2026-10-15 02:59:46 -07:00
test_assemblyai_passthrough.py test: migrate SDK and proxy-booting legacy CircleCI tests to their tiers (#45758) 2026-10-11 00:17:21 -07:00
test_assistants_evals_wire.py test(integration): pin assistants and evals endpoint contracts (#44734) 2026-10-14 07:08:35 +00:00
test_assistants_route_errors_chaos.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_assistants_route_errors_wire.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_audit_any_sweep_harness_endpoint.py refactor(types): replace Any with proven types in 9 files (#44389) 2026-10-03 21:03:16 +00:00
test_audit_any_sweep_vector_store.py refactor(types): replace Any with proven types in 9 files (#44389) 2026-10-03 21:03:16 +00:00
test_azure_ai_chat_wire.py test(integration): regression tests for August provider translation and streaming bugs (#42621) 2026-09-22 20:52:00 -07:00
test_azure_ai_claude_messages_wire.py test(integration): cover anthropic_messages endpoint contracts (#44704) 2026-10-14 06:57:54 +00:00
test_azure_ai_flux2_image_wire.py test(integration): regression tests for August provider translation and streaming bugs (#42621) 2026-09-22 20:52:00 -07:00
test_azure_ai_rerank_auth_wire.py test(integration): regression tests for August provider translation and streaming bugs (#42621) 2026-09-22 20:52:00 -07:00
test_azure_embedding_wire.py test(integration): cover embeddings endpoint contracts (#44660) 2026-10-14 03:38:12 +00:00
test_bedrock_anthropic_beta_header_wire.py refactor(types): replace Any with proven types in 157 files (#44798) 2026-10-06 15:31:54 +00:00
test_bedrock_auth_wire.py test(integration): regression tests for August provider translation and streaming bugs (#42621) 2026-09-22 20:52:00 -07:00
test_bedrock_batch_blank_s3_env_wire.py test(e2e): assert only litellm-owned batch behavior and move the blank S3 env pin to an integration test (#43321) 2026-09-26 11:17:03 -07:00
test_bedrock_batch_files_wire.py test(integration): regression tests for August provider translation and streaming bugs (#42621) 2026-09-22 20:52:00 -07:00
test_bedrock_batch_retrieve_status_wire.py fix(bedrock): map Scheduled batch jobs to in_progress on retrieve (#45157) 2026-10-07 18:08:51 -07:00
test_bedrock_claude_thinking_wire.py fix(anthropic): drop unknown legacy thinking keys under drop_params when rebuilding adaptive thinking (#46741) 2026-10-15 23:09:30 +00:00
test_bedrock_converse_client_metadata_wire.py test(integration): regression tests for July provider translation, routing and streaming bugs (#42693) 2026-09-23 09:51:06 -07:00
test_bedrock_converse_config_blocks_wire.py test(integration): regression tests for August provider translation and streaming bugs (#42621) 2026-09-22 20:52:00 -07:00
test_bedrock_converse_lookaround_regex_chaos.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_bedrock_converse_lookaround_regex_wire.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_bedrock_converse_missing_content_wire.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_bedrock_converse_redacted_thinking_wire.py test: expect the list-position index on streamed thinking blocks in the integration suite (#46618) 2026-10-15 05:52:56 +00:00
test_bedrock_converse_stream_event_frames_wire.py test(integration): update stale CircleCI integration expectations (#46570) 2026-10-14 18:21:59 -07:00
test_bedrock_deepseek_reasoning_wire.py test(integration): regression tests for July provider translation, routing and streaming bugs (#42693) 2026-09-23 09:51:06 -07:00
test_bedrock_embedding_wire.py test(integration): regression tests for August provider translation and streaming bugs (#42621) 2026-09-22 20:52:00 -07:00
test_bedrock_gpt5_reasoning_wire.py test(integration): regression tests for August provider translation and streaming bugs (#42621) 2026-09-22 20:52:00 -07:00
test_bedrock_gpt_responses_native_wire.py feat(bedrock): serve gpt-5.6+ chat completions natively by default, with chat_completions/ opt-in for gpt-oss and grok (#44307) 2026-10-02 19:55:54 -07:00
test_bedrock_invoke_cache_usage_wire.py test(integration): regression tests for July provider translation, routing and streaming bugs (#42693) 2026-09-23 09:51:06 -07:00
test_bedrock_kimi_k3_cache_point_wire.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_bedrock_knowledge_base_user_context_wire.py test(integration): regression tests for July provider translation, routing and streaming bugs (#42693) 2026-09-23 09:51:06 -07:00
test_bedrock_mantle_claude_chat_chaos.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_bedrock_mantle_claude_chat_wire.py test(integration): assert the Messages API health probe on Mantle Claude (#44612) 2026-10-05 15:28:02 -07:00
test_bedrock_mantle_codex_input_wire.py test(integration): regression tests for July provider translation, routing and streaming bugs (#42693) 2026-09-23 09:51:06 -07:00
test_bedrock_mantle_context_overflow_fallbacks_wire.py test(integration): update stale CircleCI integration expectations (#46570) 2026-10-14 18:21:59 -07:00
test_bedrock_mantle_context_overflow_wire.py fix(bedrock_mantle): map the OpenAI-shaped context overflow envelope too (#41747) 2026-10-12 21:50:22 +00:00
test_bedrock_mantle_count_tokens_wire.py fix(count_tokens): count system and tools on Vertex Gemini, fall back to Mantle on Bedrock 404 (#46430) 2026-10-15 10:36:10 -04:00
test_bedrock_mantle_gpt_prompt_cache_breakpoint_wire.py fix(bedrock_mantle): send OpenAI explicit prompt cache breakpoints for GPT-5.6 and newer (#38729) 2026-10-08 02:07:22 -07:00
test_bedrock_mantle_output_limit_wire.py test(integration): update stale CircleCI integration expectations (#46570) 2026-10-14 18:21:59 -07:00
test_bedrock_mantle_responses_wire.py test(integration): regression tests for July provider translation, routing and streaming bugs (#42693) 2026-09-23 09:51:06 -07:00
test_bedrock_mantle_wire.py test(integration): regression tests for July provider translation, routing and streaming bugs (#42693) 2026-09-23 09:51:06 -07:00
test_bedrock_marengo_embed_3_wire.py test(integration): regression tests for July provider translation, routing and streaming bugs (#42693) 2026-09-23 09:51:06 -07:00
test_bedrock_model_discovery_chaos.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_bedrock_model_discovery_wire.py fix(bedrock): list the account's invocable models behind bedrock/* when check_provider_endpoint is on (#43981) 2026-10-09 14:14:17 -07:00
test_bedrock_passthrough_stream_wire.py test(integration): cover pass_through endpoint contracts (#44690) 2026-10-14 07:38:17 +00:00
test_bedrock_rerank_wire.py test(integration): regression tests for August provider translation and streaming bugs (#42621) 2026-09-22 20:52:00 -07:00
test_bedrock_role_configuration.py test(integration): regression tests for July provider translation, routing and streaming bugs (#42693) 2026-09-23 09:51:06 -07:00
test_bedrock_runtime_chat_completions_chaos.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_bedrock_runtime_chat_completions_sad_wire.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_bedrock_runtime_chat_completions_wire.py fix(bedrock): drop stop sequences for GPT 5.6 and newer instead of forwarding them to a 400 (#45664) 2026-10-09 15:00:24 -07:00
test_bedrock_stream_timeout_chaos.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_bedrock_stream_timeout_wire.py fix(responses): stop SDK retries nesting under router retries, and unbreak CircleCI integration tests (#44791) 2026-10-06 15:57:33 +00:00
test_bedrock_thinking_tokens_wire.py test(integration): regression tests for July provider translation, routing and streaming bugs (#42693) 2026-09-23 09:51:06 -07:00
test_cache_control_tool_call_marks_owned_proxy.py fix(responses): propagate cache_control through responses->chat completion bridge (#32511) 2026-10-14 15:44:34 -07:00
test_cache_control_tool_call_marks_wire.py test(integration): follow #46101 in the breakpoint cache mark census expectation (#46509) 2026-10-14 20:55:43 -07:00
test_chat_completions_tool_search_chaos.py fix(anthropic)!: translate the OpenAI tool_search tool on /v1/chat/completions for Claude models (#46389) 2026-10-15 04:47:05 +00:00
test_chat_completions_tool_search_history_wire.py fix(anthropic)!: translate the OpenAI tool_search tool on /v1/chat/completions for Claude models (#46389) 2026-10-15 04:47:05 +00:00
test_chat_completions_tool_search_wire.py fix(anthropic)!: translate the OpenAI tool_search tool on /v1/chat/completions for Claude models (#46389) 2026-10-15 04:47:05 +00:00
test_chatgpt_responses_caller_stream_chaos.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_chatgpt_responses_caller_stream_wire.py fix(responses): honor caller stream flag when provider forces SSE (internal copy of #34095) (#41235) 2026-10-06 03:30:47 +00:00
test_chatgpt_responses_top_level_error_wire.py fix(responses): keep the provider code and message on top-level stream error events (#46438) 2026-10-14 16:16:46 -07:00
test_claude_chat_prompt_cache_breakpoint_chaos.py test(integration): audit cells for prompt_cache_breakpoint on Claude chat, messages, fallbacks, affinity and chaos 2026-10-15 00:33:23 +00:00
test_claude_chat_prompt_cache_breakpoint_placement_wire.py fix(anthropic): settle breakpoint ttls in linear time and check every breakpoint a client mark wins over 2026-10-16 00:25:57 +00:00
test_claude_chat_prompt_cache_breakpoint_routes.py test(integration): audit cells for prompt_cache_breakpoint on Claude chat, messages, fallbacks, affinity and chaos 2026-10-15 00:33:23 +00:00
test_claude_chat_prompt_cache_breakpoint_wire.py test(integration): wire cells for breakpoint placement, Mantle and Claude Platform, and the router's 1h pin 2026-10-15 19:02:03 +00:00
test_cloudflare_clef_images_chaos.py feat(decisions)!: send images to Cloudflare Clef through its images field (#45734) 2026-10-14 12:14:48 +00:00
test_cloudflare_clef_images_wire.py feat(decisions)!: send images to Cloudflare Clef through its images field (#45734) 2026-10-14 12:14:48 +00:00
test_cloudflare_clef_omni_media_chaos.py feat(decisions): support Clef-omni audio and video (#45811) 2026-10-12 20:29:49 +00:00
test_cloudflare_clef_omni_media_wire.py feat(decisions): support Clef-omni audio and video (#45811) 2026-10-12 20:29:49 +00:00
test_container_file_routes_wire.py refactor(types): replace Any with proven types in 157 files (#44798) 2026-10-06 15:31:54 +00:00
test_coralbricks_provider_chaos.py feat(providers): add CoralBricks as a native provider (#44700) 2026-10-12 21:56:59 +00:00
test_coralbricks_provider_env_wire.py feat(providers): add CoralBricks as a native provider (#44700) 2026-10-12 21:56:59 +00:00
test_coralbricks_provider_wire.py feat(providers): add CoralBricks as a native provider (#44700) 2026-10-12 21:56:59 +00:00
test_cursor_responses_wire.py fix(cache): keep cache hits free of unset fields, and follow intentional behavior changes in CircleCI suites (#46359) 2026-10-14 08:56:05 -07:00
test_dashscope_chat_wire.py fix(proxy): keep provider refusal, logprobs and system_fingerprint in chat completions (#44788) 2026-10-13 13:17:40 -07:00
test_dashscope_video_chaos.py feat(dashscope): add Wan 3.0, Wan 2.7 and HappyHorse video generation via /v1/videos (#43706) 2026-10-14 10:10:07 -07:00
test_dashscope_video_wire.py feat(dashscope): add Wan 3.0, Wan 2.7 and HappyHorse video generation via /v1/videos (#43706) 2026-10-14 10:10:07 -07:00
test_databricks_agent_wire.py feat(agents): support Databricks Agent Platform agents in the agent gateway (#45681) 2026-10-15 12:42:06 -07:00
test_databricks_chat_wire.py test: cover response contracts across translation endpoints (#46632) 2026-10-15 00:05:09 -07:00
test_databricks_decisions_wire.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_databricks_oauth_wire.py test(integration): regression tests for August provider translation and streaming bugs (#42621) 2026-09-22 20:52:00 -07:00
test_decision_model_guardrail.py fix(guardrails): resolve decision models in caller team scope (#46085) 2026-10-15 11:43:36 -07:00
test_decisions_box_deployment_flag_chaos.py fix(decisions): read a deployment's own supports_box_questions flag before the cost map (#46696) 2026-10-15 12:34:36 -07:00
test_decisions_box_deployment_flag_wire.py fix(decisions): read a deployment's own supports_box_questions flag before the cost map (#46696) 2026-10-15 12:34:36 -07:00
test_decisions_chaos.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_decisions_openai_format_wire.py fix(decisions): keep box answers from OpenAI-format providers instead of turning them into refusals (#46633) 2026-10-15 14:01:59 +00:00
test_decisions_wire.py feat(decisions): add Laya and Bespoke providers to the Add Model form (#45260) 2026-10-14 08:11:11 -07:00
test_deepseek_vision_wire.py test(integration): regression tests for August provider translation and streaming bugs (#42621) 2026-09-22 20:52:00 -07:00
test_device_code_login_guard_boot.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_device_code_login_guard_wire.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_embedding_cache_partial_hit_wire.py test(integration): cover embeddings endpoint contracts (#44660) 2026-10-14 03:38:12 +00:00
test_embedding_token_array_wire.py test(integration): cover embeddings endpoint contracts (#44660) 2026-10-14 03:38:12 +00:00
test_fake_openai_upstream_routes.py test: migrate local_testing SDK and provider tests out of build_and_test (#45760) 2026-10-10 23:14:25 +00:00
test_fal_ai_chat_wire.py fix(fal_ai): honour global api_base for image generation and reject non-string reasoning_effort with 400 (#42512) 2026-09-22 13:13:54 -07:00
test_fal_ai_image_wire.py test(integration): cover images endpoint contracts (#44653) 2026-10-14 03:40:29 +00:00
test_fal_ai_passthrough_wire.py fix(fal_ai): align /fal_ai queue gate with the pricer and normalise resolution type (#42505) 2026-09-22 13:12:07 -07:00
test_fal_ai_video_wire.py fix(fal_ai): reuse the status client and headers on the video result probe (#42511) 2026-09-22 13:12:51 -07:00
test_files_batches_wire.py test(integration): cover files_batches endpoint contracts (#44703) 2026-10-14 06:57:41 +00:00
test_fireworks_ai_router_slug_wire.py feat(fireworks_ai): route and list the auto, auto-instant and firerouter routers (#43641) 2026-09-28 19:08:16 -07:00
test_fireworks_ai_session_affinity_wire.py test(integration): regression tests for July provider translation, routing and streaming bugs (#42693) 2026-09-23 09:51:06 -07:00
test_gemini_agents_wire.py test(integration): cover google_genai endpoint contracts (#44680) 2026-10-14 04:01:35 +00:00
test_gemini_chat_video_metadata_wire.py feat(gemini): accept file content blocks with video_metadata on multimodal embeddings (#45305) 2026-10-08 13:20:03 -07:00
test_gemini_embedding_file_block_chaos.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_gemini_embedding_file_block_wire.py feat(gemini): accept file content blocks with video_metadata on multimodal embeddings (#45305) 2026-10-08 13:20:03 -07:00
test_gemini_image_config_and_usage_wire.py refactor(types): replace Any with proven types in 157 files (#44798) 2026-10-06 15:31:54 +00:00
test_gemini_interactions_wire.py test(integration): cover google_genai endpoint contracts (#44680) 2026-10-14 04:01:35 +00:00
test_gemini_long_decode_chaos.py feat(gemini)!: auto-continue gemini-4-argon long decodes with continuationToken (#46240) 2026-10-15 22:54:23 +00:00
test_gemini_long_decode_wire.py feat(gemini)!: auto-continue gemini-4-argon long decodes with continuationToken (#46240) 2026-10-15 22:54:23 +00:00
test_gemini_omitted_temperature.py test(translation): share rich fixtures across Claude, GPT and Gemini providers (#46585) 2026-10-14 19:53:09 -07:00
test_gemini_thinking_replay_chaos.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_gemini_thinking_replay_wire.py test: make 77 legacy live tests offline in litellm_utils, router_unit and responses dirs (#45298) 2026-10-08 02:37:55 -07:00
test_gemini_tool_result_part_reference_chaos.py fix(vertex_ai): prevent Gemini from misinterpreting $ref in tool results (#38284) 2026-10-15 06:26:47 -07:00
test_gemini_tool_result_part_reference_wire.py fix(vertex_ai): prevent Gemini from misinterpreting $ref in tool results (#38284) 2026-10-15 06:26:47 -07:00
test_gemini_tts_voice_mapping_owned_proxy.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_gemini_tts_voice_mapping_wire.py fix(gemini): map OpenAI voice names to Gemini prebuilt TTS voices (#45303) 2026-10-09 18:55:22 -07:00
test_google_genai_adapter_wire.py fix(vertex_ai): route Model Garden models through the generateContent adapter (#41836) 2026-10-15 08:06:49 -07:00
test_google_genai_native_wire.py test(integration): cover google_genai endpoint contracts (#44680) 2026-10-14 04:01:35 +00:00
test_google_genai_proxy_sdk_wire.py test: migrate SDK and proxy-booting legacy CircleCI tests to their tiers (#45758) 2026-10-11 00:17:21 -07:00
test_google_genai_vertex_routes_wire.py fix(vertex_ai): route Model Garden models through the generateContent adapter (#41836) 2026-10-15 08:06:49 -07:00
test_hosted_vllm_reasoning_content_chaos.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_hosted_vllm_reasoning_content_wire.py test: cover response contracts across translation endpoints (#46632) 2026-10-15 00:05:09 -07:00
test_idle_upstream_keepalive_edges_wire.py test(integration): update stale CircleCI integration expectations (#46570) 2026-10-14 18:21:59 -07:00
test_idle_upstream_keepalive_override_wire.py fix(http): restore the 15s aiohttp keep-alive default in the Python and Rust clients (#46355) 2026-10-14 10:50:11 -07:00
test_idle_upstream_reconnect_chaos_wire.py fix(http): restore the 15s aiohttp keep-alive default in the Python and Rust clients (#46355) 2026-10-14 10:50:11 -07:00
test_idle_upstream_reconnect_routes_wire.py fix(http): restore the 15s aiohttp keep-alive default in the Python and Rust clients (#46355) 2026-10-14 10:50:11 -07:00
test_idle_upstream_reconnect_rust_messages_wire.py fix(http): restore the 15s aiohttp keep-alive default in the Python and Rust clients (#46355) 2026-10-14 10:50:11 -07:00
test_idle_upstream_reconnect_wire.py fix(http): restore the 15s aiohttp keep-alive default in the Python and Rust clients (#46355) 2026-10-14 10:50:11 -07:00
test_image_gen_drop_params_wire.py test(integration): cover images endpoint contracts (#44653) 2026-10-14 03:40:29 +00:00
test_inferenceapis_context_window_fallbacks_wire.py feat(providers): add Inference APIs as OpenAI-compatible provider (#41669) 2026-10-15 08:17:36 -07:00
test_inferenceapis_wire.py feat(providers): add Inference APIs as OpenAI-compatible provider (#41669) 2026-10-15 08:17:36 -07:00
test_interactions_openai_bridge_wire.py refactor(types): replace Any with proven types in 157 files (#44798) 2026-10-06 15:31:54 +00:00
test_internal_params_wire.py fix(params): validate stream_chunk_size once, before any provider call (#43222) 2026-09-26 23:01:20 +00:00
test_lm_studio_chat_wire.py test: migrate local_testing SDK and provider tests out of build_and_test (#45760) 2026-10-10 23:14:25 +00:00
test_moderations.py test: migrate SDK and proxy-booting legacy CircleCI tests to their tiers (#45758) 2026-10-11 00:17:21 -07:00
test_moderations_wire.py test(integration): cover small_inference endpoint contracts (#44654) 2026-10-14 04:06:15 +00:00
test_nvidia_nim_ranking_wire.py test(integration): regression tests for July provider translation, routing and streaming bugs (#42693) 2026-09-23 09:51:06 -07:00
test_ocr_multipart_wire.py test(integration): cover small_inference endpoint contracts (#44654) 2026-10-14 04:06:15 +00:00
test_ocr_router_wire.py test: move 81 legacy live tests in pass-through, spend, batches, audio, search, guardrails, image and ocr dirs offline (#45288) 2026-10-08 02:38:12 -07:00
test_ollama_prompt_tools_chaos.py fix(prompt_templates): do not mutate messages in-place in function_call_prompt (#41676) 2026-10-15 14:37:11 -07:00
test_ollama_prompt_tools_wire.py fix(prompt_templates): do not mutate messages in-place in function_call_prompt (#41676) 2026-10-15 14:37:11 -07:00
test_openai_chat_prompt_cache_breakpoint_wire.py fix(openai): carry cache_control to prompt_cache_breakpoint on chat completions to gpt-5.6 (#46101) 2026-10-13 15:45:49 -07:00
test_openai_chat_wire.py test(integration): cover chat_completions endpoint contracts (#44688) 2026-10-14 03:45:38 +00:00
test_openai_completion_wire.py test(integration): cover chat_completions endpoint contracts (#44688) 2026-10-14 03:45:38 +00:00
test_openai_count_tokens_tools_wire.py fix(count_tokens): count system and tools on Vertex Gemini, fall back to Mantle on Bedrock 404 (#46430) 2026-10-15 10:36:10 -04:00
test_openai_dialect_prompt_cache_breakpoint_owned_proxy.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_openai_embedding_timeout_wire.py test: migrate local_testing SDK and provider tests out of build_and_test (#45760) 2026-10-10 23:14:25 +00:00
test_openai_embedding_wire.py test(integration): cover embeddings endpoint contracts (#44660) 2026-10-14 03:38:12 +00:00
test_openai_error_handling.py test: migrate SDK and proxy-booting legacy CircleCI tests to their tiers (#45758) 2026-10-11 00:17:21 -07:00
test_openai_fine_tuning.py test: migrate SDK and proxy-booting legacy CircleCI tests to their tiers (#45758) 2026-10-11 00:17:21 -07:00
test_openai_image_edit_wire.py test(integration): cover images endpoint contracts (#44653) 2026-10-14 03:40:29 +00:00
test_openai_image_variation_wire.py test(integration): cover images endpoint contracts (#44653) 2026-10-14 03:40:29 +00:00
test_openai_passthrough_files_wire.py test: move 81 legacy live tests in pass-through, spend, batches, audio, search, guardrails, image and ocr dirs offline (#45288) 2026-10-08 02:38:12 -07:00
test_openai_passthrough_multipart_wire.py test(integration): cover pass_through endpoint contracts (#44690) 2026-10-14 07:38:17 +00:00
test_openai_realtime_transcription_url.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_openai_responses_prompt_cache_breakpoint_wire.py fix(openai): carry cache_control to prompt_cache_breakpoint on /v1/responses to gpt-5.6 (#46105) 2026-10-13 15:23:07 -07:00
test_openai_responses_tool_search_wire.py fix(responses): map Anthropic tool search to OpenAI server tool search (#46441) 2026-10-15 05:13:56 +00:00
test_openai_responses_websocket_wire.py fix(responses): keep the provider code and message on top-level stream error events (#46438) 2026-10-14 16:16:46 -07:00
test_openai_stream_text_usage_wire.py refactor(types): replace Any with proven types in 7 files (#43704) 2026-09-29 06:12:58 -07:00
test_provider_lookup_status.py test(integration): pin assistants and evals endpoint contracts (#44734) 2026-10-14 07:08:35 +00:00
test_rag_ingest_wire.py test(integration): cover vector_stores endpoint contracts (#44739) 2026-10-14 03:59:08 +00:00
test_realtime_transcription_guardrail_session_update.py test(integration): isolate shared proxy state behind six order-dependent flakes (#46172) 2026-10-13 09:55:11 -07:00
test_realtime_unguarded_voice_turns.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_realtime_webrtc_http.py test(integration): cover realtime endpoint contracts (#44689) 2026-10-14 03:49:34 +00:00
test_request_boundary.py fix(tests): resolve the integration support package without run.py's PYTHONPATH 2026-09-15 21:10:08 -07:00
test_rerank_latency_headers_wire.py test(integration): cover small_inference endpoint contracts (#44654) 2026-10-14 04:06:15 +00:00
test_responses_bridge_incomplete.py fix(responses): return the first message item as content for structured output on the chat bridge (#46508) 2026-10-14 16:58:00 -07:00
test_responses_bridge_input_audio_wire.py fix(responses): map input_audio blocks in the chat-to-Responses bridge (#45224) 2026-10-07 22:25:35 -07:00
test_responses_bridge_item_marks_guardrail_wire.py fix(responses): propagate cache_control through responses->chat completion bridge (#32511) 2026-10-14 15:44:34 -07:00
test_responses_bridge_message_cache_control_wire.py fix(responses): propagate cache_control through responses->chat completion bridge (#32511) 2026-10-14 15:44:34 -07:00
test_responses_bridge_message_cache_control_wire_chaos.py fix(responses): propagate cache_control through responses->chat completion bridge (#32511) 2026-10-14 15:44:34 -07:00
test_responses_bridge_namespace_tools.py test(integration): regression tests for July provider translation, routing and streaming bugs (#42693) 2026-09-23 09:51:06 -07:00
test_responses_bridge_prompt_cache_breakpoint.py fix(responses): keep prompt_cache_breakpoint markers in the chat to responses bridge (#44119) 2026-10-07 16:25:36 -07:00
test_responses_bridge_prompt_cache_breakpoint_chaos.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_responses_bridge_prompt_cache_breakpoint_wire.py fix(anthropic): send detail on input_image parts bridged from /v1/messages to Responses (#46720) 2026-10-15 22:33:09 +00:00
test_responses_bridge_prompt_cache_breakpoint_wire_chaos.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_responses_bridge_stream_output_items_wire.py fix(ollama): one stream id, 400 on non-text tool content, separated tool results, no empty message item (#45177) 2026-10-07 19:08:26 -07:00
test_responses_bridge_tool_search_chaos.py fix(responses bridge): round-trip tool_search_call and tool_search_output items between turns (#46444) 2026-10-14 21:40:14 -07:00
test_responses_bridge_tool_search_replay.py fix(responses bridge): round-trip tool_search_call and tool_search_output items between turns (#46444) 2026-10-14 21:40:14 -07:00
test_responses_bridge_tool_search_replay_wire.py fix(responses): translate OpenAI tool_search to Anthropic tool search for Claude models (#45882) 2026-10-14 20:42:22 -07:00
test_responses_bridge_tool_search_undeferred_wire.py fix(responses): translate OpenAI tool_search to Anthropic tool search for Claude models (#45882) 2026-10-14 20:42:22 -07:00
test_responses_bridge_tool_search_wire.py fix(responses bridge): round-trip tool_search_call and tool_search_output items between turns (#46444) 2026-10-14 21:40:14 -07:00
test_responses_client_header_forwarding_wire.py test(integration): regression tests for July provider translation, routing and streaming bugs (#42693) 2026-09-23 09:51:06 -07:00
test_responses_compact_wire.py fix(proxy): check previous_response_id ownership on /v1/responses/compact (#46408) 2026-10-14 17:57:34 +00:00
test_responses_core_wire.py test(integration): cover responses endpoint contracts (#44733) 2026-10-14 06:59:15 +00:00
test_responses_error_status_wire.py test: move 81 legacy live tests in pass-through, spend, batches, audio, search, guardrails, image and ocr dirs offline (#45288) 2026-10-08 02:38:12 -07:00
test_responses_lifecycle_wire.py test(integration): cover responses endpoint contracts (#44733) 2026-10-14 06:59:15 +00:00
test_responses_minted_reasoning_replay_chaos.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_responses_minted_reasoning_replay_wire.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_responses_reasoning_item_wire.py refactor(types): replace Any with proven types in 157 files (#44798) 2026-10-06 15:31:54 +00:00
test_responses_websocket_session_limit.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_s3_wire.py test: fix stale and flaky tests across CircleCI, GHA and Buildkite (#45530) 2026-10-09 11:40:14 -07:00
test_sagemaker_chat_credential_sources.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_sagemaker_chat_wire.py fix(sagemaker): keep AWS credentials out of the sagemaker_chat request body (#45724) 2026-10-10 15:30:54 -07:00
test_sagemaker_embedding_chaos.py feat(sagemaker)!: inference components and OpenAI-compatible embeddings (#44985) 2026-10-12 22:25:52 +00:00
test_sagemaker_embedding_wire.py feat(sagemaker)!: inference components and OpenAI-compatible embeddings (#44985) 2026-10-12 22:25:52 +00:00
test_search_wire.py fix(search): bill an unpriced fallback search tool at its own price, not the failed primary's (#46734) 2026-10-15 12:20:33 -07:00
test_strands_decider_agentcore_wire.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_tencent_chat_wire.py fix(proxy): keep provider refusal, logprobs and system_fingerprint in chat completions (#44788) 2026-10-13 13:17:40 -07:00
test_validated_reply_boundaries_under_faults.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_vector_store_create_wire.py test(integration): cover vector_stores endpoint contracts (#44739) 2026-10-14 03:59:08 +00:00
test_vector_store_search_wire.py test(integration): cover vector_stores endpoint contracts (#44739) 2026-10-14 03:59:08 +00:00
test_vertex_anthropic_inline_tools_beta_wire.py test(integration): remove six setup causes of CircleCI integration reds (#46490) 2026-10-14 12:00:35 -07:00
test_vertex_anthropic_structured_output_wire.py test(integration): update stale CircleCI integration expectations (#46570) 2026-10-14 18:21:59 -07:00
test_vertex_batch_output_info_wire.py fix(vertex_ai): keep batch output_file_id null until Vertex reports outputInfo (#43030) 2026-09-24 14:05:31 -07:00
test_vertex_batch_row_line_errors_chaos.py fix(vertex_ai): return 400 with file line number for malformed batch JSONL row (#40097) 2026-10-15 03:21:48 -07:00
test_vertex_batch_row_line_errors_owned_proxy.py fix(vertex_ai): return 400 with file line number for malformed batch JSONL row (#40097) 2026-10-15 03:21:48 -07:00
test_vertex_batch_row_line_errors_wire.py fix(vertex_ai): return 400 with file line number for malformed batch JSONL row (#40097) 2026-10-15 03:21:48 -07:00
test_vertex_credentials_source_wire.py fix(vertex): report an unreadable credentials file as a read failure (#40173) 2026-10-15 02:03:56 -07:00
test_vertex_embedding_batch_file_block_wire.py feat(gemini): accept file content blocks with video_metadata on multimodal embeddings (#45305) 2026-10-08 13:20:03 -07:00
test_vertex_embedding_file_block_wire.py feat(gemini): accept file content blocks with video_metadata on multimodal embeddings (#45305) 2026-10-08 13:20:03 -07:00
test_vertex_embedding_wire.py test(integration): cover embeddings endpoint contracts (#44660) 2026-10-14 03:38:12 +00:00
test_vertex_gemini_count_tokens_wire.py fix(count_tokens): count system and tools on Vertex Gemini, fall back to Mantle on Bedrock 404 (#46430) 2026-10-15 10:36:10 -04:00
test_vertex_gemini_fragmented_stream_wire.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_vertex_gemini_function_call_wire.py refactor(types): replace Any with proven types in 7 files (#43704) 2026-09-29 06:12:58 -07:00
test_vertex_partner_count_tokens_wire.py fix(proxy): raise on provider count_tokens errors instead of falling back to tiktoken (#46121) 2026-10-13 19:08:01 +00:00
test_vertex_partner_messages_wire.py test(integration): cover anthropic_messages endpoint contracts (#44704) 2026-10-14 06:57:54 +00:00
test_vertex_passthrough_wire.py fix(anthropic): keep tool_search_tool_result so the search call replays with its result (#46440) 2026-10-14 18:52:32 -07:00
test_vertex_realtime_multi_region_host.py fix(vertex_ai): dial the multi-region Live API host for realtime sessions (#45166) 2026-10-07 17:39:38 -07:00
test_vertex_timestamps_utc_wire.py fix(vertex_ai): parse Vertex timestamps as UTC so created_at is not shifted by the host timezone (#45550) 2026-10-10 15:18:15 -07:00
test_voyage_mongodb_host_wire.py test(integration): fix three more CircleCI integration reds from test setup (#46510) 2026-10-14 15:23:14 -07:00
test_xai_batch_output_file_wire.py fix(xai): name a completed xAI batch's results file and stop looking it up on xAI's files API (#45559) 2026-10-14 22:05:43 -07:00
test_xai_server_tool_usage_wire.py refactor(types): replace Any with proven types in 157 files (#44798) 2026-10-06 15:31:54 +00:00
test_xai_web_search_wire.py test(integration): regression tests for August provider translation and streaming bugs (#42621) 2026-09-22 20:52:00 -07:00
test_xiaomi_mimo_wire.py fix(proxy): keep provider refusal, logprobs and system_fingerprint in chat completions (#44788) 2026-10-13 13:17:40 -07:00