scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-16 04:29:31 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 138
litellm/tests/integration/providers
History
yucheng-berri d0fa05076d
fix(xai): name a completed xAI batch's results file and stop looking it up on xAI's files API (#45559) 
* fix(xai): report a completed xAI batch's results as its output file details

* fix(xai): keep a short results download and per-call timeouts out of the reported file details

* fix(xai): describe a batch's results file from the batch alone and let the batch job fill its size

Reading an xAI batch id as a file now costs one batch read and never downloads
the results: the file reports a 0 byte placeholder, the results filename and
the registration time. The managed proxy fills in the size the batch cost job
measures when it downloads the results, keeps a known size when a provider
reports none, and drops the per-process results download cache.

* fix(managed_files): keep a measured batch output size when a refresh races the batch job

A refresh that read the row before the batch cost job stored the measured size
could save the provider's unsized details over it. The refresh now reads the
row back from the writer and saves only while the row still holds what it read,
retrying when another write landed in between. The new tests drive the real
xAI and OpenAI retrieve paths through a fake HTTP upstream.

* fix(managed_files): guard the refresh save with the row the request already read

The conditional save first compares against the row the request read, so a
refresh adds no database read unless another write landed in between; only
then does it read the row back from the writer and try again.

* fix(managed_files): save a refreshed file object with one guarded write and no extra database reads

A refresh that loses its guarded write to a newer one now leaves that row
alone and drops this worker's cached copy, so the next read loads the newer
row instead of retrying against the database on the request path

* fix(managed_files): retry the batch cost job's measured size after a read refreshes the row first

The batch cost job re-reads the row from the writer and saves again when its
guarded write loses to a concurrent refresh, so its measured size is never
dropped. Reads without a measured size still make one write and no re-read

* fix(managed_files): size a concurrently refreshed row instead of restoring the batch job's placeholder copy

When the batch cost job loses its guarded write and the row it re-reads holds
refreshed details, it applies its measured size to those details, so the
results filename and any size the provider reported are kept

* fix(managed_files): keep a batch output's earlier created_at when a refresh replaces its placeholder

A refresh saved over a stored row now keeps the earlier of the two creation
times, so an xAI placeholder keeps the time the proxy registered it instead of
the read time, while a provider's own earlier creation time still wins

* test(integration): an xAI placeholder row keeps its registration time after the proxy refreshes it

* test(integration): drop the cached output row so the placeholder cell reads the seeded database row

* test(integration): drop the cached output row without requiring it to still exist

* fix(managed_files): limit the batch output size fill, guarded save and kept created_at to xAI rows

* fix(xai): keep the batch id as the batch output filename

Managed xAI batch output files keep the filename main already returns
(the batch id) instead of renaming them to <batch id>_results.jsonl.

---------

Co-authored-by: yucheng-berri <293926326+yucheng-berri@users.noreply.github.com>
2026-10-14 22:05:43 -07:00
..
_assistants_route_errors_support.py fix(proxy): keep the mapped status on assistants and threads route errors (#44994) 2026-10-09 16:55:15 -07:00
_cache_control_marks_support.py fix(responses): propagate cache_control through responses->chat completion bridge (#32511) 2026-10-14 15:44:34 -07:00
_coralbricks.py feat(providers): add CoralBricks as a native provider (#44700) 2026-10-12 21:56:59 +00:00
_count_tokens_system_lift.py fix(anthropic): count a leading system run through count_tokens' system parameter (#45463) 2026-10-08 22:34:25 -07:00
_dashscope_video_support.py feat(dashscope): add Wan 3.0, Wan 2.7 and HappyHorse video generation via /v1/videos (#43706) 2026-10-14 10:10:07 -07:00
_gemini_tts_voice_support.py perf(ci): isolate and shard integration suites (#44146) 2026-10-10 11:15:19 -07:00
_idle_upstream_support.py fix(http): restore the 15s aiohttp keep-alive default in the Python and Rust clients (#46355) 2026-10-14 10:50:11 -07:00
_mantle_gpt_prompt_cache_support.py fix(bedrock_mantle): send OpenAI explicit prompt cache breakpoints for GPT-5.6 and newer (#38729) 2026-10-08 02:07:22 -07:00
_responses_bridge_prompt_cache_breakpoint.py fix(responses): keep prompt_cache_breakpoint markers in the chat to responses bridge (#44119) 2026-10-07 16:25:36 -07:00
_sagemaker_chat_container.py fix(sagemaker): keep AWS credentials out of the sagemaker_chat request body (#45724) 2026-10-10 15:30:54 -07:00
_tool_search_support.py fix(anthropic): keep tool_search_tool_result so the search call replays with its result (#46440) 2026-10-14 18:52:32 -07:00
fine_tuning_proxy_config.yaml test: migrate SDK and proxy-booting legacy CircleCI tests to their tiers (#45758) 2026-10-11 00:17:21 -07:00
openai_fine_tuning.jsonl test: migrate SDK and proxy-booting legacy CircleCI tests to their tiers (#45758) 2026-10-11 00:17:21 -07:00
test_a2a_sdk_upstream_wire.py test: migrate legacy unit dirs and unrun folders into tests/unit, tests/integration and tests/e2e (#45940) 2026-10-11 06:24:57 -07:00
test_anthropic_count_tokens_system_lift_wire.py fix(proxy): raise on provider count_tokens errors instead of falling back to tiktoken (#46121) 2026-10-13 19:08:01 +00:00
test_anthropic_passthrough_wire.py test(integration): cover pass_through endpoint contracts (#44690) 2026-10-14 07:38:17 +00:00
test_anthropic_refusal_stop_details_chaos.py fix(anthropic): keep refusal stop_reason and stop_details on the logged response (#46104) 2026-10-14 14:42:49 -07:00
test_anthropic_refusal_stop_details_wire.py fix(anthropic): keep refusal stop_reason and stop_details on the logged response (#46104) 2026-10-14 14:42:49 -07:00
test_anthropic_skills_create_wire.py test(integration): cover anthropic_messages endpoint contracts (#44704) 2026-10-14 06:57:54 +00:00
test_anthropic_system_cache_control_wire.py test(integration): assert /v1/responses usage reports Anthropic system cache write then read (#42855) 2026-09-23 18:52:06 -07:00
test_anthropic_thinking_signature_logging_wire.py fix(anthropic): stop repeating streamed thinking text in the signature chunk (#44127) 2026-10-02 19:38:41 -07:00
test_anthropic_thinking_signature_stream_wire.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_anthropic_tool_search_result_chaos.py fix(anthropic): keep tool_search_tool_result so the search call replays with its result (#46440) 2026-10-14 18:52:32 -07:00
test_anthropic_tool_search_result_matrix_wire.py fix(anthropic): keep tool_search_tool_result so the search call replays with its result (#46440) 2026-10-14 18:52:32 -07:00
test_anthropic_tool_search_result_modify_params_wire.py fix(anthropic): keep tool_search_tool_result so the search call replays with its result (#46440) 2026-10-14 18:52:32 -07:00
test_anthropic_tool_search_result_passthrough_wire.py fix(anthropic): keep tool_search_tool_result so the search call replays with its result (#46440) 2026-10-14 18:52:32 -07:00
test_anthropic_tool_search_result_wire.py fix(anthropic): keep tool_search_tool_result so the search call replays with its result (#46440) 2026-10-14 18:52:32 -07:00
test_assemblyai_passthrough.py test: migrate SDK and proxy-booting legacy CircleCI tests to their tiers (#45758) 2026-10-11 00:17:21 -07:00
test_assistants_evals_wire.py test(integration): pin assistants and evals endpoint contracts (#44734) 2026-10-14 07:08:35 +00:00
test_assistants_route_errors_chaos.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_assistants_route_errors_wire.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_audit_any_sweep_harness_endpoint.py refactor(types): replace Any with proven types in 9 files (#44389) 2026-10-03 21:03:16 +00:00
test_audit_any_sweep_vector_store.py refactor(types): replace Any with proven types in 9 files (#44389) 2026-10-03 21:03:16 +00:00
test_azure_ai_chat_wire.py test(integration): regression tests for August provider translation and streaming bugs (#42621) 2026-09-22 20:52:00 -07:00
test_azure_ai_claude_messages_wire.py test(integration): cover anthropic_messages endpoint contracts (#44704) 2026-10-14 06:57:54 +00:00
test_azure_ai_flux2_image_wire.py test(integration): regression tests for August provider translation and streaming bugs (#42621) 2026-09-22 20:52:00 -07:00
test_azure_ai_rerank_auth_wire.py test(integration): regression tests for August provider translation and streaming bugs (#42621) 2026-09-22 20:52:00 -07:00
test_azure_embedding_wire.py test(integration): cover embeddings endpoint contracts (#44660) 2026-10-14 03:38:12 +00:00
test_bedrock_anthropic_beta_header_wire.py refactor(types): replace Any with proven types in 157 files (#44798) 2026-10-06 15:31:54 +00:00
test_bedrock_auth_wire.py test(integration): regression tests for August provider translation and streaming bugs (#42621) 2026-09-22 20:52:00 -07:00
test_bedrock_batch_blank_s3_env_wire.py test(e2e): assert only litellm-owned batch behavior and move the blank S3 env pin to an integration test (#43321) 2026-09-26 11:17:03 -07:00
test_bedrock_batch_files_wire.py test(integration): regression tests for August provider translation and streaming bugs (#42621) 2026-09-22 20:52:00 -07:00
test_bedrock_batch_retrieve_status_wire.py fix(bedrock): map Scheduled batch jobs to in_progress on retrieve (#45157) 2026-10-07 18:08:51 -07:00
test_bedrock_claude_thinking_wire.py test(integration): regression tests for August provider translation and streaming bugs (#42621) 2026-09-22 20:52:00 -07:00
test_bedrock_converse_client_metadata_wire.py test(integration): regression tests for July provider translation, routing and streaming bugs (#42693) 2026-09-23 09:51:06 -07:00
test_bedrock_converse_config_blocks_wire.py test(integration): regression tests for August provider translation and streaming bugs (#42621) 2026-09-22 20:52:00 -07:00
test_bedrock_converse_lookaround_regex_chaos.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_bedrock_converse_lookaround_regex_wire.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_bedrock_converse_missing_content_wire.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_bedrock_converse_redacted_thinking_wire.py fix(bedrock)!: replay redacted_thinking on Converse and keep its data as Claude's own string (#46532) 2026-10-14 19:08:14 -07:00
test_bedrock_converse_stream_event_frames_wire.py test(integration): update stale CircleCI integration expectations (#46570) 2026-10-14 18:21:59 -07:00
test_bedrock_deepseek_reasoning_wire.py test(integration): regression tests for July provider translation, routing and streaming bugs (#42693) 2026-09-23 09:51:06 -07:00
test_bedrock_embedding_wire.py test(integration): regression tests for August provider translation and streaming bugs (#42621) 2026-09-22 20:52:00 -07:00
test_bedrock_gpt5_reasoning_wire.py test(integration): regression tests for August provider translation and streaming bugs (#42621) 2026-09-22 20:52:00 -07:00
test_bedrock_gpt_responses_native_wire.py feat(bedrock): serve gpt-5.6+ chat completions natively by default, with chat_completions/ opt-in for gpt-oss and grok (#44307) 2026-10-02 19:55:54 -07:00
test_bedrock_invoke_cache_usage_wire.py test(integration): regression tests for July provider translation, routing and streaming bugs (#42693) 2026-09-23 09:51:06 -07:00
test_bedrock_kimi_k3_cache_point_wire.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_bedrock_knowledge_base_user_context_wire.py test(integration): regression tests for July provider translation, routing and streaming bugs (#42693) 2026-09-23 09:51:06 -07:00
test_bedrock_mantle_claude_chat_chaos.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_bedrock_mantle_claude_chat_wire.py test(integration): assert the Messages API health probe on Mantle Claude (#44612) 2026-10-05 15:28:02 -07:00
test_bedrock_mantle_codex_input_wire.py test(integration): regression tests for July provider translation, routing and streaming bugs (#42693) 2026-09-23 09:51:06 -07:00
test_bedrock_mantle_context_overflow_fallbacks_wire.py test(integration): update stale CircleCI integration expectations (#46570) 2026-10-14 18:21:59 -07:00
test_bedrock_mantle_context_overflow_wire.py fix(bedrock_mantle): map the OpenAI-shaped context overflow envelope too (#41747) 2026-10-12 21:50:22 +00:00
test_bedrock_mantle_count_tokens_wire.py test(integration): update stale CircleCI integration expectations (#46570) 2026-10-14 18:21:59 -07:00
test_bedrock_mantle_gpt_prompt_cache_breakpoint_wire.py fix(bedrock_mantle): send OpenAI explicit prompt cache breakpoints for GPT-5.6 and newer (#38729) 2026-10-08 02:07:22 -07:00
test_bedrock_mantle_output_limit_wire.py test(integration): update stale CircleCI integration expectations (#46570) 2026-10-14 18:21:59 -07:00
test_bedrock_mantle_responses_wire.py test(integration): regression tests for July provider translation, routing and streaming bugs (#42693) 2026-09-23 09:51:06 -07:00
test_bedrock_mantle_wire.py test(integration): regression tests for July provider translation, routing and streaming bugs (#42693) 2026-09-23 09:51:06 -07:00
test_bedrock_marengo_embed_3_wire.py test(integration): regression tests for July provider translation, routing and streaming bugs (#42693) 2026-09-23 09:51:06 -07:00
test_bedrock_model_discovery_chaos.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_bedrock_model_discovery_wire.py fix(bedrock): list the account's invocable models behind bedrock/* when check_provider_endpoint is on (#43981) 2026-10-09 14:14:17 -07:00
test_bedrock_passthrough_stream_wire.py test(integration): cover pass_through endpoint contracts (#44690) 2026-10-14 07:38:17 +00:00
test_bedrock_rerank_wire.py test(integration): regression tests for August provider translation and streaming bugs (#42621) 2026-09-22 20:52:00 -07:00
test_bedrock_role_configuration.py test(integration): regression tests for July provider translation, routing and streaming bugs (#42693) 2026-09-23 09:51:06 -07:00
test_bedrock_runtime_chat_completions_chaos.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_bedrock_runtime_chat_completions_sad_wire.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_bedrock_runtime_chat_completions_wire.py fix(bedrock): drop stop sequences for GPT 5.6 and newer instead of forwarding them to a 400 (#45664) 2026-10-09 15:00:24 -07:00
test_bedrock_stream_timeout_chaos.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_bedrock_stream_timeout_wire.py fix(responses): stop SDK retries nesting under router retries, and unbreak CircleCI integration tests (#44791) 2026-10-06 15:57:33 +00:00
test_bedrock_thinking_tokens_wire.py test(integration): regression tests for July provider translation, routing and streaming bugs (#42693) 2026-09-23 09:51:06 -07:00
test_cache_control_tool_call_marks_owned_proxy.py fix(responses): propagate cache_control through responses->chat completion bridge (#32511) 2026-10-14 15:44:34 -07:00
test_cache_control_tool_call_marks_wire.py test(integration): follow #46101 in the breakpoint cache mark census expectation (#46509) 2026-10-14 20:55:43 -07:00
test_chat_completions_tool_search_chaos.py fix(anthropic)!: translate the OpenAI tool_search tool on /v1/chat/completions for Claude models (#46389) 2026-10-15 04:47:05 +00:00
test_chat_completions_tool_search_history_wire.py fix(anthropic)!: translate the OpenAI tool_search tool on /v1/chat/completions for Claude models (#46389) 2026-10-15 04:47:05 +00:00
test_chat_completions_tool_search_wire.py fix(anthropic)!: translate the OpenAI tool_search tool on /v1/chat/completions for Claude models (#46389) 2026-10-15 04:47:05 +00:00
test_chatgpt_responses_caller_stream_chaos.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_chatgpt_responses_caller_stream_wire.py fix(responses): honor caller stream flag when provider forces SSE (internal copy of #34095) (#41235) 2026-10-06 03:30:47 +00:00
test_chatgpt_responses_top_level_error_wire.py fix(responses): keep the provider code and message on top-level stream error events (#46438) 2026-10-14 16:16:46 -07:00
test_cloudflare_clef_images_chaos.py feat(decisions)!: send images to Cloudflare Clef through its images field (#45734) 2026-10-14 12:14:48 +00:00
test_cloudflare_clef_images_wire.py feat(decisions)!: send images to Cloudflare Clef through its images field (#45734) 2026-10-14 12:14:48 +00:00
test_cloudflare_clef_omni_media_chaos.py feat(decisions): support Clef-omni audio and video (#45811) 2026-10-12 20:29:49 +00:00
test_cloudflare_clef_omni_media_wire.py feat(decisions): support Clef-omni audio and video (#45811) 2026-10-12 20:29:49 +00:00
test_container_file_routes_wire.py refactor(types): replace Any with proven types in 157 files (#44798) 2026-10-06 15:31:54 +00:00
test_coralbricks_provider_chaos.py feat(providers): add CoralBricks as a native provider (#44700) 2026-10-12 21:56:59 +00:00
test_coralbricks_provider_env_wire.py feat(providers): add CoralBricks as a native provider (#44700) 2026-10-12 21:56:59 +00:00
test_coralbricks_provider_wire.py feat(providers): add CoralBricks as a native provider (#44700) 2026-10-12 21:56:59 +00:00
test_cursor_responses_wire.py fix(cache): keep cache hits free of unset fields, and follow intentional behavior changes in CircleCI suites (#46359) 2026-10-14 08:56:05 -07:00
test_dashscope_chat_wire.py fix(proxy): keep provider refusal, logprobs and system_fingerprint in chat completions (#44788) 2026-10-13 13:17:40 -07:00
test_dashscope_video_chaos.py feat(dashscope): add Wan 3.0, Wan 2.7 and HappyHorse video generation via /v1/videos (#43706) 2026-10-14 10:10:07 -07:00
test_dashscope_video_wire.py feat(dashscope): add Wan 3.0, Wan 2.7 and HappyHorse video generation via /v1/videos (#43706) 2026-10-14 10:10:07 -07:00
test_databricks_chat_wire.py test(integration): update stale CircleCI integration expectations (#46570) 2026-10-14 18:21:59 -07:00
test_databricks_decisions_wire.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_databricks_oauth_wire.py test(integration): regression tests for August provider translation and streaming bugs (#42621) 2026-09-22 20:52:00 -07:00
test_decision_model_guardrail.py feat(guardrails): add decision model guardrail with Admin UI flow (#45663) 2026-10-10 16:00:27 -07:00
test_decisions_chaos.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_decisions_openai_format_wire.py fix(cache): keep cache hits free of unset fields, and follow intentional behavior changes in CircleCI suites (#46359) 2026-10-14 08:56:05 -07:00
test_decisions_wire.py feat(decisions): add Laya and Bespoke providers to the Add Model form (#45260) 2026-10-14 08:11:11 -07:00
test_deepseek_vision_wire.py test(integration): regression tests for August provider translation and streaming bugs (#42621) 2026-09-22 20:52:00 -07:00
test_device_code_login_guard_boot.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_device_code_login_guard_wire.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_embedding_cache_partial_hit_wire.py test(integration): cover embeddings endpoint contracts (#44660) 2026-10-14 03:38:12 +00:00
test_embedding_token_array_wire.py test(integration): cover embeddings endpoint contracts (#44660) 2026-10-14 03:38:12 +00:00
test_fake_openai_upstream_routes.py test: migrate local_testing SDK and provider tests out of build_and_test (#45760) 2026-10-10 23:14:25 +00:00
test_fal_ai_chat_wire.py fix(fal_ai): honour global api_base for image generation and reject non-string reasoning_effort with 400 (#42512) 2026-09-22 13:13:54 -07:00
test_fal_ai_image_wire.py test(integration): cover images endpoint contracts (#44653) 2026-10-14 03:40:29 +00:00
test_fal_ai_passthrough_wire.py fix(fal_ai): align /fal_ai queue gate with the pricer and normalise resolution type (#42505) 2026-09-22 13:12:07 -07:00
test_fal_ai_video_wire.py fix(fal_ai): reuse the status client and headers on the video result probe (#42511) 2026-09-22 13:12:51 -07:00
test_files_batches_wire.py test(integration): cover files_batches endpoint contracts (#44703) 2026-10-14 06:57:41 +00:00
test_fireworks_ai_router_slug_wire.py feat(fireworks_ai): route and list the auto, auto-instant and firerouter routers (#43641) 2026-09-28 19:08:16 -07:00
test_fireworks_ai_session_affinity_wire.py test(integration): regression tests for July provider translation, routing and streaming bugs (#42693) 2026-09-23 09:51:06 -07:00
test_gemini_agents_wire.py test(integration): cover google_genai endpoint contracts (#44680) 2026-10-14 04:01:35 +00:00
test_gemini_chat_video_metadata_wire.py feat(gemini): accept file content blocks with video_metadata on multimodal embeddings (#45305) 2026-10-08 13:20:03 -07:00
test_gemini_embedding_file_block_chaos.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_gemini_embedding_file_block_wire.py feat(gemini): accept file content blocks with video_metadata on multimodal embeddings (#45305) 2026-10-08 13:20:03 -07:00
test_gemini_image_config_and_usage_wire.py refactor(types): replace Any with proven types in 157 files (#44798) 2026-10-06 15:31:54 +00:00
test_gemini_interactions_wire.py test(integration): cover google_genai endpoint contracts (#44680) 2026-10-14 04:01:35 +00:00
test_gemini_omitted_temperature.py test(translation): share rich fixtures across Claude, GPT and Gemini providers (#46585) 2026-10-14 19:53:09 -07:00
test_gemini_thinking_replay_chaos.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_gemini_thinking_replay_wire.py test: make 77 legacy live tests offline in litellm_utils, router_unit and responses dirs (#45298) 2026-10-08 02:37:55 -07:00
test_gemini_tts_voice_mapping_owned_proxy.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_gemini_tts_voice_mapping_wire.py fix(gemini): map OpenAI voice names to Gemini prebuilt TTS voices (#45303) 2026-10-09 18:55:22 -07:00
test_google_genai_adapter_wire.py test(integration): cover google_genai endpoint contracts (#44680) 2026-10-14 04:01:35 +00:00
test_google_genai_native_wire.py test(integration): cover google_genai endpoint contracts (#44680) 2026-10-14 04:01:35 +00:00
test_google_genai_proxy_sdk_wire.py test: migrate SDK and proxy-booting legacy CircleCI tests to their tiers (#45758) 2026-10-11 00:17:21 -07:00
test_hosted_vllm_reasoning_content_chaos.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_hosted_vllm_reasoning_content_wire.py test(ci): repair stale tests and flaky CI infrastructure (#43983) 2026-10-01 17:46:43 +00:00
test_idle_upstream_keepalive_edges_wire.py test(integration): update stale CircleCI integration expectations (#46570) 2026-10-14 18:21:59 -07:00
test_idle_upstream_keepalive_override_wire.py fix(http): restore the 15s aiohttp keep-alive default in the Python and Rust clients (#46355) 2026-10-14 10:50:11 -07:00
test_idle_upstream_reconnect_chaos_wire.py fix(http): restore the 15s aiohttp keep-alive default in the Python and Rust clients (#46355) 2026-10-14 10:50:11 -07:00
test_idle_upstream_reconnect_routes_wire.py fix(http): restore the 15s aiohttp keep-alive default in the Python and Rust clients (#46355) 2026-10-14 10:50:11 -07:00
test_idle_upstream_reconnect_rust_messages_wire.py fix(http): restore the 15s aiohttp keep-alive default in the Python and Rust clients (#46355) 2026-10-14 10:50:11 -07:00
test_idle_upstream_reconnect_wire.py fix(http): restore the 15s aiohttp keep-alive default in the Python and Rust clients (#46355) 2026-10-14 10:50:11 -07:00
test_image_gen_drop_params_wire.py test(integration): cover images endpoint contracts (#44653) 2026-10-14 03:40:29 +00:00
test_interactions_openai_bridge_wire.py refactor(types): replace Any with proven types in 157 files (#44798) 2026-10-06 15:31:54 +00:00
test_internal_params_wire.py fix(params): validate stream_chunk_size once, before any provider call (#43222) 2026-09-26 23:01:20 +00:00
test_lm_studio_chat_wire.py test: migrate local_testing SDK and provider tests out of build_and_test (#45760) 2026-10-10 23:14:25 +00:00
test_moderations.py test: migrate SDK and proxy-booting legacy CircleCI tests to their tiers (#45758) 2026-10-11 00:17:21 -07:00
test_moderations_wire.py test(integration): cover small_inference endpoint contracts (#44654) 2026-10-14 04:06:15 +00:00
test_nvidia_nim_ranking_wire.py test(integration): regression tests for July provider translation, routing and streaming bugs (#42693) 2026-09-23 09:51:06 -07:00
test_ocr_multipart_wire.py test(integration): cover small_inference endpoint contracts (#44654) 2026-10-14 04:06:15 +00:00
test_ocr_router_wire.py test: move 81 legacy live tests in pass-through, spend, batches, audio, search, guardrails, image and ocr dirs offline (#45288) 2026-10-08 02:38:12 -07:00
test_ollama_prompt_tools_chaos.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_ollama_prompt_tools_wire.py fix(ollama): one stream id, 400 on non-text tool content, separated tool results, no empty message item (#45177) 2026-10-07 19:08:26 -07:00
test_openai_chat_prompt_cache_breakpoint_wire.py fix(openai): carry cache_control to prompt_cache_breakpoint on chat completions to gpt-5.6 (#46101) 2026-10-13 15:45:49 -07:00
test_openai_chat_wire.py test(integration): cover chat_completions endpoint contracts (#44688) 2026-10-14 03:45:38 +00:00
test_openai_completion_wire.py test(integration): cover chat_completions endpoint contracts (#44688) 2026-10-14 03:45:38 +00:00
test_openai_dialect_prompt_cache_breakpoint_owned_proxy.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_openai_embedding_timeout_wire.py test: migrate local_testing SDK and provider tests out of build_and_test (#45760) 2026-10-10 23:14:25 +00:00
test_openai_embedding_wire.py test(integration): cover embeddings endpoint contracts (#44660) 2026-10-14 03:38:12 +00:00
test_openai_error_handling.py test: migrate SDK and proxy-booting legacy CircleCI tests to their tiers (#45758) 2026-10-11 00:17:21 -07:00
test_openai_fine_tuning.py test: migrate SDK and proxy-booting legacy CircleCI tests to their tiers (#45758) 2026-10-11 00:17:21 -07:00
test_openai_image_edit_wire.py test(integration): cover images endpoint contracts (#44653) 2026-10-14 03:40:29 +00:00
test_openai_image_variation_wire.py test(integration): cover images endpoint contracts (#44653) 2026-10-14 03:40:29 +00:00
test_openai_passthrough_files_wire.py test: move 81 legacy live tests in pass-through, spend, batches, audio, search, guardrails, image and ocr dirs offline (#45288) 2026-10-08 02:38:12 -07:00
test_openai_passthrough_multipart_wire.py test(integration): cover pass_through endpoint contracts (#44690) 2026-10-14 07:38:17 +00:00
test_openai_realtime_transcription_url.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_openai_responses_prompt_cache_breakpoint_wire.py fix(openai): carry cache_control to prompt_cache_breakpoint on /v1/responses to gpt-5.6 (#46105) 2026-10-13 15:23:07 -07:00
test_openai_responses_websocket_wire.py fix(responses): keep the provider code and message on top-level stream error events (#46438) 2026-10-14 16:16:46 -07:00
test_openai_stream_text_usage_wire.py refactor(types): replace Any with proven types in 7 files (#43704) 2026-09-29 06:12:58 -07:00
test_provider_lookup_status.py test(integration): pin assistants and evals endpoint contracts (#44734) 2026-10-14 07:08:35 +00:00
test_rag_ingest_wire.py test(integration): cover vector_stores endpoint contracts (#44739) 2026-10-14 03:59:08 +00:00
test_realtime_transcription_guardrail_session_update.py test(integration): isolate shared proxy state behind six order-dependent flakes (#46172) 2026-10-13 09:55:11 -07:00
test_realtime_unguarded_voice_turns.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_realtime_webrtc_http.py test(integration): cover realtime endpoint contracts (#44689) 2026-10-14 03:49:34 +00:00
test_request_boundary.py fix(tests): resolve the integration support package without run.py's PYTHONPATH 2026-09-15 21:10:08 -07:00
test_rerank_latency_headers_wire.py test(integration): cover small_inference endpoint contracts (#44654) 2026-10-14 04:06:15 +00:00
test_responses_bridge_incomplete.py fix(responses): return the first message item as content for structured output on the chat bridge (#46508) 2026-10-14 16:58:00 -07:00
test_responses_bridge_input_audio_wire.py fix(responses): map input_audio blocks in the chat-to-Responses bridge (#45224) 2026-10-07 22:25:35 -07:00
test_responses_bridge_item_marks_guardrail_wire.py fix(responses): propagate cache_control through responses->chat completion bridge (#32511) 2026-10-14 15:44:34 -07:00
test_responses_bridge_message_cache_control_wire.py fix(responses): propagate cache_control through responses->chat completion bridge (#32511) 2026-10-14 15:44:34 -07:00
test_responses_bridge_message_cache_control_wire_chaos.py fix(responses): propagate cache_control through responses->chat completion bridge (#32511) 2026-10-14 15:44:34 -07:00
test_responses_bridge_namespace_tools.py test(integration): regression tests for July provider translation, routing and streaming bugs (#42693) 2026-09-23 09:51:06 -07:00
test_responses_bridge_prompt_cache_breakpoint.py fix(responses): keep prompt_cache_breakpoint markers in the chat to responses bridge (#44119) 2026-10-07 16:25:36 -07:00
test_responses_bridge_prompt_cache_breakpoint_chaos.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_responses_bridge_prompt_cache_breakpoint_wire.py fix(anthropic): cache tool result screenshots on the /v1/messages to Responses bridge (#46324) 2026-10-14 08:12:11 +00:00
test_responses_bridge_prompt_cache_breakpoint_wire_chaos.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_responses_bridge_stream_output_items_wire.py fix(ollama): one stream id, 400 on non-text tool content, separated tool results, no empty message item (#45177) 2026-10-07 19:08:26 -07:00
test_responses_bridge_tool_search_chaos.py fix(responses bridge): round-trip tool_search_call and tool_search_output items between turns (#46444) 2026-10-14 21:40:14 -07:00
test_responses_bridge_tool_search_replay.py fix(responses bridge): round-trip tool_search_call and tool_search_output items between turns (#46444) 2026-10-14 21:40:14 -07:00
test_responses_bridge_tool_search_replay_wire.py fix(responses): translate OpenAI tool_search to Anthropic tool search for Claude models (#45882) 2026-10-14 20:42:22 -07:00
test_responses_bridge_tool_search_undeferred_wire.py fix(responses): translate OpenAI tool_search to Anthropic tool search for Claude models (#45882) 2026-10-14 20:42:22 -07:00
test_responses_bridge_tool_search_wire.py fix(responses bridge): round-trip tool_search_call and tool_search_output items between turns (#46444) 2026-10-14 21:40:14 -07:00
test_responses_client_header_forwarding_wire.py test(integration): regression tests for July provider translation, routing and streaming bugs (#42693) 2026-09-23 09:51:06 -07:00
test_responses_compact_wire.py fix(proxy): check previous_response_id ownership on /v1/responses/compact (#46408) 2026-10-14 17:57:34 +00:00
test_responses_core_wire.py test(integration): cover responses endpoint contracts (#44733) 2026-10-14 06:59:15 +00:00
test_responses_error_status_wire.py test: move 81 legacy live tests in pass-through, spend, batches, audio, search, guardrails, image and ocr dirs offline (#45288) 2026-10-08 02:38:12 -07:00
test_responses_lifecycle_wire.py test(integration): cover responses endpoint contracts (#44733) 2026-10-14 06:59:15 +00:00
test_responses_minted_reasoning_replay_chaos.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_responses_minted_reasoning_replay_wire.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_responses_reasoning_item_wire.py refactor(types): replace Any with proven types in 157 files (#44798) 2026-10-06 15:31:54 +00:00
test_responses_websocket_session_limit.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_s3_wire.py test: fix stale and flaky tests across CircleCI, GHA and Buildkite (#45530) 2026-10-09 11:40:14 -07:00
test_sagemaker_chat_credential_sources.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_sagemaker_chat_wire.py fix(sagemaker): keep AWS credentials out of the sagemaker_chat request body (#45724) 2026-10-10 15:30:54 -07:00
test_sagemaker_embedding_chaos.py feat(sagemaker)!: inference components and OpenAI-compatible embeddings (#44985) 2026-10-12 22:25:52 +00:00
test_sagemaker_embedding_wire.py feat(sagemaker)!: inference components and OpenAI-compatible embeddings (#44985) 2026-10-12 22:25:52 +00:00
test_search_wire.py test(integration): cover small_inference endpoint contracts (#44654) 2026-10-14 04:06:15 +00:00
test_strands_decider_agentcore_wire.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_tencent_chat_wire.py fix(proxy): keep provider refusal, logprobs and system_fingerprint in chat completions (#44788) 2026-10-13 13:17:40 -07:00
test_validated_reply_boundaries_under_faults.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_vector_store_create_wire.py test(integration): cover vector_stores endpoint contracts (#44739) 2026-10-14 03:59:08 +00:00
test_vector_store_search_wire.py test(integration): cover vector_stores endpoint contracts (#44739) 2026-10-14 03:59:08 +00:00
test_vertex_anthropic_inline_tools_beta_wire.py test(integration): remove six setup causes of CircleCI integration reds (#46490) 2026-10-14 12:00:35 -07:00
test_vertex_anthropic_structured_output_wire.py test(integration): update stale CircleCI integration expectations (#46570) 2026-10-14 18:21:59 -07:00
test_vertex_batch_output_info_wire.py fix(vertex_ai): keep batch output_file_id null until Vertex reports outputInfo (#43030) 2026-09-24 14:05:31 -07:00
test_vertex_embedding_batch_file_block_wire.py feat(gemini): accept file content blocks with video_metadata on multimodal embeddings (#45305) 2026-10-08 13:20:03 -07:00
test_vertex_embedding_file_block_wire.py feat(gemini): accept file content blocks with video_metadata on multimodal embeddings (#45305) 2026-10-08 13:20:03 -07:00
test_vertex_embedding_wire.py test(integration): cover embeddings endpoint contracts (#44660) 2026-10-14 03:38:12 +00:00
test_vertex_gemini_fragmented_stream_wire.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_vertex_gemini_function_call_wire.py refactor(types): replace Any with proven types in 7 files (#43704) 2026-09-29 06:12:58 -07:00
test_vertex_partner_count_tokens_wire.py fix(proxy): raise on provider count_tokens errors instead of falling back to tiktoken (#46121) 2026-10-13 19:08:01 +00:00
test_vertex_partner_messages_wire.py test(integration): cover anthropic_messages endpoint contracts (#44704) 2026-10-14 06:57:54 +00:00
test_vertex_passthrough_wire.py fix(anthropic): keep tool_search_tool_result so the search call replays with its result (#46440) 2026-10-14 18:52:32 -07:00
test_vertex_realtime_multi_region_host.py fix(vertex_ai): dial the multi-region Live API host for realtime sessions (#45166) 2026-10-07 17:39:38 -07:00
test_vertex_timestamps_utc_wire.py fix(vertex_ai): parse Vertex timestamps as UTC so created_at is not shifted by the host timezone (#45550) 2026-10-10 15:18:15 -07:00
test_voyage_mongodb_host_wire.py test(integration): fix three more CircleCI integration reds from test setup (#46510) 2026-10-14 15:23:14 -07:00
test_xai_batch_output_file_wire.py fix(xai): name a completed xAI batch's results file and stop looking it up on xAI's files API (#45559) 2026-10-14 22:05:43 -07:00
test_xai_server_tool_usage_wire.py refactor(types): replace Any with proven types in 157 files (#44798) 2026-10-06 15:31:54 +00:00
test_xai_web_search_wire.py test(integration): regression tests for August provider translation and streaming bugs (#42621) 2026-09-22 20:52:00 -07:00
test_xiaomi_mimo_wire.py fix(proxy): keep provider refusal, logprobs and system_fingerprint in chat completions (#44788) 2026-10-13 13:17:40 -07:00