devin-ai-integration[bot] 41ca4656cd test(e2e): tolerate provider-side flakes on five full-suite cells (#42628) ... * test(e2e): tolerate provider-side flakes on five full-suite cells Mistral OCR retries a provider-relayed 429 with backoff, the Vertex vision probe turns reasoning off so its 32 tokens go to the answer, the Vertex cache cell spaces eight never-seen prefixes 15s apart around Google's nondeterministic minimum-token rejection and prices the cached tokens instead of prompt_tokens, and the Azure content-policy cell resends the jailbreak prompt while Azure skips its filter * test(e2e): shorten the new helper docstrings * test(e2e): accept a relayed provider 429 on the rust OCR cells The gateway already retries a provider 429 three times per call and the Mistral key is shared across pipelines, so a throttle can hold across all four attempts of the OCR cell. After the bounded retries the cell now accepts the gateway's faithful relay of the provider's 429 (throttling_error, code 429) as its second expected outcome; the gateway's own 429 and any other error still fail the cell at once. * test(e2e): drop the harness unit tests, the live cells cover the helpers --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>		2026-09-22 20:12:14 -07:00
..
fixtures	test(e2e): serve the vision image from our own fixture	2026-08-27 14:29:53 -07:00
realtime	fix(realtime): surface an upstream handshake refusal as an error event and policy close (#42388)	2026-09-22 11:31:15 -07:00
conftest.py	test(e2e): replace custom endpoints_client with provider SDK clients	2026-07-23 03:43:20 +00:00
conversational_matrix.py	test(e2e): add conversational matrix across chat, messages and responses (#42359)	2026-09-21 22:32:56 -07:00
LLM_TRANSLATION_COVERAGE_MATRIX.md	test(e2e): add vertex_ai passthrough spend-log coverage (#31781)	2026-07-02 17:32:46 -07:00
passthrough_client.py	test(e2e): correlate anthropic passthrough spend rows by the served message id	2026-09-03 00:46:07 -07:00
sdk_clients.py	test(e2e): send no-cache on cacheable SDK calls and accept Bedrock's 400 on the responses leg	2026-09-21 12:39:16 -07:00
test_audio_speech_e2e.py	Merge branch 'main' into litellm_e2e_provider_sdk_clients	2026-09-15 06:34:31 -07:00
test_audio_transcriptions_e2e.py	Merge branch 'main' into litellm_e2e_provider_sdk_clients	2026-09-15 06:34:31 -07:00
test_bedrock_native_e2e.py	test(e2e): vendor API testing coverage (#34557)	2026-08-12 01:07:52 +00:00
test_bedrock_provider_matrix_e2e.py	fix(bedrock): keep batch S3 credentials out of chat requests and debug logs	2026-09-21 20:42:04 +00:00
test_bedrock_web_search_server_tool_e2e.py	test(e2e): send no-cache on cacheable SDK calls and accept Bedrock's 400 on the responses leg	2026-09-21 12:39:16 -07:00
test_cache_control.py	test(e2e): tolerate provider-side flakes on five full-suite cells (#42628)	2026-09-22 20:12:14 -07:00
test_chat_completions_contract_e2e.py	ci(e2e): record the e2e suite weekly and replay it on weekdays with zero egress (#38163)	2026-08-24 23:49:03 -04:00
test_chat_completions_regression_e2e.py	test(e2e): tolerate provider-side flakes on five full-suite cells (#42628)	2026-09-22 20:12:14 -07:00
test_chat_stream_contract_e2e.py	test(e2e): verify streamed answers and tool continuation	2026-09-14 22:46:55 -07:00
test_completions_endpoint_e2e.py	test(e2e): send no-cache on cacheable SDK calls and accept Bedrock's 400 on the responses leg	2026-09-21 12:39:16 -07:00
test_conversational_matrix_e2e.py	test(e2e): add conversational matrix across chat, messages and responses (#42359)	2026-09-21 22:32:56 -07:00
test_credential_messages_e2e.py	fix(e2e): route credential, cost map, and UI login calls to the control plane (#42506)	2026-09-22 12:59:56 -07:00
test_custom_pricing_e2e.py	test(e2e): replace custom endpoints_client with provider SDK clients	2026-07-23 03:43:20 +00:00
test_deepseek_reasoning_e2e.py	test(e2e): point four suites at models the providers still serve (#34567)	2026-07-25 10:32:08 -07:00
test_embeddings_endpoint_e2e.py	test(e2e): send no-cache on cacheable SDK calls and accept Bedrock's 400 on the responses leg	2026-09-21 12:39:16 -07:00
test_files_batches_contract_e2e.py	test(e2e): unskip four tests whose blockers no longer hold	2026-08-28 00:01:51 -07:00
test_google_native_e2e.py	Merge branch 'main' into litellm_e2e_provider_sdk_clients	2026-09-15 06:34:31 -07:00
test_image_edits_e2e.py	Merge branch 'main' into litellm_e2e_provider_sdk_clients	2026-09-15 06:34:31 -07:00
test_image_generation_e2e.py	Merge branch 'main' into litellm_e2e_provider_sdk_clients	2026-09-15 06:34:31 -07:00
test_messages_azure_foundry_e2e.py	test(e2e): send no-cache on cacheable SDK calls and accept Bedrock's 400 on the responses leg	2026-09-21 12:39:16 -07:00
test_messages_e2e.py	test(e2e): send no-cache on cacheable SDK calls and accept Bedrock's 400 on the responses leg	2026-09-21 12:39:16 -07:00
test_messages_mid_conversation_system_e2e.py	test(e2e): send no-cache on cacheable SDK calls and accept Bedrock's 400 on the responses leg	2026-09-21 12:39:16 -07:00
test_messages_mid_conversation_system_native_providers_e2e.py	test(e2e): send no-cache on cacheable SDK calls and accept Bedrock's 400 on the responses leg	2026-09-21 12:39:16 -07:00
test_moderations_e2e.py	Merge branch 'main' into litellm_e2e_provider_sdk_clients	2026-09-15 06:34:31 -07:00
test_ocr_rust_e2e.py	test(e2e): tolerate provider-side flakes on five full-suite cells (#42628)	2026-09-22 20:12:14 -07:00
test_passthrough_e2e.py	style(e2e): wrap the openai passthrough content assertion under 120 columns	2026-09-05 11:47:15 -07:00
test_passthrough_headers_e2e.py	Merge branch 'main' into litellm_e2e_provider_sdk_clients	2026-09-15 06:34:31 -07:00
test_provider_features_e2e.py	test(e2e): rename Gateway to ProxyClient and expose it as a session-scoped fixture (#33750)	2026-07-18 18:41:18 +00:00
test_realtime_http_e2e.py	test(e2e): vendor API testing coverage (#34557)	2026-08-12 01:07:52 +00:00
test_rerank_e2e.py	Merge branch 'main' into litellm_e2e_provider_sdk_clients	2026-09-15 06:34:31 -07:00
test_responses_bridge_streaming_e2e.py	fix(mcp): honor an explicit null on toolset update, cover MCP lifecycle e2e (#40022)	2026-09-08 22:50:13 -07:00
test_responses_e2e.py	Merge branch 'main' into litellm_wt_0918_138e	2026-09-21 17:05:35 -07:00
test_responses_retrieve_e2e.py	test(e2e): vendor API testing coverage (#34557)	2026-08-12 01:07:52 +00:00
test_together_ai_e2e.py	feat(e2e): cache exact provider responses for 24 hours	2026-09-15 18:14:18 -07:00
test_vector_stores_e2e.py	test(e2e): vendor API testing coverage (#34557)	2026-08-12 01:07:52 +00:00
test_vertex_passthrough_e2e.py	test(e2e): settle control-plane writes across every replica, not just one	2026-08-07 19:36:30 -07:00