scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-07 02:59:05 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 103
litellm/tests/e2e/llm_translation
History
Yuneng Jiang 4717cb37b8
test(e2e): serve the vision image from our own fixture 
The two vision tests pointed at a Wikipedia-hosted cat photo, so every run
depended on upload.wikimedia.org staying up and unthrottled. It throttled,
and the 429 surfaced as a bedrock APIConnectionError, which reads as a
gateway failure rather than what it was.

The image is now a fixture in the repo, passed as a data URL. That also puts
the two providers on the same bytes: litellm downloads the image itself for
bedrock, while openai is handed the link and fetches it from its own servers,
so the hosted URL quietly meant the two tests were not testing the same thing.

The image was generated for this repo rather than borrowed, so nothing here
carries a third-party license. Also drops a stale comment about openai prompt
caching that sat above the vision helper; no caching test uses it.

(cherry picked from commit ff418ffb9c)
2026-08-29 16:54:13 -07:00
..
fixtures test(e2e): serve the vision image from our own fixture 2026-08-29 16:54:13 -07:00
realtime fix(e2e): move the vertex realtime suite off the retired Live preview model 2026-08-29 16:54:13 -07:00
conftest.py test(e2e): rename Gateway to ProxyClient and expose it as a session-scoped fixture (#33750) 2026-07-18 18:41:18 +00:00
endpoints_client.py test(e2e): harden the suite against response-cache cross-talk, slow providers and single upstream blips (#37957) 2026-08-22 14:47:03 -07:00
LLM_TRANSLATION_COVERAGE_MATRIX.md test(e2e): add vertex_ai passthrough spend-log coverage (#31781) 2026-07-02 17:32:46 -07:00
passthrough_client.py [e2e] Pin the OpenAI websocket passthrough prefixes 2026-08-21 17:26:18 -07:00
test_audio_speech_e2e.py test(e2e): vendor API testing coverage (#34557) 2026-08-12 01:07:52 +00:00
test_audio_transcriptions_e2e.py test(e2e): accept both model-not-found phrasings on a shared proxy 2026-08-20 02:03:27 -07:00
test_bedrock_native_e2e.py test(e2e): vendor API testing coverage (#34557) 2026-08-12 01:07:52 +00:00
test_bedrock_web_search_server_tool_e2e.py test(e2e): skip the bedrock web search cell the stack cannot provision 2026-08-15 16:09:02 -07:00
test_cache_control.py test(e2e): rename Gateway to ProxyClient and expose it as a session-scoped fixture (#33750) 2026-07-18 18:41:18 +00:00
test_chat_completions_contract_e2e.py test(e2e): record and replay the non-streaming provider flows 2026-08-21 18:50:41 -07:00
test_chat_completions_regression_e2e.py test(e2e): serve the vision image from our own fixture 2026-08-29 16:54:13 -07:00
test_chat_stream_contract_e2e.py test(e2e): vendor API testing coverage (#34557) 2026-08-12 01:07:52 +00:00
test_completions_endpoint_e2e.py test(e2e): cover legacy text /completions endpoint (#34431) 2026-08-04 13:48:07 -07:00
test_credential_messages_e2e.py test(e2e): cover credential-backed /v1/messages request (#33863) 2026-07-18 18:30:55 -07:00
test_custom_pricing_e2e.py test(e2e): rename Gateway to ProxyClient and expose it as a session-scoped fixture (#33750) 2026-07-18 18:41:18 +00:00
test_deepseek_reasoning_e2e.py test(e2e): point four suites at models the providers still serve (#34567) 2026-07-25 10:32:08 -07:00
test_embeddings_endpoint_e2e.py test(e2e): record and replay the non-streaming provider flows 2026-08-21 18:50:41 -07:00
test_files_batches_contract_e2e.py test(e2e): vendor API testing coverage (#34557) 2026-08-12 01:07:52 +00:00
test_google_native_e2e.py test(e2e): cover google-native generateContent framing and prometheus queue time (#34650) 2026-08-12 01:28:26 +00:00
test_image_edits_e2e.py test(e2e): vendor API testing coverage (#34557) 2026-08-12 01:07:52 +00:00
test_image_generation_e2e.py test(e2e): vendor API testing coverage (#34557) 2026-08-12 01:07:52 +00:00
test_messages_azure_foundry_e2e.py test(e2e): fail the run when a Rust gateway silently serves /messages through Python (#34208) 2026-07-22 14:43:13 -07:00
test_messages_e2e.py test(e2e): record and replay the non-streaming provider flows 2026-08-21 18:50:41 -07:00
test_messages_mid_conversation_system_e2e.py test(e2e): require consecutive cache reads before the reminder turn 2026-08-20 03:03:08 -07:00
test_messages_mid_conversation_system_native_providers_e2e.py test(e2e): pin the unflagged Vertex cache test to us-east5 2026-08-20 03:13:14 -07:00
test_moderations_e2e.py test(e2e): vendor API testing coverage (#34557) 2026-08-12 01:07:52 +00:00
test_ocr_rust_e2e.py test(e2e): vendor API testing coverage (#34557) 2026-08-12 01:07:52 +00:00
test_passthrough_e2e.py test(e2e): drop the REALTIME_MODEL comment 2026-08-21 17:38:23 -07:00
test_passthrough_headers_e2e.py test(e2e): skip passthrough headers test until stage can route custom paths to provider creds (#34980) 2026-07-28 11:36:09 -07:00
test_provider_features_e2e.py test(e2e): rename Gateway to ProxyClient and expose it as a session-scoped fixture (#33750) 2026-07-18 18:41:18 +00:00
test_realtime_http_e2e.py test(e2e): vendor API testing coverage (#34557) 2026-08-12 01:07:52 +00:00
test_rerank_e2e.py test(e2e): point four suites at models the providers still serve (#34567) 2026-07-25 10:32:08 -07:00
test_responses_bridge_streaming_e2e.py fix(responses_bridge): keep one chat completion id per stream and always stream completed responses 2026-07-24 20:09:05 +00:00
test_responses_e2e.py test(e2e): vendor API testing coverage (#34557) 2026-08-12 01:07:52 +00:00
test_responses_retrieve_e2e.py test(e2e): vendor API testing coverage (#34557) 2026-08-12 01:07:52 +00:00
test_vector_stores_e2e.py test(e2e): vendor API testing coverage (#34557) 2026-08-12 01:07:52 +00:00
test_vertex_passthrough_e2e.py test(e2e): settle control-plane writes across every replica, not just one 2026-08-07 19:36:30 -07:00