scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-16 23:41:43 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 102
litellm/tests/e2e/llm_translation
History
Yuneng Jiang d6bc8fe289
test(e2e): ask the streamed /v1/messages pin for a reply long enough to span several deltas 
Anthropic now returns the 64-token 'count to 20' reply in one to three content_block_delta events, measured directly against api.anthropic.com and through proxies at 7672399 and 49a1145 alike, so the incrementality assertion (at least two deltas) failed in litellm-e2e builds 125, 130 and the 278 rerun with no proxy change behind it. A 'count to 100' reply at max_tokens 400 arrived in five to fifty deltas across every measured run
2026-09-05 13:23:38 -07:00
..
fixtures test(e2e): serve the vision image from our own fixture 2026-08-27 14:29:53 -07:00
realtime fix(e2e): move the vertex realtime suite off the retired Live preview model 2026-08-26 20:58:38 -07:00
conftest.py test(e2e): rename Gateway to ProxyClient and expose it as a session-scoped fixture (#33750) 2026-07-18 18:41:18 +00:00
endpoints_client.py test(e2e): harden the suite against response-cache cross-talk, slow providers and single upstream blips (#37957) 2026-08-22 14:47:03 -07:00
LLM_TRANSLATION_COVERAGE_MATRIX.md test(e2e): add vertex_ai passthrough spend-log coverage (#31781) 2026-07-02 17:32:46 -07:00
passthrough_client.py test(e2e): correlate anthropic passthrough spend rows by the served message id 2026-09-03 00:46:07 -07:00
test_audio_speech_e2e.py test(e2e): vendor API testing coverage (#34557) 2026-08-12 01:07:52 +00:00
test_audio_transcriptions_e2e.py test(e2e): accept both model-not-found phrasings on a shared proxy 2026-08-20 02:03:27 -07:00
test_bedrock_native_e2e.py test(e2e): vendor API testing coverage (#34557) 2026-08-12 01:07:52 +00:00
test_bedrock_provider_matrix_e2e.py test(e2e): cover the Bedrock provider-feature cells customers run 2026-08-25 10:15:29 -07:00
test_bedrock_web_search_server_tool_e2e.py test(e2e): skip the bedrock web search cell the stack cannot provision 2026-08-15 16:09:02 -07:00
test_cache_control.py test(e2e): cover Anthropic and OpenAI prompt caching, Cohere embeddings, and costed /openai chat passthrough 2026-09-05 10:52:46 -07:00
test_chat_completions_contract_e2e.py ci(e2e): record the e2e suite weekly and replay it on weekdays with zero egress (#38163) 2026-08-24 23:49:03 -04:00
test_chat_completions_regression_e2e.py test(e2e): cover Anthropic /chat/completions streaming and tool calls 2026-09-05 10:37:14 -07:00
test_chat_stream_contract_e2e.py test(e2e): vendor API testing coverage (#34557) 2026-08-12 01:07:52 +00:00
test_completions_endpoint_e2e.py test(e2e): cover legacy text /completions endpoint (#34431) 2026-08-04 13:48:07 -07:00
test_credential_messages_e2e.py test(e2e): cover credential-backed /v1/messages request (#33863) 2026-07-18 18:30:55 -07:00
test_custom_pricing_e2e.py test(e2e): rename Gateway to ProxyClient and expose it as a session-scoped fixture (#33750) 2026-07-18 18:41:18 +00:00
test_deepseek_reasoning_e2e.py test(e2e): point four suites at models the providers still serve (#34567) 2026-07-25 10:32:08 -07:00
test_embeddings_endpoint_e2e.py test(e2e): cover Anthropic and OpenAI prompt caching, Cohere embeddings, and costed /openai chat passthrough 2026-09-05 10:52:46 -07:00
test_files_batches_contract_e2e.py test(e2e): unskip four tests whose blockers no longer hold 2026-08-28 00:01:51 -07:00
test_google_native_e2e.py test(e2e): cover google-native generateContent framing and prometheus queue time (#34650) 2026-08-12 01:28:26 +00:00
test_image_edits_e2e.py test(e2e): vendor API testing coverage (#34557) 2026-08-12 01:07:52 +00:00
test_image_generation_e2e.py test(e2e): vendor API testing coverage (#34557) 2026-08-12 01:07:52 +00:00
test_messages_azure_foundry_e2e.py test(e2e): fail the run when a Rust gateway silently serves /messages through Python (#34208) 2026-07-22 14:43:13 -07:00
test_messages_e2e.py test(e2e): ask the streamed /v1/messages pin for a reply long enough to span several deltas 2026-09-05 13:23:38 -07:00
test_messages_mid_conversation_system_e2e.py fix(e2e): size the mid-conversation-system cache prefix above the minimum deterministically 2026-08-26 23:34:19 -07:00
test_messages_mid_conversation_system_native_providers_e2e.py fix(e2e): size the mid-conversation-system cache prefix above the minimum deterministically 2026-08-26 23:34:19 -07:00
test_moderations_e2e.py test(e2e): vendor API testing coverage (#34557) 2026-08-12 01:07:52 +00:00
test_ocr_rust_e2e.py test(e2e): vendor API testing coverage (#34557) 2026-08-12 01:07:52 +00:00
test_passthrough_e2e.py style(e2e): wrap the openai passthrough content assertion under 120 columns 2026-09-05 11:47:15 -07:00
test_passthrough_headers_e2e.py test(e2e): skip passthrough headers test until stage can route custom paths to provider creds (#34980) 2026-07-28 11:36:09 -07:00
test_provider_features_e2e.py test(e2e): rename Gateway to ProxyClient and expose it as a session-scoped fixture (#33750) 2026-07-18 18:41:18 +00:00
test_realtime_http_e2e.py test(e2e): vendor API testing coverage (#34557) 2026-08-12 01:07:52 +00:00
test_rerank_e2e.py test(e2e): point four suites at models the providers still serve (#34567) 2026-07-25 10:32:08 -07:00
test_responses_bridge_streaming_e2e.py fix(responses_bridge): keep one chat completion id per stream and always stream completed responses 2026-07-24 20:09:05 +00:00
test_responses_e2e.py test(e2e): vendor API testing coverage (#34557) 2026-08-12 01:07:52 +00:00
test_responses_retrieve_e2e.py test(e2e): vendor API testing coverage (#34557) 2026-08-12 01:07:52 +00:00
test_together_ai_e2e.py test(e2e): cover Together reasoning_effort=none, json_schema, and cache-read pricing 2026-08-28 12:49:09 -07:00
test_vector_stores_e2e.py test(e2e): vendor API testing coverage (#34557) 2026-08-12 01:07:52 +00:00
test_vertex_passthrough_e2e.py test(e2e): settle control-plane writes across every replica, not just one 2026-08-07 19:36:30 -07:00