scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-11-14 09:03:16 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 214
litellm/tests/e2e/llm_translation
History
Exact
yuneng-jiang e1c03c5423
fix(e2e): restore main qualification across providers and UI (#49763) 
* test: check an unsupported budget sort field

* test: wait for dashboard navigation readiness

* test: support clustered provider caches with live fallback

* Revert "test: check an unsupported budget sort field"

This reverts commit 9694787bb5.

* test: check an unsupported budget sort field

* test(e2e): refresh retired provider fixtures

* test(e2e): read Langfuse observations through the supported API

* test(e2e): identify trace roots by kind and request path

* fix(e2e): repair UI readiness and regional AWS qualification

* fix(e2e): scope LangSmith batch readback to its workspace

* fix(e2e): use the configured global Bedrock GPT profile

* test(e2e): defer unprovisioned GovCloud coverage

* test(e2e): defer Bedrock Runtime Codex cases blocked by IAM

* test(e2e): defer intermittent Azure translation coverage
2026-11-03 20:38:47 +00:00
..
fixtures test(e2e): serve the vision image from our own fixture 2026-08-27 14:29:53 -07:00
realtime fix(e2e): restore main qualification across providers and UI (#49763) 2026-11-03 20:38:47 +00:00
conftest.py test(e2e): replace custom endpoints_client with provider SDK clients 2026-07-23 03:43:20 +00:00
conversational_matrix.py test(e2e): tag llm_translation tests with Subject metadata and record harness steps (#44950) 2026-10-07 10:30:37 -07:00
LLM_TRANSLATION_COVERAGE_MATRIX.md fix(proxy): record streamed /v1/responses container ownership before the response.completed frame (#43140) 2026-09-25 13:45:43 -07:00
passthrough_client.py test(e2e): tag llm_translation tests with Subject metadata and record harness steps (#44950) 2026-10-07 10:30:37 -07:00
responses_helpers.py fix(providers): route Azure Responses past dated api versions, count Grok reasoning tokens on azure_ai, refresh provider sets after register_model, fix gpt-6.1-sol Azure rows (#46377) 2026-10-16 15:23:46 -07:00
sdk_clients.py test(e2e): send no-cache on cacheable SDK calls and accept Bedrock's 400 on the responses leg 2026-09-21 12:39:16 -07:00
structured_output.py test(e2e): move live-provider legacy tests into tests/e2e (#44120) 2026-10-02 00:02:18 -07:00
test_anthropic_completion_e2e.py test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
test_audio_speech_e2e.py test(e2e): tag llm_translation tests with Subject metadata and record harness steps (#44950) 2026-10-07 10:30:37 -07:00
test_audio_transcriptions_e2e.py feat(realtime)!: support latest OpenAI audio models (#35600) 2026-11-03 07:27:52 +00:00
test_bedrock_converse_redacted_thinking_e2e.py fix(bedrock)!: replay redacted_thinking on Converse and keep its data as Claude's own string (#46532) 2026-10-14 19:08:14 -07:00
test_bedrock_guardrail_config_e2e.py test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
test_bedrock_kimi_k3_messages_e2e.py fix(bedrock): apply the 16 maxTokens floor to Kimi K3 on Converse (#43875) 2026-10-20 13:01:45 +00:00
test_bedrock_mantle_output_limit_e2e.py fix(bedrock_mantle): send max_completion_tokens for gpt-5.6 and gpt-6 chat completions (#43982) 2026-10-11 18:28:48 +00:00
test_bedrock_native_e2e.py test: migrate local_testing SDK and provider tests out of build_and_test (#45760) 2026-10-10 23:14:25 +00:00
test_bedrock_provider_matrix_e2e.py test(e2e): tag llm_translation tests with Subject metadata and record harness steps (#44950) 2026-10-07 10:30:37 -07:00
test_bedrock_web_search_server_tool_e2e.py test(e2e): tag llm_translation tests with Subject metadata and record harness steps (#44950) 2026-10-07 10:30:37 -07:00
test_cache_control.py test(e2e): assert the Vertex cache-creation billing contract in the prompt caching test (#47027) 2026-10-16 18:51:38 -07:00
test_cache_control_injection_tool_calls_e2e.py test(e2e): tag llm_translation tests with Subject metadata and record harness steps (#44950) 2026-10-07 10:30:37 -07:00
test_chat_completion_prediction_e2e.py test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
test_chat_completions_contract_e2e.py test(e2e): tag llm_translation tests with Subject metadata and record harness steps (#44950) 2026-10-07 10:30:37 -07:00
test_chat_completions_regression_e2e.py fix(e2e): restore main qualification across providers and UI (#49763) 2026-11-03 20:38:47 +00:00
test_chat_mid_conversation_system_e2e.py test(e2e): tag llm_translation tests with Subject metadata and record harness steps (#44950) 2026-10-07 10:30:37 -07:00
test_chat_stream_contract_e2e.py test(e2e): tag llm_translation tests with Subject metadata and record harness steps (#44950) 2026-10-07 10:30:37 -07:00
test_chat_tool_round_trip_e2e.py test(e2e): tag llm_translation tests with Subject metadata and record harness steps (#44950) 2026-10-07 10:30:37 -07:00
test_completions_endpoint_e2e.py test(e2e): tag llm_translation tests with Subject metadata and record harness steps (#44950) 2026-10-07 10:30:37 -07:00
test_containers_e2e.py test(e2e): tag llm_translation tests with Subject metadata and record harness steps (#44950) 2026-10-07 10:30:37 -07:00
test_conversational_matrix_e2e.py test(e2e): add conversational matrix across chat, messages and responses (#42359) 2026-09-21 22:32:56 -07:00
test_count_tokens_source_e2e.py feat(count_tokens)!: configurable token_count_framing and token_count_model, count source in the response (#46777) 2026-10-17 17:11:27 -07:00
test_credential_messages_e2e.py test(e2e): tag llm_translation tests with Subject metadata and record harness steps (#44950) 2026-10-07 10:30:37 -07:00
test_custom_pricing_e2e.py fix(e2e): restore main qualification across providers and UI (#49763) 2026-11-03 20:38:47 +00:00
test_deepseek_reasoning_e2e.py test(e2e): tag llm_translation tests with Subject metadata and record harness steps (#44950) 2026-10-07 10:30:37 -07:00
test_embedding_only_supported_endpoints_e2e.py feat(router): opt in to non-chat deployment filtering (#46257) 2026-10-29 08:33:36 -07:00
test_embeddings_endpoint_e2e.py test: migrate local_testing SDK and provider tests out of build_and_test (#45760) 2026-10-10 23:14:25 +00:00
test_files_batches_contract_e2e.py test(e2e): tag llm_translation tests with Subject metadata and record harness steps (#44950) 2026-10-07 10:30:37 -07:00
test_gemini_tool_call_id_length_e2e.py fix(gemini): add thought_signature_in_tool_call_id to keep tool call ids short for clients that cap them (#47692) 2026-10-31 03:20:42 -07:00
test_gemini_url_context_e2e.py test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
test_google_interactions_e2e.py test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
test_google_native_e2e.py fix(e2e): restore main qualification across providers and UI (#49763) 2026-11-03 20:38:47 +00:00
test_image_edits_e2e.py fix(e2e): restore main qualification across providers and UI (#49763) 2026-11-03 20:38:47 +00:00
test_image_generation_e2e.py test(e2e): fix MCP tool search hit matching and the retired Bedrock image model (#46998) 2026-10-16 15:45:39 -07:00
test_messages_azure_foundry_e2e.py test(e2e): tag llm_translation tests with Subject metadata and record harness steps (#44950) 2026-10-07 10:30:37 -07:00
test_messages_bedrock_e2e.py fix(bedrock): keep tool_reference results as text on converse (#44590) 2026-10-09 17:52:17 -07:00
test_messages_beta_headers_e2e.py fix(anthropic)!: sync the beta header allowlist with what anthropic, vertex_ai, and azure_ai accept (#49409) 2026-11-02 05:57:51 -08:00
test_messages_count_tokens_bedrock_e2e.py fix(bedrock): count tokens on bedrock-mantle when bedrock-runtime cannot count a Claude model (#45317) 2026-10-08 15:26:49 -07:00
test_messages_e2e.py test(e2e): tag llm_translation tests with Subject metadata and record harness steps (#44950) 2026-10-07 10:30:37 -07:00
test_messages_mid_conversation_system_e2e.py fix(e2e): restore main qualification across providers and UI (#49763) 2026-11-03 20:38:47 +00:00
test_messages_mid_conversation_system_native_providers_e2e.py test(e2e): tag llm_translation tests with Subject metadata and record harness steps (#44950) 2026-10-07 10:30:37 -07:00
test_messages_prompt_caching_and_tool_search_e2e.py test: migrate SDK and proxy-booting legacy CircleCI tests to their tiers (#45758) 2026-10-11 00:17:21 -07:00
test_moderations_e2e.py test(e2e): tag llm_translation tests with Subject metadata and record harness steps (#44950) 2026-10-07 10:30:37 -07:00
test_ocr_rust_e2e.py test(e2e): tag llm_translation tests with Subject metadata and record harness steps (#44950) 2026-10-07 10:30:37 -07:00
test_ollama_e2e.py test(e2e): cover ollama and ollama_chat on chat completions, responses and messages (#45079) 2026-10-07 20:17:35 +00:00
test_passthrough_e2e.py fix(e2e): restore main qualification across providers and UI (#49763) 2026-11-03 20:38:47 +00:00
test_passthrough_headers_e2e.py test(e2e): tag llm_translation tests with Subject metadata and record harness steps (#44950) 2026-10-07 10:30:37 -07:00
test_provider_features_e2e.py fix(completion): send prompt_cache_retention to OpenAI instead of silently dropping it (#47480) 2026-10-20 09:41:22 -07:00
test_realtime_http_e2e.py feat(realtime)!: support latest OpenAI audio models (#35600) 2026-11-03 07:27:52 +00:00
test_reasoning_effort_e2e.py test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
test_rerank_e2e.py test(e2e): tag llm_translation tests with Subject metadata and record harness steps (#44950) 2026-10-07 10:30:37 -07:00
test_responses_bridge_streaming_e2e.py test(e2e): tag llm_translation tests with Subject metadata and record harness steps (#44950) 2026-10-07 10:30:37 -07:00
test_responses_codex_tool_search_e2e.py feat(responses): support Codex client tool_search on Anthropic and other bridged providers (#48972) 2026-10-29 00:29:40 -07:00
test_responses_e2e.py fix(e2e): restore main qualification across providers and UI (#49763) 2026-11-03 20:38:47 +00:00
test_responses_retrieve_e2e.py fix(spend)!: save streamed /v1/responses spend rows under the encrypted resp_ id the client received (#49493) 2026-11-02 14:13:17 -08:00
test_sagemaker_nova_e2e.py test: migrate local_testing SDK and provider tests out of build_and_test (#45760) 2026-10-10 23:14:25 +00:00
test_together_ai_e2e.py test: fix stale and flaky tests across CircleCI, GHA and Buildkite (#45530) 2026-10-09 11:40:14 -07:00
test_token_counter_gemini_contents_e2e.py test(e2e): tag llm_translation tests with Subject metadata and record harness steps (#44950) 2026-10-07 10:30:37 -07:00
test_vector_stores_e2e.py test(e2e): tag llm_translation tests with Subject metadata and record harness steps (#44950) 2026-10-07 10:30:37 -07:00
test_vertex_capabilities_e2e.py test: migrate local_testing SDK and provider tests out of build_and_test (#45760) 2026-10-10 23:14:25 +00:00
test_vertex_passthrough_e2e.py test(e2e): tag llm_translation tests with Subject metadata and record harness steps (#44950) 2026-10-07 10:30:37 -07:00