litellm/tests/integration/translation

Translation tests

These tests check one request through the proxy as literals on a TranslationTestCase: the litellm_endpoint and litellm_request the test sends, the expected_provider_endpoint, expected_provider_headers and expected_provider_request the fake provider must receive, the mock_provider_response it answers with, and the expected_litellm_status_code and expected_litellm_response the test must get back. The runner compares the provider request body, every provider header other than transport headers, and the LiteLLM response body in full, so an added, removed, renamed or moved field fails. Folders follow the client endpoint and then the feature, for example translation/messages/reasoning/. Each endpoint keeps one complete base case per model in <endpoint>/bases/<provider>.py, named <MODEL>_TEST_CASE (for example CLAUDE_SONNET_4_6_TEST_CASE), which <endpoint>/basic/ runs on its own. A feature case is <MODEL>_<SCENARIO>_TEST_CASE = dataclasses.replace(<MODEL>_TEST_CASE, ...), imports the base under its own name rather than as BASE, and lists only the fields it changes

Some response fields are generated by LiteLLM on every run, such as a chat completion's id and created or a response's id, created_at and output message id. A case writes those as unittest.mock.ANY, so the field must still be present but any value passes. Use ANY only for a value LiteLLM generates, never for one copied from the provider reply

Shared templates

Each client endpoint has a request and an expected response in <endpoint>/templates.py. Each upstream provider endpoint has a request and a captured response in templates/<provider>.py. All three client endpoints in the Anthropic pilot share the /v1/messages provider templates. Native Messages reuses the provider response shape. These are plain dictionaries, not builders that call production translators

The base template count is twice the number of client endpoints plus twice the total number of supported upstream endpoint formats. Models select templates and override their differing fields, rather than creating another template per model. Streaming and other distinct shapes need variants

For example, the Chat Completions Sonnet base binds four templates

litellm_request={**CHAT_COMPLETIONS_REQUEST, "model": "anthropic/claude-sonnet-5-5"},
expected_provider_request={**ANTHROPIC_MESSAGES_REQUEST, "model": "claude-sonnet-5-5"},
mock_provider_response={**ANTHROPIC_MESSAGES_RESPONSE, "model": "claude-sonnet-5-5"},
expected_litellm_response={**CHAT_COMPLETIONS_RESPONSE, "model": "anthropic/claude-sonnet-5-5"},

The shared samples come from the Sonnet 5.5 captures. Adding another model reuses the endpoint request, with small explicit overrides for unsupported features. Make a live capture with that exact request, inspect its translation, then override differing provider and client response fields in the model's case. Do not relabel the Sonnet capture as live evidence for another model. A new provider endpoint needs its own wire templates, not another client conversation

The templates omit model, so the case binds the deployment, provider model and returned model explicitly. Keep captured IDs, content, thinking signatures and usage exact. The Messages and Responses pilot cases retain their endpoint-specific capture differences. Native Messages can share unchanged content and usage between the provider and client response, while translated response fields remain independently recorded expectations

Build overrides with dictionary unpacking and new nested values, never mutate a template or its nested values. Do not compute expected translations by calling LiteLLM. When editing a shared template, review and run every case that uses it. Older bases keep their existing captures until migrated with matching live evidence

Rich capture examples

Use the Anthropic CLAUDE_SONNET_5_5_TEST_CASE bases as the rich conversation examples. They contain synthetic system instructions, multiple turns, two read-only tool definitions, a prior tool call and result, and a follow-up request with cache_control. Explicit high reasoning effort uses reasoning_effort for Chat Completions, reasoning.effort for Responses, and native thinking plus output_config.effort for Messages. Their provider bodies were captured separately for each client endpoint, including tool calls, thinking signatures and full usage. Keep the endpoint's native request shape and inspect the translation before recording its expected output. A changed LiteLLM output is not automatically correct just because the proxy returned it

Each rich base captures a real provider cache write. Its derived CLAUDE_SONNET_5_5_CACHE_HIT_TEST_CASE in prompt_caching/ uses the identical request and the subsequent real cache-hit response. CI replays these bodies independently through the fake provider, checking request translation and usage conversion without requiring a live cache or test ordering. cache_control configures the provider's prompt cache, while cache: {"no-cache": true} prevents LiteLLM's response cache from bypassing the provider

Keep specialized feature captures paired with their original requests. If enriching a base changes an inherited request, recapture that feature or override its request explicitly, as the separate Anthropic streaming case does. The maintained provider/model target list lives only in the daily translation-sync automation prompt, not in a second repository manifest

StreamingTranslationTestCase stores the stream request, exact provider request and provider and LiteLLM SSE frames. The runner compares parsed frames exactly and checks the final message they join into against expected_litellm_response, with streaming cases living in <endpoint>/streaming/

Deployments used by translation tests are shared by the whole suite and declared in proxy_config.yaml under model_list, with model_name equal to the litellm model string, api_base: http://127.0.0.1:8191 and a synthetic key. A case names the deployment literally in its client request. The fake provider behind them is the provider fixture: one wire_server on port 8191 inside the pytest process, started the first time a test asks for it. A test queues its replies with provider.expect(...) and reads what the proxy sent with provider.received(). Tests that use it run one at a time. After each test the fixture fails if a queued reply was never requested or a received request was never read, and before each test it fails if a request arrived in between, naming the previous test. It refuses to start under pytest-xdist, so the mcp and cost groups cannot use it. Client requests carry "cache": {"no-cache": True} because the proxy caches responses in Redis

Provider responses in translation cases are captured once from the real provider and stored verbatim. First run the new case against the fake provider; the body the proxy sends is the case's expected_provider_request. Send that exact body to the real provider endpoint with a key from the 1Password Shared vault (/qa-keys), and only store the case when the provider answers 2xx. Keep only the response body and drop every response header, since headers carry account identifiers such as the organization id and rate limits. Never print or save the request headers you sent. Before committing, check the body contains no key and no account identifier, such as an organization id, an AWS account id inside an ARN, a GCP project id or an Azure resource name, and that the prompt is synthetic. Paste the body as the case's mock_provider_response without shortening ids, token counts or signatures, and move long opaque values such as thinking signatures into module-level constants used by both mock_provider_response and expected_litellm_response. Do not commit the script used for the capture