# Translation tests These tests check one request through the proxy as literals on a `TranslationTestCase`: the `litellm_endpoint` and `litellm_request` the test sends, the `expected_provider_endpoint`, `expected_provider_headers` and `expected_provider_request` the fake provider must receive, the `mock_provider_response` it answers with, and the `expected_litellm_status_code` and `expected_litellm_response` the test must get back. The runner compares the provider request body, every provider header other than transport headers, and the LiteLLM response body in full, so an added, removed, renamed or moved field fails. Folders follow the client endpoint and then the feature, for example `translation/messages/reasoning/`. Each endpoint keeps one complete base case per model in `/bases/.py`, named `_TEST_CASE` (for example `CLAUDE_SONNET_4_6_TEST_CASE`), which `/basic/` runs on its own. A feature case is `__TEST_CASE = dataclasses.replace(_TEST_CASE, ...)`, imports the base under its own name rather than as `BASE`, and lists only the fields it changes Some response fields are generated by LiteLLM on every run, such as a chat completion's `id` and `created` or a response's `id`, `created_at` and output message `id`. A case writes those as `unittest.mock.ANY`, so the field must still be present but any value passes. Use `ANY` only for a value LiteLLM generates, never for one copied from the provider reply ## Shared templates Each client endpoint has a request and an expected response in `/templates.py`. Each upstream provider endpoint has a request and a captured response in `templates/.py`. All three client endpoints in the Anthropic pilot share the `/v1/messages` provider templates. Native Messages reuses the provider response shape. These are plain dictionaries, not builders that call production translators The base template count is twice the number of client endpoints plus twice the total number of supported upstream endpoint formats. Models select templates and override their differing fields, rather than creating another template per model. Streaming and other distinct shapes need variants For example, the Chat Completions Sonnet base binds four templates ```python litellm_request={**CHAT_COMPLETIONS_REQUEST, "model": "anthropic/claude-sonnet-5-5"}, expected_provider_request={**ANTHROPIC_MESSAGES_REQUEST, "model": "claude-sonnet-5-5"}, mock_provider_response={**ANTHROPIC_MESSAGES_RESPONSE, "model": "claude-sonnet-5-5"}, expected_litellm_response={**CHAT_COMPLETIONS_RESPONSE, "model": "anthropic/claude-sonnet-5-5"}, ``` The shared samples come from the Sonnet 5.5 captures. Adding another model reuses the endpoint request, with small explicit overrides for unsupported features. Make a live capture with that exact request, inspect its translation, then override differing provider and client response fields in the model's case. Do not relabel the Sonnet capture as live evidence for another model. A new provider endpoint needs its own wire templates, not another client conversation The templates omit `model`, so the case binds the deployment, provider model and returned model explicitly. Keep captured IDs, content, thinking signatures and usage exact. The Messages and Responses pilot cases retain their endpoint-specific capture differences. Native Messages can share unchanged content and usage between the provider and client response, while translated response fields remain independently recorded expectations Build overrides with dictionary unpacking and new nested values, never mutate a template or its nested values. Do not compute expected translations by calling LiteLLM. When editing a shared template, review and run every case that uses it. Older bases keep their existing captures until migrated with matching live evidence ## Rich capture examples Use the Anthropic `CLAUDE_SONNET_5_5_TEST_CASE` bases as the rich conversation examples. They contain synthetic system instructions, multiple turns, two read-only tool definitions, a prior tool call and result, and a follow-up request with `cache_control`. Explicit high reasoning effort uses `reasoning_effort` for Chat Completions, `reasoning.effort` for Responses, and native `thinking` plus `output_config.effort` for Messages. Their provider bodies were captured separately for each client endpoint, including tool calls, thinking signatures and full usage. Keep the endpoint's native request shape and inspect the translation before recording its expected output. A changed LiteLLM output is not automatically correct just because the proxy returned it Each rich base captures a real provider cache write. Its derived `CLAUDE_SONNET_5_5_CACHE_HIT_TEST_CASE` in `prompt_caching/` uses the identical request and the subsequent real cache-hit response. CI replays these bodies independently through the fake provider, checking request translation and usage conversion without requiring a live cache or test ordering. `cache_control` configures the provider's prompt cache, while `cache: {"no-cache": true}` prevents LiteLLM's response cache from bypassing the provider Keep specialized feature captures paired with their original requests. If enriching a base changes an inherited request, recapture that feature or override its request explicitly, as the separate Anthropic streaming case does. The maintained provider/model target list lives only in the daily translation-sync automation prompt, not in a second repository manifest `StreamingTranslationTestCase` stores the stream request, exact provider request and provider and LiteLLM SSE frames. The runner compares parsed frames exactly and checks the final message they join into against `expected_litellm_response`, with streaming cases living in `/streaming/` Deployments used by translation tests are shared by the whole suite and declared in `proxy_config.yaml` under `model_list`, with `model_name` equal to the litellm model string, `api_base: http://127.0.0.1:8191` and a synthetic key. A case names the deployment literally in its client request. The fake provider behind them is the `provider` fixture: one `wire_server` on port 8191 inside the pytest process, started the first time a test asks for it. A test queues its replies with `provider.expect(...)` and reads what the proxy sent with `provider.received()`. Tests that use it run one at a time. After each test the fixture fails if a queued reply was never requested or a received request was never read, and before each test it fails if a request arrived in between, naming the previous test. It refuses to start under pytest-xdist, so the `mcp` and `cost` groups cannot use it. Client requests carry `"cache": {"no-cache": True}` because the proxy caches responses in Redis Provider responses in translation cases are captured once from the real provider and stored verbatim. First run the new case against the fake provider; the body the proxy sends is the case's `expected_provider_request`. Send that exact body to the real provider endpoint with a key from the 1Password `Shared` vault (`/qa-keys`), and only store the case when the provider answers 2xx. Keep only the response body and drop every response header, since headers carry account identifiers such as the organization id and rate limits. Never print or save the request headers you sent. Before committing, check the body contains no key and no account identifier, such as an organization id, an AWS account id inside an ARN, a GCP project id or an Azure resource name, and that the prompt is synthetic. Paste the body as the case's `mock_provider_response` without shortening ids, token counts or signatures, and move long opaque values such as thinking signatures into module-level constants used by both `mock_provider_response` and `expected_litellm_response`. Do not commit the script used for the capture