litellm/tests/unit/rust_bridge
maciej-berri 36edf9c4b9
feat(messages): serve azure ai foundry messages on rust only (#51721)
* feat(messages): resolve azure_ai keys and entra tokens like python on rust

The native azure_ai Messages config now authenticates in the order Python's
BaseAzureLLM._base_validate_azure_environment uses. The key comes from the
call, then litellm.api_key, litellm.azure_key, AZURE_OPENAI_API_KEY and
AZURE_API_KEY, each skipped only when empty, so a blank key is sent as given
as Python does. Without a key the call mints an Entra ID token through
litellm-auth-azure from azure_ad_token, AZURE_AD_TOKEN, tenant_id, client_id
and client_secret, the caller's azure_ad_token_provider, OIDC references and
litellm.enable_azure_ad_token_refresh, and sends it as a bearer. Without
either the request goes out unauthenticated and Azure's 401 is mapped as
Python maps it, attributed to azure_ai instead of anthropic.

The bridge reads litellm.api_key, litellm.azure_key and the token refresh
switch from the provider defaults snapshot, and calls the caller's token
provider on the blocking pool so its exception reaches the caller as Python
raises it.

Approved differences that stay: a caller x-api-key or non-blank bearer
replaces the operator key instead of being sent beside it.

Known differences: Python's azure_username/azure_password flow has no
native counterpart, and a token provider combined with an OIDC reference
uses the OIDC token where Python calls the provider.

* fix(messages): let a caller api-key replace the azure_ai operator key on rust

A caller api-key header on an azure_ai Messages call was forwarded next to
the operator's x-api-key, so Azure received both credentials. It now
replaces the operator key and is sent as x-api-key, as Python does for a
lowercase api-key.

Approved difference: Python only matches the lowercase spelling and sends
a mixed-case Api-Key beside the operator key. Rust matches any casing.

* fix(messages): order azure_ai entra id flows exactly as python on rust

Without a key the azure_ai config now walks Python's get_azure_ad_token
order itself instead of letting litellm-auth-azure pick. Every setting is
read from the call and then the secret lookup, as AZURE_API_KEY is.

A static token, the caller's azure_ad_token_provider and client secret run
natively. Username and password, OIDC references, os.environ/ references,
an uncallable provider and token refresh go to Python's own
get_azure_ad_token through the host, as Python does them, so a static token
with refresh on and a provider with an OIDC reference behave as in Python,
and a call Python leaves without a token goes out unauthenticated.

Every azure_ai call now asks the secret lookup for the Entra ID names up
front, where Python asks only when no key is set.

* fix(messages): read azure_ai entra settings where python does on rust

Python's get_azure_ad_token reads AZURE_AD_TOKEN through get_secret_str and
AZURE_TENANT_ID, AZURE_CLIENT_ID, AZURE_CLIENT_SECRET, AZURE_USERNAME,
AZURE_PASSWORD and AZURE_SCOPE with os.getenv
(litellm/llms/azure/common_utils.py:580-588). The azure_ai config now does
the same, reading the process environment only when no key is set, and only
AZURE_AD_TOKEN stays in the secret names, so a value only a secret manager
holds is ignored where Python ignores it.

Approved by the owner, same class as A06-A08: any caller credential header
(x-api-key, api-key or Authorization other than a blank bearer, in any
casing) is the only credential sent, and the Entra ID path never runs for
it. That covers a caller bearer on an Entra-only deployment (A10), which
Python overwrites with the deployment's token, a lowercase authorization
Python joins with the Entra token, and a caller x-api-key Python sends
beside the Entra token.

* fix(messages): look up AZURE_AD_TOKEN only where python does on rust

Python reads AZURE_AD_TOKEN through get_secret_str only when the azure_ai
call has no key, no caller credential and no azure_ad_token param
(litellm/llms/azure/common_utils.py:580). Messages configs can now name
deferred secrets, resolved right before validate_environment and only when
the config names them for that call, so a keyed call or a call carrying its
own credential never asks the secret lookup for the Entra token.

Approved by the owner, same class as A06-A08: a caller Authorization other
than a bearer, such as Basic, replaces the azure_ai operator key instead of
being sent beside it as Python does.

* fix(messages): build azure_ai urls and resolve the base like python on rust

The native azure_ai Messages URL builder now ports Python's
azure_anthropic_messages_url. It cuts at /anthropic in the path only, so a
base whose userinfo starts with "anthropic" no longer sends the key to
http://anthropic/v1/messages (security regression fix). It keeps the base
untrimmed and moves the slash before a query as Python does.

The base resolves as Python's `api_base or AZURE_API_BASE`: a blank call base
is kept, an empty one falls back to the environment, and the environment
value is used as given. With neither, the call raises Python's bare
ValueError ("Missing Azure API Base ..."), flagged as raised by Python and as
a late rejection, so the failure log carries the resolved provider and model
parameters as Python's does.

* fix(messages): send bases the native parser misreads through python's client on rust

A Messages api_base with credentials, no host or a port Python cannot read
now goes through Python's client, as bases without an http(s) scheme or with
whitespace already did. Before, reqwest read such a URL differently from
httpx: for http:///d it sent the request, with the operator key, to host
`d`. This is a security fix for both azure_ai and anthropic; the
anthropic_base_empty_host parity case covers anthropic.

For azure_ai the choice reads the base Python's URL builder reads, falling
back to AZURE_API_BASE, so a padded environment base also goes through
Python's client. Python reads AZURE_API_BASE once: transport_handler returns
the value it read and the bridge seeds the call's secret source with it, so
the native route reuses it rather than reading the secret again. _sent_url
only rewrites anthropic URLs, since the azure_ai builder no longer trims.

Parity: 20 azure_ai and anthropic URL cases added; 65 aspects failed on the
previous extension, and the full messages parity suite now passes.

* fix(messages): look up AZURE_API_BASE only where python does on rust

Python reads AZURE_API_BASE only through the `or` fallback in
`api_base or get_secret_str("AZURE_API_BASE")`. The native azure_ai config
listed it among the secrets it always reads, so a call with its own base
still looked it up through the secret manager. It is now a deferred secret,
read only when the call's base is empty or absent. A blank base is kept
without a lookup, as in Python.

deferred_secret_names now takes the call's api_base, the same base the URL
builder reads. A lookup-counting test covers a call base (0 lookups), an
empty base (1) and no base (1); before this change the call-base case
looked it up once.

* fix(messages): filter azure_ai anthropic-beta through its row like python on rust

Python's handler runs update_headers_with_filtered_beta over the azure_ai row of
anthropic_beta_headers_config.json after the config adds its feature betas, so
unknown and null-mapped betas are dropped whatever the header casing, the
caller's and the auto-added ones alike. Rust forwarded the caller's betas and
every auto-added beta as sent.

A caller anthropic-beta replaces a deployment's extra_headers anthropic-beta
before either lane runs, as Python does, and a Router parity test pins it with
the filter applied to both. The advisor tool already matched, since the bridge
hands it to Python's interceptor and its beta is null in the row, so a parity
case pins it. extra_body, unknown top-level params and the -latest alias also
matched and get parity cases.

* fix(messages): strip beta-gated system fields under drop_params like python on rust

Python's transform runs strip_beta_gated_native_request with the final,
filtered headers, so under drop_params a mid-conversation system turn loses
the clear_at, output_config and tool_addition/tool_removal fields no sent beta
unlocks, and a turn left empty is dropped unless that would end the
conversation on an assistant turn. _normalize_system_role_messages strips a
turn it converts to a user turn again with no betas at all. Rust sent those
fields as they came.

azure_ai now picks its betas from the request as sent and runs the strip,
system-role normalization and the cache_control.scope strip in a new
finalize_request step, once the final header is known. Vertex gets the
conversion-time strip too, since it shares the normalization helper.

The strips and the speed drop log Python's warnings word for word. A
non-object output_format under drop_params is still popped after the headers
were built, so structured-outputs-2025-11-13 is sent without it, as Python
does.

* fix(messages): send the system prompt as text blocks like python on rust

Python's _normalize_system_role_messages always rebuilds the top-level system
as a list of content blocks, so a string system reaches azure_ai and Vertex as
one text block, and the success log, which only lists a string system as a
message, logs the messages as sent. Rust kept the string unless a leading
system turn was hoisted.

Python's billing strip in transform_anthropic_messages_request also pops a
system prompt it leaves empty, an empty string included, as Python does. The
shared helper now does the same for every config that strips billing
metadata: azure_ai, vertex_ai, deepseek, minimax, tencent and edenai, each
with its own test.

* fix(messages): keep non-claude azure_ai messages passthroughs on python

A deployment whose provider has no Messages config of its own, like a
non-Claude azure_ai model, but lists /v1/messages in
model_info.supported_endpoints is forwarded untranslated by Python's
OpenAILikeAnthropicMessagesConfig, with a bearer key to <base>/v1/messages.
Dispatch gave it the MESSAGES target, so a rule that serves azure_ai on Rust
would hand it to the native Anthropic config, which sends x-api-key to
/anthropic/v1/messages and breaks the success log.

Dispatch now targets MESSAGES only when the provider's own config serves
the call, and MESSAGES_PASSTHROUGH for the opt-in passthrough. The catalog
keeps that target on Python next to the chat and Responses bridges, so once
azure_ai is Rust-required these calls resolve to Python and the Rust-only
refusal does not fire for them. Proxy token counting treats both targets as
before.

The parity forced rule now serves a provider natively only for the MESSAGES
target, as a Rust-required rule below the target exclusions does. A pre-request
hook that sets a non-Claude model raises functools.partial's TypeError on both
lanes, as on main, and a parity case pins it.

* fix(messages): raise azure_ai transport and provider errors like python on rust

The bridge mapped a native failure with the whole call as exception_type's
extra_kwargs, so litellm_debug_info gained an API Base line from the named
api_base. Python's entry passes only its **kwargs there, so its debug info
names the model alone. The bridge now passes the same kwargs, for anthropic
and azure_ai alike, and a decoding failure on a stream carries the
deployment's debug info as Python's stream iterator attaches it.

A connect failure to an azure_ai base read from AZURE_API_BASE kept the
native "could not reach the provider" text, since the call had no api_base.
The host now hands Python's failure mapping the secrets it read to send the
call, so the aiohttp and httpx texts name the base the call went to, without
reading AZURE_API_BASE again.

A key or header value with a control character, like a deployment key with
a trailing newline, failed natively as APIConnectionError with the model
stripped of its provider. It is now the InternalServerError 500 Python's
transport raises, with aiohttp's "Forbidden control character detected in
headers" text and the model as called. On the httpx transport Python's text
quotes the header value, which may be the key, so Rust keeps its own text
there.

The parity outcome now compares litellm_debug_info. The caller client's
ReadTimeout from a stalled stream still has none on Rust and is recorded as
a gap.

* fix(messages): send a caller azure_ai api-key without an operator key like python on rust

With no deployment key, Rust renamed a caller api-key header in any casing
to x-api-key. Python's AzureAnthropicMessagesConfig renames only a header
spelled exactly api-key and sends any other casing, like API-Key, as given.
Rust now does the same when no operator key would be sent. With an operator
key the approved behaviour stays, and a caller api-key in any casing replaces
it as x-api-key.

* fix(auth-azure): retry entra token posts like python's azure-identity

A client secret or workload identity token endpoint that keeps answering
500 made the native path POST 9 times over about 80 s, because
azure_identity 1.0 uses the SDK's default exponential retry (8 retries,
200 ms doubling, 60 s cap). Python's azure-identity 1.25.2 posts 4 times
in about 5 s and then raises, so the Rust credential now carries a custom
retry policy with the same numbers.

Python sources in the wt-auth venv
- azure/identity/_internal/pipeline.py:67 builds the token pipeline with
  azure-core's RetryPolicy(**kwargs) and no overrides
- azure/identity/_internal/msal_client.py:98 (client secret) and
  aad_client.py:77 (client assertion, used by workload identity) run the
  token POST with retry_on_methods=["POST"]
- azure/core/pipeline/policies/_retry.py (azure-core 1.41.0) lines 77-95
  give retry_status=3, retry_backoff_factor=0.8, retry_backoff_max=120,
  retried codes 408, 429, 500, 502, 503, 504 and 506-998. Lines 127-143
  sleep 0 before the first retry and factor * 2^(n-1) after that, so
  0 s, 1.6 s, 3.2 s. Lines 455-509 sleep exactly Retry-After (then
  retry-after-ms, x-ms-retry-after-ms) instead of the backoff when it is
  present and non-zero

azure_core only hands the policy a Retry-After value for 429 and 503, so
a Retry-After on another status still falls back to the backoff. Managed
identity keeps the SDK default since Python uses a separate policy there,
and client assertion keeps it because Python mints OIDC tokens through
litellm's own HTTP client. A 401 invalid_client is still one POST.

* fix(auth-azure): post an oidc token exchange once like python

Python exchanges an OIDC assertion through litellm's own HTTPHandler
(_exchange_oidc_token in litellm/llms/azure/common_utils.py), a single
client.post with no retry, so a 503 is one POST. The native client
assertion credential used azure_identity's default retry (9 POSTs over
about 80 s) and now runs with RetryOptions::none().

* fix(auth-azure): honour retry-after on entra token posts like python

azure_core runs a custom RetryPolicy through its own loop, which only
reads Retry-After for 429 and 503 and never retries a status outside the
policy's list. Python's azure-core RetryPolicy (1.41.0,
azure/core/pipeline/policies/_retry.py) retries any 4xx or 5xx that
carries Retry-After (is_retry, lines 210-242) and waits exactly the
Retry-After, retry-after-ms or x-ms-retry-after-ms value when it is
non-zero (get_retry_after in _utils.py:79-96, sleep at lines 455-509).

The client secret and workload identity credentials now run with the
SDK retry turned off and a per-call policy that follows Python's
loop. It keeps separate budgets of 3 for status, connect and read
retries (increment and is_exhausted, lines 265-330), backs off 0 s,
then 0.8 * 2^(n-1) capped at 120 s, and retries 408, 429, 500, 502,
503, 504 and 506-998 for the token POST. Client assertion still posts
once and managed identity keeps the SDK default.

A Retry-After that is neither a number nor an HTTP date now fails the
token request after the first POST with "Invalid HTTP date", as Python
does. azure-core's _parse_http_date (_utils.py:48-57) raises ValueError
from the sleep before the retry, azure-identity's wrap_exceptions
(decorators.py:63-80) turns it into ClientAuthenticationError
"Authentication failed: Invalid HTTP date", and get_azure_ad_token
(common_utils.py:663) raises RuntimeError "Failed to get Azure AD token:
Authentication failed: Invalid HTTP date".

* fix(messages): raise python's entra token errors for native azure_ai flows

On the Messages route every Entra token failure now reaches the caller as
Python's get_azure_ad_token and anthropic_messages raise it, with the same
class, status_code, llm_provider, message and log lines.

A native client secret flow keeps the token endpoint's final answer
(status, headers, body) in a typed EntraTokenFailure from the azure-identity
retry policy. The bridge hands it to the Messages host, which replays it
through azure-identity's own ClientSecretCredential over a transport that
returns that answer, so MSAL and azure-core read it exactly as they read it
off the wire. That covers a rejected secret (EntraRejectionError 401 with
the first AADSTS sentence and GUIDs redacted), an exhausted 5xx ("HTTP
Error: 500"), a malformed Retry-After ("Invalid HTTP date"), non-JSON and
odd JSON bodies, including the content-type dependent decode error. A
transport failure has no answer to replay and keeps Rust's own reason
after "Authentication failed:", so its text differs from requests' text.

The caller's azure_ad_token_provider goes through the helper
get_azure_ad_token now shares (azure_ad_token_failure), so a listed AADSTS
rejection is an AuthenticationError 401 instead of a RuntimeError, and a
non-string token raises the same TypeError. Both log Python's "Error
calling Azure AD token provider" and "returned non-string value" lines.

Every Python exception raised while getting the call's credential,
including the host-minted username/password, OIDC and workload identity
flows, now goes through Python's handler rule: a BaseLLMException is mapped
by exception_type (AuthenticationError 401, BadRequestError 422 with
llm_provider azure_ai), anything else reaches the caller as raised.

The Retry-After date parser stays RFC 7231 only. Python's
email.utils.parsedate_tz also accepts dates without a weekday, two-digit
years, numeric zones and missing seconds, which Rust rejects as an invalid
HTTP date and Python waits on. A NaN or infinite number of seconds now
falls back to the backoff, where Python sleeps 0 for NaN and fails on
infinity.

* fix(auth-azure): wait on retry-after and report connect failures like python

The Entra token retry policy now reads Retry-After the way azure-core
1.41.0 does. Seconds go through Python's float rules, including
underscores between digits, and anything else through a port of
email.utils.parsedate_tz, so a date without a weekday, a two-digit year,
a numeric or named zone, a zone glued to the time, missing seconds, a
dotted time, month before day and RFC 850 dates all wait as Python waits.
A date datetime() rejects (day out of range, an offset of a day or more)
still fails the request after one POST, as do unparseable values.

A wait that time.sleep cannot take (infinity, or 2^63 ns and past) now
fails after one POST as Python does. The host replays that answer with a
sleep that still calls time.sleep for such a wait, so the caller gets
"Authentication failed: timestamp out of range for platform time_t".

A transport failure keeps the token endpoint's host and port. A direct
connect that fails with an OS error, or a failed name lookup glibc words,
reaches Python as urllib3 2 words it after "Authentication failed:", for
example "HTTPSConnection(host='login.microsoftonline.com', port=443):
Failed to establish a new connection: [Errno 111] Connection refused".
Python hits it on MSAL's discovery GET to the same host. Failures through
a proxy, TLS failures, timeouts and resets while reading keep Rust's own
text, since urllib3's wording for them depends on details Rust does not
expose.

* fix(messages): treat a contentless system message as a system turn like python on rust

Python's _normalize_system_role_messages reads only the role, so a role
system message sent without content or with null content is still a system
turn. A leading one is hoisted and adds nothing to the top-level system, and
a later one on a model without mid-conversation system support becomes the
operator note user turn, placed after a tool result it sits before and
dropped under drop_params when its gated field is stripped. Rust failed to
parse the message and sent it raw, which also showed up in the logged
messages. A model that accepts the role still gets it as sent.

* fix(messages): let the provider filter a non-ascii anthropic-beta like python on rust

Python merges every anthropic-beta spelling into one header and the
provider config filters it before the HTTP client encodes it, so on azure_ai
an unknown non-ASCII beta is dropped and the call goes out, with or without
drop_params. Rust held the header back as unencodable before the beta filter
ran and failed the call with the ascii codec error. The beta header now
reaches the provider filter, and the wire check fails the call only when a
value Python's client cannot encode survives it. A value with a lone
surrogate still fails up front since it cannot reach Rust as text.

* fix(messages): log nothing for a stream the caller never read like python on rust

Python's Messages stream iterator only starts on the first read, so a
stream the caller closes or drops unread bills nothing and logs nothing.
Rust billed the empty stream on close and teardown, which logged "Unable to
build complete streaming response" and the read() error from the success
handler.

* fix(messages): log a traceback for failures python raises before the send on rust

Python raises its pre-flight errors, such as the missing azure_ai api_base
ValueError, the missing key AuthenticationError and the Entra token
provider failures, through Python frames, so the failure log and the OTel
span carry a stack trace. Rust built those exceptions without raising them,
so error_information.traceback was empty. The host now raises them through
a Python frame before handing them back.

* fix(messages): return python's streaming response types from native messages streams

Python's Messages handler returns an AnthropicMessagesStreamingResponse for
a stream, so isinstance checks, hidden_params and aclose behave the same for
every caller. The native async entrypoint returned the bare execution
AsyncStream. It now wraps the stream the way the deployment success hook
already did, sharing the stream's hidden params. The entrypoint is shared, so
this moves anthropic as well as azure_ai toward Python.

With a response cache configured Python hands back its caching handler's
types too. A streamed cache hit is now the CachedAnthropicMessagesStreamIterator
Python's handler builds from the cached events, stamped with the metadata
@client stamps, and the native execution closes unread so only that iterator
logs the hit, as Python does. A miss is wrapped in the
AnthropicMessagesStreamCacheWriter Python returns, marked persisted since Rust
stores the stream itself. The parity suite now compares the stream type for
every streamed case.

* fix(messages): read unparsed messages in beta detection and gated strips like python on rust

Python reads beta triggers and gated fields off any message mapping, while
Rust only looked at messages that parsed. A role system message sent without
content, or any message with a content block Rust cannot type, was skipped:

- a message carrying output_config now adds per-turn-control-2026-07-01
- a system message with a tool_reference addition or removal now adds
  mid-conversation-tool-changes-2026-07-01
- a signed compaction block now adds compact-2026-09-04
- under drop_params an unparsed system message loses its gated clear_at,
  output_config and tool changes, is dropped once emptied, and the
  warning names the strip
- an unparsed assistant message counts as the last assistant turn when
  deciding whether a strip would leave a prefill
- an unparsed message loses cache_control.scope on its content blocks

* fix(messages): rewrite and normalize unparsed messages and system prompts like python on rust

Python's sanitizer, advisor and encrypted-reasoning strips and system
normalization read any message mapping, while Rust skipped every message
or system block it could not type:

- an unparsed message now gets the sanitizer's rules: a text block whose
  text is not a non-blank string and an empty unsigned thinking block go,
  a message left with no blocks is dropped, tool ids are sanitized,
  replayed web search results are flattened and provider_specific_fields
  dropped; a message the rewrite leaves parseable is parsed again
- an unparsed assistant turn loses advisor exchanges, and an unparsed
  message loses bridge-tagged reasoning blocks
- a kept redacted_thinking block gets Claude's own data back from a
  Bedrock Converse base64 wrapper, parsed or not
- system is kept as sent when it is a list holding a block that does not
  parse, or an object; billing blocks and cache_control.scope are still
  stripped from such a list
- a leading system turn whose content does not parse is hoisted as sent,
  and a later one becomes the operator note user turn as sent, placed
  after a tool result turn that did not parse either

* fix(messages): keep anthropic's API Base debug line on rust failures

Mapping a native failure with only the Python entry's **kwargs dropped the
named api_base from exception_type's input. That matches azure_ai, whose
Python entry maps with **kwargs alone, but it removed the API Base line that
main's Rust-only anthropic route has always put in litellm_debug_info when the
caller passes api_base.

Python gives no rule that tells the two apart, because the legacy async entry
maps both providers the same way. The difference is that anthropic has no
Python path, so main's native route is its contract. The bridge now maps the
whole call for anthropic and the entry kwargs for every other provider.

The parity harness still uses the legacy Python entry as anthropic's
reference, so it now expects the API Base line main's anthropic route adds.
New cases cover an anthropic call on an azure_ai model failing with api_base,
unary and stream.

* fix(messages): keep anthropic stream types and header errors as main while azure_ai matches python

By the owner's ruling anthropic keeps main's behaviour for two things the azure_ai port had changed for every
native provider.

A native async stream returns Python's AnthropicMessagesStreamingResponse, and with a response cache the
CachedAnthropicMessagesStreamIterator or AnthropicMessagesStreamCacheWriter, only for azure_ai. The host asks
Python for those types only when the call resolves to azure_ai, so anthropic gets main's execution AsyncStream
again, for a cache hit or miss too.

A header value the native client cannot send, like a key with a control character, is Python's
InternalServerError with aiohttp's "Forbidden control character" text only for azure_ai. Anthropic raises
main's APIConnectionError "invalid request: failed to parse header value" with the bug report link again.

The parity stream_type aspect compares azure_ai with Python and expects main's AsyncStream for anthropic,
in the same way the outcome aspect already expects main's API Base debug line. New azure_ai cache hit and
miss stream cases check the cached types.

* fix(messages): keep anthropic as main for azure_ai parity side effects

By the owner's ruling anthropic, which has no Python path, keeps main's native
route; the azure_ai port's Python-parity changes now apply to azure_ai only.

- An unread stream: only azure_ai's starts on the first read and bills
  nothing when closed or dropped unread. The logging adapter learns the
  provider from the execution facts; anthropic bills it as main does.
- Failures before the send: the missing-key AuthenticationError and the
  unmapped builtin errors are raised through a Python frame only for azure_ai;
  anthropic hands them back unraised, without a traceback, as main does.
- A stream's decoding failure carries the deployment's debug info only for
  azure_ai; anthropic raises it bare.
- A non-ASCII anthropic-beta is left to the provider filter only for
  azure_ai. Anthropic holds it back before the beta merge as main does, so
  the error names it ahead of any later unencodable header.
- Unparsed messages: the shared shaping reads messages and system prompts
  Rust could not type only when the provider asks for it
  (UnparsedMessages::Read), which only the azure_ai config does. Elsewhere
  the sanitizer, advisor and encrypted-reasoning strips leave such messages
  as sent, their beta triggers add no beta, a redacted_thinking Converse
  wrapper is kept as sent, and a system prompt that does not parse is
  main's "untagged enum SystemPrompt" request error from messages_body.

Each item has an anthropic test pinned to main's behaviour that fails on the
previous commit, beside the azure_ai cases comparing with Python.

* test(messages): pin the malformed api_base fix for anthropic and azure_ai

d1efdd2683 sends a base the native URL parser reads otherwise than httpx and
aiohttp through Python's client. Parity cases now cover, for anthropic and
azure_ai alike, the three shapes that sent the operator key to an unintended
host natively (an empty host, credentials in the URL, userinfo that reads like
a trusted host) and the two port shapes, asserting that nothing reaches the
upstream and that the caller gets Python's exception. They fail with the
check disabled.

The rust_bridge AGENTS.md states the rule.

* refactor(messages): serve azure_ai messages on rust only and delete its python path

Provider azure_ai on /v1/messages is now RUST_REQUIRED next to anthropic, so a
Claude deployment whose AzureAnthropicMessagesConfig serves the call runs on
Rust without LITELLM_RUST or litellm.rust(True) and never falls back to Python.
The PYTHON_ONLY targets rule still sits above it, so a non-Claude azure_ai
deployment that opts into /v1/messages (MESSAGES_PASSTHROUGH) and an azure_ai
call bridged to Chat Completions or Responses stay on Python. A raising
detector on a Claude azure_ai call gets the same UnsupportedParamsError as
anthropic. MESSAGES_PYTHON_FEATURES is empty, so no feature refuses today.

Deleted Python code that only served azure_ai Messages:
- AzureAnthropicMessagesConfig._remove_scope_from_cache_control, a copy of
  the AnthropicMessagesConfig method it overrode. Its only caller is the
  azure transform_anthropic_messages_request (grep "_remove_scope_from_cache_control"
  litellm/: azure, databricks and vertex call self., the base defines it)

Kept, because the parity harness still runs every azure_ai case through the
Python handler (dispatch._PYTHON_AMESSAGES) as its baseline, and removing them
would send that baseline to api.anthropic.com with anthropic auth:
validate_anthropic_messages_environment, get_complete_url,
transform_anthropic_messages_request and should_strip_billing_metadata on
AzureAnthropicMessagesConfig (grep: only async_anthropic_messages_handler in
llm_http_handler.py calls them, reached from the Python fallback and the
harness). The Foundry beta-row branch was already removed with anthropic, and
resolve_beta_provider's Foundry row now serves only anthropic chat. Count
tokens has its own config and the Rust host reaches no deleted method.

The parity harness drops the rust_providers forcing, so azure_ai cases run
through the shipped rules and assert RUST_REQUIRED like anthropic. The azure
count tokens beta test sends its inference call through a caller client, as
Rust now serves it.

* test(messages): cover the azure_ai agentic follow-up deployment on rust

The Rust host builds an agentic follow-up's kwargs with _agentic_hook_kwargs,
so the follow-up must reach the azure_ai deployment's own api_base with its key.
This replaces the deleted LIT-5418 test, which drove the Python handler.
Dropping api_base from that helper fails it with Missing Azure API Base, and
dropping api_key sends the follow-up without x-api-key.

* fix(auth-azure): retry Entra ID token posts like Python only for azure_ai Messages

The azure-identity retry policy and the single OIDC post now apply only when a
consumer asks for them through AzureAuthInputs.token_retry. The azure_ai
Messages config asks for it, while Key Vault, Redis, blob cache, OCR and the
anthropic federation keep the SDK default as on main. The policy is part of
the credential cache key so consumers sharing a client never share a policy.

* fix(messages): keep azure_ai without a key as main when no Python host is wired in

Entra ID auth for an azure_ai call without a key now runs only when the route
has a Python host. The Rust gateway has none, so such a deployment fails with
the local MissingApiKey main raised instead of going out unauthenticated or
failing on an Azure AD token without a host, and a deployment string named
callable is never read as a caller token provider there.

* fix(messages): log the dropped speed warning only for azure_ai

The shared drop_unsupported_params helper logged Python's speed warning for
every Messages provider. Configs now opt in through logs_dropped_params, which
only azure_ai sets, so anthropic and the other providers log as main.

* test(messages): drop the azure_ai default headers structure test and date pinned error texts

default_headers_match_python only restated a constant. The MSAL, azure-core and
CPython texts the replay tests pin now carry the date they were checked.

* refactor(messages): name the anthropic error-mapping set for what it is

_RUST_ONLY_PROVIDERS held only anthropic, the provider that keeps main's native
error mapping, while azure_ai is Rust-only too. It is now
_MAIN_NATIVE_ERROR_PROVIDERS, and the docstrings no longer call azure_ai a
provider with a Python path.

* docs(messages): keep azure_ai's python request code as the live parity reference

azure_ai Messages stays Rust-required, but its Python request code is kept so
the parity suite runs the live Python leg, as anthropic, Vertex and Bedrock do.

* fix(messages): send azure_ai caller and operator credentials exactly as python does on rust

The owner reversed the earlier approved difference on 2026-11-13 (Maciej).
For azure_ai on /v1/messages every caller, operator and Entra ID credential
combination now sends the header set main's Python sends, quirks included.
Python builds a dict keyed by exact name, so a caller api-key spelled exactly
so stops the operator key and the Entra ID path, the operator key otherwise
goes in as api-key and the Entra ID token as Authorization beside or over the
caller's headers, api-key is renamed to x-api-key unless x-api-key is spelled
exactly so, and the HTTP client joins names that differ only in case.

Ledger rows that change, all previously "Rust sends only the caller credential".
A06 caller x-api-key now goes out beside the operator api-key.
A07 caller X-Api-Key is joined with the operator key in one x-api-key.
A08 caller bearer now goes out beside the operator x-api-key.
A09b caller API-Key (other casing) goes out as api-key beside the operator x-api-key.
A10 a caller Authorization bearer on an Entra-only deployment is overwritten
by the minted Entra ID token.
A10b a caller lowercase authorization is joined with the Entra ID bearer.
A10c a caller x-api-key goes out beside the Entra ID bearer.
A caller non-bearer Authorization (Basic) now goes out beside the operator key.
A09 (an exact api-key replaces the operator key as x-api-key) is unchanged.

AZURE_AD_TOKEN is now read wherever Python reads it, which is any call
without a key, an exact caller api-key or an azure_ad_token param. The parity
harness gaps for these rows are removed so they run as normal cases.

* fix(messages): log the azure_ai call flag on failures before the send

Python's Messages handler sets the aanthropic_messages flag before it gets the
Entra ID credential or refuses an unsupported param, so those failures log it.
The native host now flags azure_ai calls the same way when a token request
fails or the request is refused, and anthropic keeps main's unflagged failure.

* fix(auth-azure): fetch azure_ai entra tokens with python's proxy and ca settings

Python's azure-identity sends the client secret token request through
requests, which reads https_proxy or all_proxy unless its no_proxy matching
skips the host, trusts REQUESTS_CA_BUNDLE or CURL_CA_BUNDLE or certifi's
Mozilla roots, never SSL_CERT_FILE, and times out after 300 seconds.
azure_core's reqwest client read HTTPS_PROXY before https_proxy, matched
NO_PROXY its own way, trusted the system store and SSL_CERT_FILE and timed out
after 20 and 60 seconds.

An explicit token_transport input on the auth request selects a transport
that applies requests' rules over a litellm-http client. Only azure_ai
Messages asks for it, so Key Vault, Azure Redis, blob cache and OCR keep
azure_core's transport.

* fix(messages): word azure_ai entra transport failures as python's requests does

Python's azure-identity reports a failed client secret token request with
requests' own text after "Authentication failed:", while the native flow kept
reqwest's text for anything but a direct connect or name lookup failure.

The requests transport now tags each failed send with the proxy it used, and
classifies what reqwest, hyper and rustls observed into the failures requests
words the same way every time: an unreachable or unresolvable proxy, a 407
answer to CONNECT, a proxy or server closing or resetting the connection, an
EOF in the TLS handshake, an expired, not yet valid or misnamed certificate,
and the 300 second read timeout. The host builds each text from the
urllib3, http.client and ssl exceptions Python raises for it, so build
details such as the _ssl.c line come from the running interpreter. A missing
REQUESTS_CA_BUNDLE path now raises requests' own text too.

* fix(messages): look up AZURE_AD_TOKEN for azure_ai only when a Python host is wired in

The Rust gateway has no Python host, so a key-less azure_ai Messages call
there fails with main's local MissingApiKey before any Entra ID flow. It no
longer asks the secret source for AZURE_AD_TOKEN first, which main never did.
Calls with a Python host read it where Python does, as before.

* fix(messages): forward a caller credential on a key-less azure_ai call without a Python host as main

Without a key or a Python host, as in the Rust gateway, main's key-only
azure_ai check forwarded a caller x-api-key or bearer and failed with
MissingApiKey otherwise. The branch made every such call a missing key, so
the gateway's entra_bearer_without_key case failed; it now forwards those
credentials again. Calls with a Python host are unchanged.

* fix(messages): time an azure_ai exchange through python's client once on rust

A call the Rust host sends through Python's client, such as a malformed or
padded api_base, is already timed by that client's track_llm_api_timing post.
The native logging adapter timed the same exchange again, so the Rust lane
reported two LLM service events and recorded the window twice where Python
reports one. The route host now tells the adapter when it sends through its
own client, and for azure_ai the adapter leaves the timing to it. Anthropic
keeps main's behaviour.

* fix(messages): look up advisor ids in a set like python on rust

* test(messages): cover the native cache, stream and host failure paths
2026-11-14 13:40:38 -08:00
..
host feat(messages): serve azure ai foundry messages on rust only (#51721) 2026-11-14 13:40:38 -08:00
messages feat(messages): serve azure ai foundry messages on rust only (#51721) 2026-11-14 13:40:38 -08:00
ocr refactor(rust-bridge): require the native extension (#48852) 2026-10-28 13:16:56 -07:00
trace refactor(lens): connect LiteLLM to the independent Lens service (#45529) 2026-10-09 16:17:05 -07:00
__init__.py test: migrate phase 15 legacy tests to tests/unit 2026-09-20 11:55:43 +00:00
AGENTS.md refactor(rust-bridge): decide native routing in the catalog, drop the bypass predicates (#46750) 2026-10-20 15:06:24 -07:00
native_route_wheel_test.py fix(bedrock): omit system prompt from audio transcription and report its token usage (#41565) 2026-10-28 19:23:15 -07:00
stubtest.ini test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_bindings.py refactor(rust-bridge): require the native extension (#48852) 2026-10-28 13:16:56 -07:00
test_catalog.py feat(messages): serve anthropic messages on rust only (#51119) 2026-11-12 00:07:20 +00:00
test_clickhouse.py refactor(lens): connect LiteLLM to the independent Lens service (#45529) 2026-10-09 16:17:05 -07:00
test_configuration.py refactor(proxy): let the gunicorn server take a private listener (#50871) 2026-11-11 19:40:00 +00:00
test_detectors.py feat(messages): serve anthropic messages on rust only (#51119) 2026-11-12 00:07:20 +00:00
test_execution.py refactor(rust-bridge): return an unstarted Execution from native entrypoints and drive it in Python (#46299) 2026-10-14 17:50:44 -07:00
test_failures.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_fork_guard.py refactor(rust-bridge): require the native extension (#48852) 2026-10-28 13:16:56 -07:00
test_logger.py refactor(rust_bridge): route every Rust-to-Python call through host modules (#50822) 2026-11-11 01:57:54 +00:00
test_public_call.py fix(rust): run native async calls through python's async wrapper steps (#50423) 2026-11-10 00:24:20 +00:00
test_runtime.py feat(messages): serve anthropic messages on rust only (#51119) 2026-11-12 00:07:20 +00:00
test_token_counter.py fix(proxy): count the request's tool_choice on the token count routes (#51133) 2026-11-14 07:53:27 -05:00
test_tokenizer.py refactor(sdk): separate core AWS and tokenizer dependencies (#44447) 2026-10-08 19:07:57 -07:00
test_verify_linux_native_wheel.py chore: move scripts and runtime sources under litellm (#51006) 2026-11-12 04:19:57 +00:00