* feat(messages): resolve azure_ai keys and entra tokens like python on rust
The native azure_ai Messages config now authenticates in the order Python's
BaseAzureLLM._base_validate_azure_environment uses. The key comes from the
call, then litellm.api_key, litellm.azure_key, AZURE_OPENAI_API_KEY and
AZURE_API_KEY, each skipped only when empty, so a blank key is sent as given
as Python does. Without a key the call mints an Entra ID token through
litellm-auth-azure from azure_ad_token, AZURE_AD_TOKEN, tenant_id, client_id
and client_secret, the caller's azure_ad_token_provider, OIDC references and
litellm.enable_azure_ad_token_refresh, and sends it as a bearer. Without
either the request goes out unauthenticated and Azure's 401 is mapped as
Python maps it, attributed to azure_ai instead of anthropic.
The bridge reads litellm.api_key, litellm.azure_key and the token refresh
switch from the provider defaults snapshot, and calls the caller's token
provider on the blocking pool so its exception reaches the caller as Python
raises it.
Approved differences that stay: a caller x-api-key or non-blank bearer
replaces the operator key instead of being sent beside it.
Known differences: Python's azure_username/azure_password flow has no
native counterpart, and a token provider combined with an OIDC reference
uses the OIDC token where Python calls the provider.
* fix(messages): let a caller api-key replace the azure_ai operator key on rust
A caller api-key header on an azure_ai Messages call was forwarded next to
the operator's x-api-key, so Azure received both credentials. It now
replaces the operator key and is sent as x-api-key, as Python does for a
lowercase api-key.
Approved difference: Python only matches the lowercase spelling and sends
a mixed-case Api-Key beside the operator key. Rust matches any casing.
* fix(messages): order azure_ai entra id flows exactly as python on rust
Without a key the azure_ai config now walks Python's get_azure_ad_token
order itself instead of letting litellm-auth-azure pick. Every setting is
read from the call and then the secret lookup, as AZURE_API_KEY is.
A static token, the caller's azure_ad_token_provider and client secret run
natively. Username and password, OIDC references, os.environ/ references,
an uncallable provider and token refresh go to Python's own
get_azure_ad_token through the host, as Python does them, so a static token
with refresh on and a provider with an OIDC reference behave as in Python,
and a call Python leaves without a token goes out unauthenticated.
Every azure_ai call now asks the secret lookup for the Entra ID names up
front, where Python asks only when no key is set.
* fix(messages): read azure_ai entra settings where python does on rust
Python's get_azure_ad_token reads AZURE_AD_TOKEN through get_secret_str and
AZURE_TENANT_ID, AZURE_CLIENT_ID, AZURE_CLIENT_SECRET, AZURE_USERNAME,
AZURE_PASSWORD and AZURE_SCOPE with os.getenv
(litellm/llms/azure/common_utils.py:580-588). The azure_ai config now does
the same, reading the process environment only when no key is set, and only
AZURE_AD_TOKEN stays in the secret names, so a value only a secret manager
holds is ignored where Python ignores it.
Approved by the owner, same class as A06-A08: any caller credential header
(x-api-key, api-key or Authorization other than a blank bearer, in any
casing) is the only credential sent, and the Entra ID path never runs for
it. That covers a caller bearer on an Entra-only deployment (A10), which
Python overwrites with the deployment's token, a lowercase authorization
Python joins with the Entra token, and a caller x-api-key Python sends
beside the Entra token.
* fix(messages): look up AZURE_AD_TOKEN only where python does on rust
Python reads AZURE_AD_TOKEN through get_secret_str only when the azure_ai
call has no key, no caller credential and no azure_ad_token param
(litellm/llms/azure/common_utils.py:580). Messages configs can now name
deferred secrets, resolved right before validate_environment and only when
the config names them for that call, so a keyed call or a call carrying its
own credential never asks the secret lookup for the Entra token.
Approved by the owner, same class as A06-A08: a caller Authorization other
than a bearer, such as Basic, replaces the azure_ai operator key instead of
being sent beside it as Python does.
* fix(messages): build azure_ai urls and resolve the base like python on rust
The native azure_ai Messages URL builder now ports Python's
azure_anthropic_messages_url. It cuts at /anthropic in the path only, so a
base whose userinfo starts with "anthropic" no longer sends the key to
http://anthropic/v1/messages (security regression fix). It keeps the base
untrimmed and moves the slash before a query as Python does.
The base resolves as Python's `api_base or AZURE_API_BASE`: a blank call base
is kept, an empty one falls back to the environment, and the environment
value is used as given. With neither, the call raises Python's bare
ValueError ("Missing Azure API Base ..."), flagged as raised by Python and as
a late rejection, so the failure log carries the resolved provider and model
parameters as Python's does.
* fix(messages): send bases the native parser misreads through python's client on rust
A Messages api_base with credentials, no host or a port Python cannot read
now goes through Python's client, as bases without an http(s) scheme or with
whitespace already did. Before, reqwest read such a URL differently from
httpx: for http:///d it sent the request, with the operator key, to host
`d`. This is a security fix for both azure_ai and anthropic; the
anthropic_base_empty_host parity case covers anthropic.
For azure_ai the choice reads the base Python's URL builder reads, falling
back to AZURE_API_BASE, so a padded environment base also goes through
Python's client. Python reads AZURE_API_BASE once: transport_handler returns
the value it read and the bridge seeds the call's secret source with it, so
the native route reuses it rather than reading the secret again. _sent_url
only rewrites anthropic URLs, since the azure_ai builder no longer trims.
Parity: 20 azure_ai and anthropic URL cases added; 65 aspects failed on the
previous extension, and the full messages parity suite now passes.
* fix(messages): look up AZURE_API_BASE only where python does on rust
Python reads AZURE_API_BASE only through the `or` fallback in
`api_base or get_secret_str("AZURE_API_BASE")`. The native azure_ai config
listed it among the secrets it always reads, so a call with its own base
still looked it up through the secret manager. It is now a deferred secret,
read only when the call's base is empty or absent. A blank base is kept
without a lookup, as in Python.
deferred_secret_names now takes the call's api_base, the same base the URL
builder reads. A lookup-counting test covers a call base (0 lookups), an
empty base (1) and no base (1); before this change the call-base case
looked it up once.
* fix(messages): filter azure_ai anthropic-beta through its row like python on rust
Python's handler runs update_headers_with_filtered_beta over the azure_ai row of
anthropic_beta_headers_config.json after the config adds its feature betas, so
unknown and null-mapped betas are dropped whatever the header casing, the
caller's and the auto-added ones alike. Rust forwarded the caller's betas and
every auto-added beta as sent.
A caller anthropic-beta replaces a deployment's extra_headers anthropic-beta
before either lane runs, as Python does, and a Router parity test pins it with
the filter applied to both. The advisor tool already matched, since the bridge
hands it to Python's interceptor and its beta is null in the row, so a parity
case pins it. extra_body, unknown top-level params and the -latest alias also
matched and get parity cases.
* fix(messages): strip beta-gated system fields under drop_params like python on rust
Python's transform runs strip_beta_gated_native_request with the final,
filtered headers, so under drop_params a mid-conversation system turn loses
the clear_at, output_config and tool_addition/tool_removal fields no sent beta
unlocks, and a turn left empty is dropped unless that would end the
conversation on an assistant turn. _normalize_system_role_messages strips a
turn it converts to a user turn again with no betas at all. Rust sent those
fields as they came.
azure_ai now picks its betas from the request as sent and runs the strip,
system-role normalization and the cache_control.scope strip in a new
finalize_request step, once the final header is known. Vertex gets the
conversion-time strip too, since it shares the normalization helper.
The strips and the speed drop log Python's warnings word for word. A
non-object output_format under drop_params is still popped after the headers
were built, so structured-outputs-2025-11-13 is sent without it, as Python
does.
* fix(messages): send the system prompt as text blocks like python on rust
Python's _normalize_system_role_messages always rebuilds the top-level system
as a list of content blocks, so a string system reaches azure_ai and Vertex as
one text block, and the success log, which only lists a string system as a
message, logs the messages as sent. Rust kept the string unless a leading
system turn was hoisted.
Python's billing strip in transform_anthropic_messages_request also pops a
system prompt it leaves empty, an empty string included, as Python does. The
shared helper now does the same for every config that strips billing
metadata: azure_ai, vertex_ai, deepseek, minimax, tencent and edenai, each
with its own test.
* fix(messages): keep non-claude azure_ai messages passthroughs on python
A deployment whose provider has no Messages config of its own, like a
non-Claude azure_ai model, but lists /v1/messages in
model_info.supported_endpoints is forwarded untranslated by Python's
OpenAILikeAnthropicMessagesConfig, with a bearer key to <base>/v1/messages.
Dispatch gave it the MESSAGES target, so a rule that serves azure_ai on Rust
would hand it to the native Anthropic config, which sends x-api-key to
/anthropic/v1/messages and breaks the success log.
Dispatch now targets MESSAGES only when the provider's own config serves
the call, and MESSAGES_PASSTHROUGH for the opt-in passthrough. The catalog
keeps that target on Python next to the chat and Responses bridges, so once
azure_ai is Rust-required these calls resolve to Python and the Rust-only
refusal does not fire for them. Proxy token counting treats both targets as
before.
The parity forced rule now serves a provider natively only for the MESSAGES
target, as a Rust-required rule below the target exclusions does. A pre-request
hook that sets a non-Claude model raises functools.partial's TypeError on both
lanes, as on main, and a parity case pins it.
* fix(messages): raise azure_ai transport and provider errors like python on rust
The bridge mapped a native failure with the whole call as exception_type's
extra_kwargs, so litellm_debug_info gained an API Base line from the named
api_base. Python's entry passes only its **kwargs there, so its debug info
names the model alone. The bridge now passes the same kwargs, for anthropic
and azure_ai alike, and a decoding failure on a stream carries the
deployment's debug info as Python's stream iterator attaches it.
A connect failure to an azure_ai base read from AZURE_API_BASE kept the
native "could not reach the provider" text, since the call had no api_base.
The host now hands Python's failure mapping the secrets it read to send the
call, so the aiohttp and httpx texts name the base the call went to, without
reading AZURE_API_BASE again.
A key or header value with a control character, like a deployment key with
a trailing newline, failed natively as APIConnectionError with the model
stripped of its provider. It is now the InternalServerError 500 Python's
transport raises, with aiohttp's "Forbidden control character detected in
headers" text and the model as called. On the httpx transport Python's text
quotes the header value, which may be the key, so Rust keeps its own text
there.
The parity outcome now compares litellm_debug_info. The caller client's
ReadTimeout from a stalled stream still has none on Rust and is recorded as
a gap.
* fix(messages): send a caller azure_ai api-key without an operator key like python on rust
With no deployment key, Rust renamed a caller api-key header in any casing
to x-api-key. Python's AzureAnthropicMessagesConfig renames only a header
spelled exactly api-key and sends any other casing, like API-Key, as given.
Rust now does the same when no operator key would be sent. With an operator
key the approved behaviour stays, and a caller api-key in any casing replaces
it as x-api-key.
* fix(auth-azure): retry entra token posts like python's azure-identity
A client secret or workload identity token endpoint that keeps answering
500 made the native path POST 9 times over about 80 s, because
azure_identity 1.0 uses the SDK's default exponential retry (8 retries,
200 ms doubling, 60 s cap). Python's azure-identity 1.25.2 posts 4 times
in about 5 s and then raises, so the Rust credential now carries a custom
retry policy with the same numbers.
Python sources in the wt-auth venv
- azure/identity/_internal/pipeline.py:67 builds the token pipeline with
azure-core's RetryPolicy(**kwargs) and no overrides
- azure/identity/_internal/msal_client.py:98 (client secret) and
aad_client.py:77 (client assertion, used by workload identity) run the
token POST with retry_on_methods=["POST"]
- azure/core/pipeline/policies/_retry.py (azure-core 1.41.0) lines 77-95
give retry_status=3, retry_backoff_factor=0.8, retry_backoff_max=120,
retried codes 408, 429, 500, 502, 503, 504 and 506-998. Lines 127-143
sleep 0 before the first retry and factor * 2^(n-1) after that, so
0 s, 1.6 s, 3.2 s. Lines 455-509 sleep exactly Retry-After (then
retry-after-ms, x-ms-retry-after-ms) instead of the backoff when it is
present and non-zero
azure_core only hands the policy a Retry-After value for 429 and 503, so
a Retry-After on another status still falls back to the backoff. Managed
identity keeps the SDK default since Python uses a separate policy there,
and client assertion keeps it because Python mints OIDC tokens through
litellm's own HTTP client. A 401 invalid_client is still one POST.
* fix(auth-azure): post an oidc token exchange once like python
Python exchanges an OIDC assertion through litellm's own HTTPHandler
(_exchange_oidc_token in litellm/llms/azure/common_utils.py), a single
client.post with no retry, so a 503 is one POST. The native client
assertion credential used azure_identity's default retry (9 POSTs over
about 80 s) and now runs with RetryOptions::none().
* fix(auth-azure): honour retry-after on entra token posts like python
azure_core runs a custom RetryPolicy through its own loop, which only
reads Retry-After for 429 and 503 and never retries a status outside the
policy's list. Python's azure-core RetryPolicy (1.41.0,
azure/core/pipeline/policies/_retry.py) retries any 4xx or 5xx that
carries Retry-After (is_retry, lines 210-242) and waits exactly the
Retry-After, retry-after-ms or x-ms-retry-after-ms value when it is
non-zero (get_retry_after in _utils.py:79-96, sleep at lines 455-509).
The client secret and workload identity credentials now run with the
SDK retry turned off and a per-call policy that follows Python's
loop. It keeps separate budgets of 3 for status, connect and read
retries (increment and is_exhausted, lines 265-330), backs off 0 s,
then 0.8 * 2^(n-1) capped at 120 s, and retries 408, 429, 500, 502,
503, 504 and 506-998 for the token POST. Client assertion still posts
once and managed identity keeps the SDK default.
A Retry-After that is neither a number nor an HTTP date now fails the
token request after the first POST with "Invalid HTTP date", as Python
does. azure-core's _parse_http_date (_utils.py:48-57) raises ValueError
from the sleep before the retry, azure-identity's wrap_exceptions
(decorators.py:63-80) turns it into ClientAuthenticationError
"Authentication failed: Invalid HTTP date", and get_azure_ad_token
(common_utils.py:663) raises RuntimeError "Failed to get Azure AD token:
Authentication failed: Invalid HTTP date".
* fix(messages): raise python's entra token errors for native azure_ai flows
On the Messages route every Entra token failure now reaches the caller as
Python's get_azure_ad_token and anthropic_messages raise it, with the same
class, status_code, llm_provider, message and log lines.
A native client secret flow keeps the token endpoint's final answer
(status, headers, body) in a typed EntraTokenFailure from the azure-identity
retry policy. The bridge hands it to the Messages host, which replays it
through azure-identity's own ClientSecretCredential over a transport that
returns that answer, so MSAL and azure-core read it exactly as they read it
off the wire. That covers a rejected secret (EntraRejectionError 401 with
the first AADSTS sentence and GUIDs redacted), an exhausted 5xx ("HTTP
Error: 500"), a malformed Retry-After ("Invalid HTTP date"), non-JSON and
odd JSON bodies, including the content-type dependent decode error. A
transport failure has no answer to replay and keeps Rust's own reason
after "Authentication failed:", so its text differs from requests' text.
The caller's azure_ad_token_provider goes through the helper
get_azure_ad_token now shares (azure_ad_token_failure), so a listed AADSTS
rejection is an AuthenticationError 401 instead of a RuntimeError, and a
non-string token raises the same TypeError. Both log Python's "Error
calling Azure AD token provider" and "returned non-string value" lines.
Every Python exception raised while getting the call's credential,
including the host-minted username/password, OIDC and workload identity
flows, now goes through Python's handler rule: a BaseLLMException is mapped
by exception_type (AuthenticationError 401, BadRequestError 422 with
llm_provider azure_ai), anything else reaches the caller as raised.
The Retry-After date parser stays RFC 7231 only. Python's
email.utils.parsedate_tz also accepts dates without a weekday, two-digit
years, numeric zones and missing seconds, which Rust rejects as an invalid
HTTP date and Python waits on. A NaN or infinite number of seconds now
falls back to the backoff, where Python sleeps 0 for NaN and fails on
infinity.
* fix(auth-azure): wait on retry-after and report connect failures like python
The Entra token retry policy now reads Retry-After the way azure-core
1.41.0 does. Seconds go through Python's float rules, including
underscores between digits, and anything else through a port of
email.utils.parsedate_tz, so a date without a weekday, a two-digit year,
a numeric or named zone, a zone glued to the time, missing seconds, a
dotted time, month before day and RFC 850 dates all wait as Python waits.
A date datetime() rejects (day out of range, an offset of a day or more)
still fails the request after one POST, as do unparseable values.
A wait that time.sleep cannot take (infinity, or 2^63 ns and past) now
fails after one POST as Python does. The host replays that answer with a
sleep that still calls time.sleep for such a wait, so the caller gets
"Authentication failed: timestamp out of range for platform time_t".
A transport failure keeps the token endpoint's host and port. A direct
connect that fails with an OS error, or a failed name lookup glibc words,
reaches Python as urllib3 2 words it after "Authentication failed:", for
example "HTTPSConnection(host='login.microsoftonline.com', port=443):
Failed to establish a new connection: [Errno 111] Connection refused".
Python hits it on MSAL's discovery GET to the same host. Failures through
a proxy, TLS failures, timeouts and resets while reading keep Rust's own
text, since urllib3's wording for them depends on details Rust does not
expose.
* fix(messages): treat a contentless system message as a system turn like python on rust
Python's _normalize_system_role_messages reads only the role, so a role
system message sent without content or with null content is still a system
turn. A leading one is hoisted and adds nothing to the top-level system, and
a later one on a model without mid-conversation system support becomes the
operator note user turn, placed after a tool result it sits before and
dropped under drop_params when its gated field is stripped. Rust failed to
parse the message and sent it raw, which also showed up in the logged
messages. A model that accepts the role still gets it as sent.
* fix(messages): let the provider filter a non-ascii anthropic-beta like python on rust
Python merges every anthropic-beta spelling into one header and the
provider config filters it before the HTTP client encodes it, so on azure_ai
an unknown non-ASCII beta is dropped and the call goes out, with or without
drop_params. Rust held the header back as unencodable before the beta filter
ran and failed the call with the ascii codec error. The beta header now
reaches the provider filter, and the wire check fails the call only when a
value Python's client cannot encode survives it. A value with a lone
surrogate still fails up front since it cannot reach Rust as text.
* fix(messages): log nothing for a stream the caller never read like python on rust
Python's Messages stream iterator only starts on the first read, so a
stream the caller closes or drops unread bills nothing and logs nothing.
Rust billed the empty stream on close and teardown, which logged "Unable to
build complete streaming response" and the read() error from the success
handler.
* fix(messages): log a traceback for failures python raises before the send on rust
Python raises its pre-flight errors, such as the missing azure_ai api_base
ValueError, the missing key AuthenticationError and the Entra token
provider failures, through Python frames, so the failure log and the OTel
span carry a stack trace. Rust built those exceptions without raising them,
so error_information.traceback was empty. The host now raises them through
a Python frame before handing them back.
* fix(messages): return python's streaming response types from native messages streams
Python's Messages handler returns an AnthropicMessagesStreamingResponse for
a stream, so isinstance checks, hidden_params and aclose behave the same for
every caller. The native async entrypoint returned the bare execution
AsyncStream. It now wraps the stream the way the deployment success hook
already did, sharing the stream's hidden params. The entrypoint is shared, so
this moves anthropic as well as azure_ai toward Python.
With a response cache configured Python hands back its caching handler's
types too. A streamed cache hit is now the CachedAnthropicMessagesStreamIterator
Python's handler builds from the cached events, stamped with the metadata
@client stamps, and the native execution closes unread so only that iterator
logs the hit, as Python does. A miss is wrapped in the
AnthropicMessagesStreamCacheWriter Python returns, marked persisted since Rust
stores the stream itself. The parity suite now compares the stream type for
every streamed case.
* fix(messages): read unparsed messages in beta detection and gated strips like python on rust
Python reads beta triggers and gated fields off any message mapping, while
Rust only looked at messages that parsed. A role system message sent without
content, or any message with a content block Rust cannot type, was skipped:
- a message carrying output_config now adds per-turn-control-2026-07-01
- a system message with a tool_reference addition or removal now adds
mid-conversation-tool-changes-2026-07-01
- a signed compaction block now adds compact-2026-09-04
- under drop_params an unparsed system message loses its gated clear_at,
output_config and tool changes, is dropped once emptied, and the
warning names the strip
- an unparsed assistant message counts as the last assistant turn when
deciding whether a strip would leave a prefill
- an unparsed message loses cache_control.scope on its content blocks
* fix(messages): rewrite and normalize unparsed messages and system prompts like python on rust
Python's sanitizer, advisor and encrypted-reasoning strips and system
normalization read any message mapping, while Rust skipped every message
or system block it could not type:
- an unparsed message now gets the sanitizer's rules: a text block whose
text is not a non-blank string and an empty unsigned thinking block go,
a message left with no blocks is dropped, tool ids are sanitized,
replayed web search results are flattened and provider_specific_fields
dropped; a message the rewrite leaves parseable is parsed again
- an unparsed assistant turn loses advisor exchanges, and an unparsed
message loses bridge-tagged reasoning blocks
- a kept redacted_thinking block gets Claude's own data back from a
Bedrock Converse base64 wrapper, parsed or not
- system is kept as sent when it is a list holding a block that does not
parse, or an object; billing blocks and cache_control.scope are still
stripped from such a list
- a leading system turn whose content does not parse is hoisted as sent,
and a later one becomes the operator note user turn as sent, placed
after a tool result turn that did not parse either
* fix(messages): keep anthropic's API Base debug line on rust failures
Mapping a native failure with only the Python entry's **kwargs dropped the
named api_base from exception_type's input. That matches azure_ai, whose
Python entry maps with **kwargs alone, but it removed the API Base line that
main's Rust-only anthropic route has always put in litellm_debug_info when the
caller passes api_base.
Python gives no rule that tells the two apart, because the legacy async entry
maps both providers the same way. The difference is that anthropic has no
Python path, so main's native route is its contract. The bridge now maps the
whole call for anthropic and the entry kwargs for every other provider.
The parity harness still uses the legacy Python entry as anthropic's
reference, so it now expects the API Base line main's anthropic route adds.
New cases cover an anthropic call on an azure_ai model failing with api_base,
unary and stream.
* fix(messages): keep anthropic stream types and header errors as main while azure_ai matches python
By the owner's ruling anthropic keeps main's behaviour for two things the azure_ai port had changed for every
native provider.
A native async stream returns Python's AnthropicMessagesStreamingResponse, and with a response cache the
CachedAnthropicMessagesStreamIterator or AnthropicMessagesStreamCacheWriter, only for azure_ai. The host asks
Python for those types only when the call resolves to azure_ai, so anthropic gets main's execution AsyncStream
again, for a cache hit or miss too.
A header value the native client cannot send, like a key with a control character, is Python's
InternalServerError with aiohttp's "Forbidden control character" text only for azure_ai. Anthropic raises
main's APIConnectionError "invalid request: failed to parse header value" with the bug report link again.
The parity stream_type aspect compares azure_ai with Python and expects main's AsyncStream for anthropic,
in the same way the outcome aspect already expects main's API Base debug line. New azure_ai cache hit and
miss stream cases check the cached types.
* fix(messages): keep anthropic as main for azure_ai parity side effects
By the owner's ruling anthropic, which has no Python path, keeps main's native
route; the azure_ai port's Python-parity changes now apply to azure_ai only.
- An unread stream: only azure_ai's starts on the first read and bills
nothing when closed or dropped unread. The logging adapter learns the
provider from the execution facts; anthropic bills it as main does.
- Failures before the send: the missing-key AuthenticationError and the
unmapped builtin errors are raised through a Python frame only for azure_ai;
anthropic hands them back unraised, without a traceback, as main does.
- A stream's decoding failure carries the deployment's debug info only for
azure_ai; anthropic raises it bare.
- A non-ASCII anthropic-beta is left to the provider filter only for
azure_ai. Anthropic holds it back before the beta merge as main does, so
the error names it ahead of any later unencodable header.
- Unparsed messages: the shared shaping reads messages and system prompts
Rust could not type only when the provider asks for it
(UnparsedMessages::Read), which only the azure_ai config does. Elsewhere
the sanitizer, advisor and encrypted-reasoning strips leave such messages
as sent, their beta triggers add no beta, a redacted_thinking Converse
wrapper is kept as sent, and a system prompt that does not parse is
main's "untagged enum SystemPrompt" request error from messages_body.
Each item has an anthropic test pinned to main's behaviour that fails on the
previous commit, beside the azure_ai cases comparing with Python.
* test(messages): pin the malformed api_base fix for anthropic and azure_ai
d1efdd2683 sends a base the native URL parser reads otherwise than httpx and
aiohttp through Python's client. Parity cases now cover, for anthropic and
azure_ai alike, the three shapes that sent the operator key to an unintended
host natively (an empty host, credentials in the URL, userinfo that reads like
a trusted host) and the two port shapes, asserting that nothing reaches the
upstream and that the caller gets Python's exception. They fail with the
check disabled.
The rust_bridge AGENTS.md states the rule.
* refactor(messages): serve azure_ai messages on rust only and delete its python path
Provider azure_ai on /v1/messages is now RUST_REQUIRED next to anthropic, so a
Claude deployment whose AzureAnthropicMessagesConfig serves the call runs on
Rust without LITELLM_RUST or litellm.rust(True) and never falls back to Python.
The PYTHON_ONLY targets rule still sits above it, so a non-Claude azure_ai
deployment that opts into /v1/messages (MESSAGES_PASSTHROUGH) and an azure_ai
call bridged to Chat Completions or Responses stay on Python. A raising
detector on a Claude azure_ai call gets the same UnsupportedParamsError as
anthropic. MESSAGES_PYTHON_FEATURES is empty, so no feature refuses today.
Deleted Python code that only served azure_ai Messages:
- AzureAnthropicMessagesConfig._remove_scope_from_cache_control, a copy of
the AnthropicMessagesConfig method it overrode. Its only caller is the
azure transform_anthropic_messages_request (grep "_remove_scope_from_cache_control"
litellm/: azure, databricks and vertex call self., the base defines it)
Kept, because the parity harness still runs every azure_ai case through the
Python handler (dispatch._PYTHON_AMESSAGES) as its baseline, and removing them
would send that baseline to api.anthropic.com with anthropic auth:
validate_anthropic_messages_environment, get_complete_url,
transform_anthropic_messages_request and should_strip_billing_metadata on
AzureAnthropicMessagesConfig (grep: only async_anthropic_messages_handler in
llm_http_handler.py calls them, reached from the Python fallback and the
harness). The Foundry beta-row branch was already removed with anthropic, and
resolve_beta_provider's Foundry row now serves only anthropic chat. Count
tokens has its own config and the Rust host reaches no deleted method.
The parity harness drops the rust_providers forcing, so azure_ai cases run
through the shipped rules and assert RUST_REQUIRED like anthropic. The azure
count tokens beta test sends its inference call through a caller client, as
Rust now serves it.
* test(messages): cover the azure_ai agentic follow-up deployment on rust
The Rust host builds an agentic follow-up's kwargs with _agentic_hook_kwargs,
so the follow-up must reach the azure_ai deployment's own api_base with its key.
This replaces the deleted LIT-5418 test, which drove the Python handler.
Dropping api_base from that helper fails it with Missing Azure API Base, and
dropping api_key sends the follow-up without x-api-key.
* fix(auth-azure): retry Entra ID token posts like Python only for azure_ai Messages
The azure-identity retry policy and the single OIDC post now apply only when a
consumer asks for them through AzureAuthInputs.token_retry. The azure_ai
Messages config asks for it, while Key Vault, Redis, blob cache, OCR and the
anthropic federation keep the SDK default as on main. The policy is part of
the credential cache key so consumers sharing a client never share a policy.
* fix(messages): keep azure_ai without a key as main when no Python host is wired in
Entra ID auth for an azure_ai call without a key now runs only when the route
has a Python host. The Rust gateway has none, so such a deployment fails with
the local MissingApiKey main raised instead of going out unauthenticated or
failing on an Azure AD token without a host, and a deployment string named
callable is never read as a caller token provider there.
* fix(messages): log the dropped speed warning only for azure_ai
The shared drop_unsupported_params helper logged Python's speed warning for
every Messages provider. Configs now opt in through logs_dropped_params, which
only azure_ai sets, so anthropic and the other providers log as main.
* test(messages): drop the azure_ai default headers structure test and date pinned error texts
default_headers_match_python only restated a constant. The MSAL, azure-core and
CPython texts the replay tests pin now carry the date they were checked.
* refactor(messages): name the anthropic error-mapping set for what it is
_RUST_ONLY_PROVIDERS held only anthropic, the provider that keeps main's native
error mapping, while azure_ai is Rust-only too. It is now
_MAIN_NATIVE_ERROR_PROVIDERS, and the docstrings no longer call azure_ai a
provider with a Python path.
* docs(messages): keep azure_ai's python request code as the live parity reference
azure_ai Messages stays Rust-required, but its Python request code is kept so
the parity suite runs the live Python leg, as anthropic, Vertex and Bedrock do.
* fix(messages): send azure_ai caller and operator credentials exactly as python does on rust
The owner reversed the earlier approved difference on 2026-11-13 (Maciej).
For azure_ai on /v1/messages every caller, operator and Entra ID credential
combination now sends the header set main's Python sends, quirks included.
Python builds a dict keyed by exact name, so a caller api-key spelled exactly
so stops the operator key and the Entra ID path, the operator key otherwise
goes in as api-key and the Entra ID token as Authorization beside or over the
caller's headers, api-key is renamed to x-api-key unless x-api-key is spelled
exactly so, and the HTTP client joins names that differ only in case.
Ledger rows that change, all previously "Rust sends only the caller credential".
A06 caller x-api-key now goes out beside the operator api-key.
A07 caller X-Api-Key is joined with the operator key in one x-api-key.
A08 caller bearer now goes out beside the operator x-api-key.
A09b caller API-Key (other casing) goes out as api-key beside the operator x-api-key.
A10 a caller Authorization bearer on an Entra-only deployment is overwritten
by the minted Entra ID token.
A10b a caller lowercase authorization is joined with the Entra ID bearer.
A10c a caller x-api-key goes out beside the Entra ID bearer.
A caller non-bearer Authorization (Basic) now goes out beside the operator key.
A09 (an exact api-key replaces the operator key as x-api-key) is unchanged.
AZURE_AD_TOKEN is now read wherever Python reads it, which is any call
without a key, an exact caller api-key or an azure_ad_token param. The parity
harness gaps for these rows are removed so they run as normal cases.
* fix(messages): log the azure_ai call flag on failures before the send
Python's Messages handler sets the aanthropic_messages flag before it gets the
Entra ID credential or refuses an unsupported param, so those failures log it.
The native host now flags azure_ai calls the same way when a token request
fails or the request is refused, and anthropic keeps main's unflagged failure.
* fix(auth-azure): fetch azure_ai entra tokens with python's proxy and ca settings
Python's azure-identity sends the client secret token request through
requests, which reads https_proxy or all_proxy unless its no_proxy matching
skips the host, trusts REQUESTS_CA_BUNDLE or CURL_CA_BUNDLE or certifi's
Mozilla roots, never SSL_CERT_FILE, and times out after 300 seconds.
azure_core's reqwest client read HTTPS_PROXY before https_proxy, matched
NO_PROXY its own way, trusted the system store and SSL_CERT_FILE and timed out
after 20 and 60 seconds.
An explicit token_transport input on the auth request selects a transport
that applies requests' rules over a litellm-http client. Only azure_ai
Messages asks for it, so Key Vault, Azure Redis, blob cache and OCR keep
azure_core's transport.
* fix(messages): word azure_ai entra transport failures as python's requests does
Python's azure-identity reports a failed client secret token request with
requests' own text after "Authentication failed:", while the native flow kept
reqwest's text for anything but a direct connect or name lookup failure.
The requests transport now tags each failed send with the proxy it used, and
classifies what reqwest, hyper and rustls observed into the failures requests
words the same way every time: an unreachable or unresolvable proxy, a 407
answer to CONNECT, a proxy or server closing or resetting the connection, an
EOF in the TLS handshake, an expired, not yet valid or misnamed certificate,
and the 300 second read timeout. The host builds each text from the
urllib3, http.client and ssl exceptions Python raises for it, so build
details such as the _ssl.c line come from the running interpreter. A missing
REQUESTS_CA_BUNDLE path now raises requests' own text too.
* fix(messages): look up AZURE_AD_TOKEN for azure_ai only when a Python host is wired in
The Rust gateway has no Python host, so a key-less azure_ai Messages call
there fails with main's local MissingApiKey before any Entra ID flow. It no
longer asks the secret source for AZURE_AD_TOKEN first, which main never did.
Calls with a Python host read it where Python does, as before.
* fix(messages): forward a caller credential on a key-less azure_ai call without a Python host as main
Without a key or a Python host, as in the Rust gateway, main's key-only
azure_ai check forwarded a caller x-api-key or bearer and failed with
MissingApiKey otherwise. The branch made every such call a missing key, so
the gateway's entra_bearer_without_key case failed; it now forwards those
credentials again. Calls with a Python host are unchanged.
* fix(messages): time an azure_ai exchange through python's client once on rust
A call the Rust host sends through Python's client, such as a malformed or
padded api_base, is already timed by that client's track_llm_api_timing post.
The native logging adapter timed the same exchange again, so the Rust lane
reported two LLM service events and recorded the window twice where Python
reports one. The route host now tells the adapter when it sends through its
own client, and for azure_ai the adapter leaves the timing to it. Anthropic
keeps main's behaviour.
* fix(messages): look up advisor ids in a set like python on rust
* test(messages): cover the native cache, stream and host failure paths
* feat(messages): apply modify_params rewrites on rust
* feat(messages): mint anthropic workload identity tokens on rust
* feat(messages): apply prompt caching injection on rust
* feat(messages): serve the easy python-only detectors on rust
* feat(messages): retry rejected thinking signatures on rust
* feat(messages): serve response cache reads on rust
* feat(messages): retry rejected strict tools on rust
* refactor(messages): delete the python path for anthropic messages
Provider anthropic on /v1/messages is now RUST_REQUIRED, so it runs on Rust
without LITELLM_RUST or litellm.rust(True) and never falls back to Python.
A call the rules would send to Python (a MESSAGES_PYTHON_FEATURES feature or a
detector that raises) gets UnsupportedParamsError naming the feature, raised
from one place in litellm/messages/dispatch.py. Other providers keep the
Python fallback. A call whose arguments the handler signature rejects still
gets Python's TypeError.
Deleted Python code that only served anthropic Messages:
- workload identity in AnthropicMessagesConfig.validate/avalidate (subclasses
were never eligible; the Rust host mints the token itself)
- the anthropic Foundry beta-row and affinity-base branch in
async_anthropic_messages_handler, and the "forward" mode of the beta filter
that only that branch used
The proxy MCP tools detector now uses the light tool precheck so a function
tool never imports the gateway handler, since a raising detector now refuses.
* feat(messages): forward message threads on rust
The Rust path for anthropic /v1/messages dropped the thread body field, so
a create or continue turn reached Anthropic without its thread or the
message-threads-2026-08-12 beta. It now matches the Python handler from
#50976:
- the host forwards thread with the other body params
- a non-null thread adds the message-threads beta
- a well-formed continue turn keeps advisor blocks in history and derives
no compaction, per-turn-control or tool-change beta from its messages
- the recorded httpx_response carries the request that was sent, for JSON
and streamed calls, so the proxy's thread ownership hook resends the
create turn's betas on a continue as it does for Python
Upstream error.details already reached the exception message unchanged, so
thread_not_found relays and keeps the deployment out of cooldown on both
paths. Parity cases cover create, continue, streaming, sync, retries,
thread_not_found and the ownership hook round trip.
* feat(messages): serve mock responses on rust
* feat(messages): send caller clients and unsupported tls through python on rust
* feat(messages): run pre-request hooks on rust
* feat(messages): run agentic loop hooks on rust
* feat(messages): run the safeguards classifier on rust
* feat(messages): run registered interceptors on rust
* fix(messages): match python stream cache gate, logging and interceptor order on rust
Store a streamed Messages response in the cache only when Python would: a
message_stop event line, no provider error frame and at least one frame with
output. Leave dict results untouched by mark_rust_response so hidden params
never reach result dicts, logging objects or hook digests. Ask interceptors
whether they handle the call after the cache lookup, not at decode time, so a
cache hit never calls can_handle. Emit the track_llm_api_timing service event
for the provider exchange. Bill and log a stream superseded by a router
fallback as the failure of its error frame, with the partial usage warning.
Hand litellm_logging_obj to deployment pre-call hooks as @client does, so the
autorouter baseline cache sees retried requests.
* fix(messages): forward python-only json values to anthropic as python does
Accept any max_tokens, sampling scalar and messages collection that
Python main accepts, and carry values serde_json cannot hold (ints beyond
64 bits, NaN and infinities, strings with lone surrogates) as validated
raw JSON leaves written back verbatim on the wire, so the upstream body
matches Python's json.dumps
* fix(messages): match python's body reads, error bodies, encoded heads and tls warnings on rust
* fix(messages): log python's plain provider header names on rust streams
* fix(messages): match python's request params, logged params and validation on rust
Read a non-bool stream by Python truthiness and send it as given, log the
caller's stream flag as Python's handler does, raise the raw Python errors
main raises for malformed messages, tools, model_info and hook-set
litellm_params or providers, read the tool custom-field drop from
drops_unsupported_params, log the caller's nested fields that
additional_drop_params strips and the Python values of raw JSON leaves,
keep a missing stop_reason absent, and map provider-config rejections
(missing max_tokens, unknown reasoning effort) as Python's HTTP handler
maps them, with Python's text, effort order, model and provider
* fix(messages): match python's tls curve names, redirects, auth token, header encoding and timeouts on rust
* fix(messages): match python on hooked streams, empty cache writes and mcp cache hits
* fix(messages): join already buffered stream reads and trim the env api key as python does on rust
* fix(messages): deliver stream bytes read before a cut, keep wide reply numbers and follow python's redirect limit on rust
A stream cut by a transport error now hands the caller the bytes already
buffered before raising, as aiohttp does for a caller that reads them before
the event loop sees the cut, so the frames and their usage reach the caller and
the failure log. Provider replies keep integers wider than 64 bits exactly and
read numbers that overflow a float as infinity, through the response, the
response cache and the logged result. The provider client follows up to 20
redirects as httpx does, and running out raises Python's text. A
provider_specific_header value the ascii codec cannot encode fails before the
send like a caller extra header. The deployment success hook gets the stream as
AnthropicMessagesStreamingResponse, as Python's handler hands it.
* feat(messages): serve proxy mcp tools on rust
* test(messages): cover hook, agentic and interceptor edge cases in the parity suite
* fix(messages): keep provider raw-json marked strings, drop non-string tool_result texts and key clamped streams as python does on rust
* fix(messages): match python's api key encoding, malformed api_base and string provider header errors on rust
A header value Python's client cannot encode as ASCII now fails before the send
whichever header carries it, so a caller api_key with a non-ASCII character
raises Python's InternalServerError instead of a raw UnicodeEncodeError after
the reply. A caller api_base without an http(s) scheme, or with whitespace,
control or non-ASCII characters, is sent through Python's client at the URL
Python builds, so its relative-URL and invalid-URL errors match, and a base
urlsplit cannot split raises Python's bare ValueError before the send. A
provider_specific_header of the wrong shape is scoped through Python's
ProviderSpecificHeaderUtils first, so a plain string raises Python's
AttributeError.
* fix(messages): keep integer temperature and top_p digits a float cannot hold and raise python's typeerror for a string max_tokens while capping the thinking budget on rust
temperature and top_p now deserialize through recognized::exact_float, which keeps an integer
where float(n) != n (beyond 2**53) as an unrecognized value with its exact digits, so it is sent
upstream and logged as the integer python forwards, and comparisons still go through scalar.rs
cap_thinking_budget_to_max_tokens compares max_tokens <= 1024 first, so a max_tokens python
cannot order against an int (a string, list or dict) raises TypeError "'<=' not supported between
instances of 'str' and 'int'" in both the reasoning effort translation and the adaptive effort
reshape. rust now returns ErrorDetail::UnorderableTypes there, the bridge raises it as a python
TypeError marked as a late rejection, and map_failure hands it to the caller unmapped
* test(messages): format the non-string tool_result parity helper
* fix(messages): flag and log the call before python's pre-send raises on rust
The bridge now hands settings() and map_failure() whether the call is async.
A provider_specific_header of the wrong shape sets the aanthropic_messages
flag before re-raising, as Python's handler has already flagged the call. A
base urlsplit cannot split also logs the resolved model, provider and optional
params first, as Python raises it only after its logger is updated. The
TypeError raised while capping the thinking budget for a string max_tokens
sets the flag too
* feat(messages): serve the web search short-circuit override on rust through the intercept host call
* fix(messages): give the web search entry kwargs its mutable-ok reason
* test(messages): drop the body hook gap the rust short-circuit closes
* fix(messages): name the partially unknown handler types on the short-circuit host call
Runs every case through the Python /v1/messages path and the Rust path against one recording upstream and compares dispatch, upstream request, outcome, hidden params, standard logging fields and callback kwargs. Known Rust differences are strict xfails tagged with their inventory ID.
Native Responses now hands back the same managed ids as Python: the
resp_ id carries the provider and deployment, encrypted-content items get
encitem_ ids and wrapped content under affinity routing, and container ids
become cntr_ ids. Encoded encrypted-content items in the input are restored
before the provider request. The codec lives in inference-responses with
goldens taken from litellm/responses/utils.py
Each case runs one aresponses call through Python and through the native
route against the same recorded OpenAI response and compares the dispatch,
provider request, return value and success log. Known gaps are strict xfails
so the PRs that close them must flip the marker
* docs(agents): ban semicolons and colons in human-facing text
Tighten the writing rule so ";" and ":" are only allowed where the sentence would be nonsensical without them, rewrite the existing semicolons in AGENTS.md to follow it, and note that docs (litellm-docs) keep their trailing "."
* docs(agents): rewrite semicolons in nested AGENTS.md files
* docs(agents): hand-maintain the dashboard Next.js warning and sync e2e CONTRIBUTING lines
* docs(agents): point at the PR template path that exists
* docs(agents): restore the generated Next.js block in the dashboard AGENTS.md
* test: move 76 live top-level tests to offline unit and integration coverage
* test: keep the mock-param opt-in guard valid once the top-level senders are gone
* test: address review on offline replacements
* docs(test): drop the deleted harness test file from the harness check command
* test: make the key rebind, team member delete and routes integration tests exercise the legacy paths
* test: make fallback, rpm and spend integration contracts deterministic and clean up their rows, assert the rpm limit in usage-based routing
* test: expect the no-deployments error at the rpm limit and cover the strategy check without pre-call checks
* test: hand member cleanups to the scenario instead of growing a budget list, flatten callback kinds
* test: scope the admin health check to the test's own deployment
---------
Co-authored-by: yuneng <yuneng@berri.ai>
* refactor(ocr): remove the Python OCR execution path and require the Rust route
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fmt
* refactor(ocr): tidy the native OCR passthrough binding
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(ocr): ruff format the azure passthrough transformation
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(ocr): resolve passthrough OCR costing in one Rust call
Replace passthrough_url/passthrough_transform with passthrough_response,
which matches the relayed endpoint against each Azure config's path
segments instead of building a fake request to call get_complete_url.
The binding drops the unused headers, status and api_base arguments.
Catch the ValueError/RuntimeError the binding raises so a relayed body
that is not OCR-shaped falls back to the passthrough object instead of
failing logging, and cover the relay against the real binding.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(ocr): drop the unused LlmProviders import from health check helpers
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* ci: drop the ocr_testing job now that tests/ocr_tests is gone
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(ocr): restore the live OCR matrix and the ocr_testing job
The public litellm.ocr / aocr / Router interface is unchanged by the Rust
migration, so the live provider matrix still applies. Drops the stale VCR skip
list for the deleted test_rust_bridge.py.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(ocr): import Final in the health check helper tests
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
The repo rule is that a test must only fail when litellm code changes, never when a vendor updates a price, renames a field, or drops a model. These tests asserted shipped catalog entries directly, comparing lookup results to literals copied from model_prices_and_context_window.json or requiring named entries to exist or be absent, so every cost map sync could break them without any litellm code changing
Tests that exercise real litellm behavior with an injected local model_cost, invariants like backup parity, and assertions on non-lookup code paths are untouched
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Drop the two new docstrings, annotate the new locals Final, and replace the
mutable call recorder with a rebuild stub that fails the test if it is ever
reached.
test_no_linear_scans_in_router: #39468 added config_deployments() and
heuristic_v2_router_limit_violation(), which both scan the whole model_list
from admin-only paths (model add/upsert), so add them to the allowlist. The
allowlist becomes a mapping so each exemption carries its reason as data.
test_missing_model_parameter_curl: a request with no model is rejected by the
proxy when nothing can serve it and by the router when a wildcard or default
deployment exists, and by the upstream provider when a wildcard forwards it,
so the message text is not a stable contract. Assert the contract that holds
in every case: HTTP 400 with a non-empty error message.
test_model_group_info_e2e: /model_group/info resolves wildcards, so it can
never return "anthropic/*" verbatim. cc3f9cd65b rewrote the assertion to
expect the raw pattern after claude-3-5-haiku-20241022 left the price map,
which made it unsatisfiable. Assert the expansion instead.
test_should_derive_ocr_mapping_status_from_live_tests: the audit needs a
native bridge built with the trace-parity feature, which CI never builds, so
skip with the harness's own diagnostic instead of erroring. Extract that
check out of ensure_trace_bridge as trace_bridge_error so a pytest run
reports the state without kicking off a maturin rebuild.
Adds chat_completions and transcription as SDK function columns, backed
by the existing rust_bridge test files. Adds a fourth strategy folder,
existing_e2e_test_sdk, that points at already-existing live-API SDK
tests (tests/ocr_tests/ as a whole folder, plus chat completion and
Whisper transcription tests) instead of writing new parity tests.
Extends selector_matches_node with trailing-slash folder selectors so
a whole test folder can back one matrix cell.
* test: add OCR python-to-rust test parity ledger
* feat: add ledger loader for OCR test parity data
* feat: add drift audit for OCR test parity ledger (WIP, untested)
* fix: correct drift in OCR test-parity ledger
Two entries referenced a typo'd Python test name, four duplicated entries already tracked under TestProxySecurityGuard, five real Python tests in test_rust_bridge.py were untracked, and three real Rust custom_logger tests were missing from rust_only_tests. Found by running validate_ledger.py's audit against the live repo.
* test: add regression coverage for the OCR ledger and audit script
Covers schema validation, AST/regex test enumeration, drift detection on both the Python and Rust sides, LedgerDriftError content, and a live-repo clean-audit guard against future drift.
* test: simplify ledger test to one drift-guard assertion
Replace the ledger-internals unit tests with a single test that runs the real audit against the live repo and asserts every OCR test is accounted for (mapped, unmapped-with-reason, or rust_only), printing the exact diff on failure.
* chore: move OCR test-parity ledger to core/ocr
validate_sub_methods/ mixes strategy-catalog metadata with the ledger. Ledger data belongs under a per-function core/<function>/ path instead.
* fix: point LEDGER_PATH at the new core/ocr location
