* feat(gateway): forward HTTP requests to the Python proxy
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(gateway): preserve required forwarding headers and separate upgrades
* test(gateway): clarify hash compatibility and relocate forwarding docs
* test(gateway): define URL fragment forwarding behavior
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Adds the litellm-router-core crate, its step-driven PyO3 bridge
(RouterCoreHandle / RouterCall) and RustCore (litellm/router_rust/),
served behind LITELLM_RUST through the ROUTER route rule. The host
call-out is the only boundary: every attempt goes through Host::invoke,
there is no native executor path, and RustCore never builds, wraps or
forwards to PythonCore, so router_python_core.py stays deletable.
Attempt preparation lives in litellm/router_attempt.py so it outlives
router_python_core.py.
RustCore matches PythonCore's behavior, including its known bugs (fixed
in follow-up PRs). Routing decisions (strategy picks, routing groups,
PTU, pre-routing, pre-call checks, fallback walk, health routing, access
groups, weights, cooldowns, allowed fails, usage) live in the crate;
work that needs live Python objects (request shaping, endpoints,
streaming, model info facts, callbacks, alerting, prompt management)
lives in the host-side litellm/router_rust/ package (steps/, invoke/,
surface/) driven by Rust steps.
The crate owns the registry. It runs the deployment build loop (secret
references, ids, organization fan-out, install order) and asks the host
for what needs Python, one deployment at a time so errors surface where
Python raised them: secrets, validate, provider and added steps. It
holds the model list, the index maps and the pattern routers with the
Python core's history rules (host_index.rs), and answers the registry
lookups (lookup.rs). The Python views of that state in
litellm/router_rust/surface/ (model_list, index(), names(), patterns())
read the handle, and every write to them becomes a changeset. Model
info, deployment model info, configured limits and model group info are
merged in the crate (model_info.rs, group_info.rs) from catalog rows and
facts the host answers on a model_info step asked at query time, so
price reloads and register_model stay visible. The usage cache key
format is defined once, in the crate.
The crate records each attempt's outcome itself rather than through
litellm callbacks. A streamed attempt is deferred: it counts its
request when the stream opens, and the host's stream wrappers report
the success with the final usage, or a failure after the first chunk,
when the stream ends, as the Python core's callbacks do.
tests/unit/test_router/core characterizes PythonCore's public behavior
and runs every test on both cores, with nothing pending; only the names
of the Python core's own outcome callbacks are asserted on Python alone.
litellm.Router is now final: subclassing it raises TypeError at the class statement, and the
machinery that kept subclass overrides working is gone. The Rust core never honoured subclass
overrides, so they would have silently stopped running there. Custom routing goes through
register_plugin(), settings through apply() and outcomes through observe(). This is a breaking
change for anyone subclassing Router.
Fuzz parity fixes on top of the rebase: the Rust core keeps num_retries,
max_fallbacks, allowed_fails, retry_after, timeout, the tag settings,
routing_strategy_args and the fallback lists as given (the crate reads
them as numbers, update_settings casts as Python does); a deprecated
`model_list = [...]` write reaches the Rust core; a failed set_model_list
is all or nothing on both cores (the deployments served before the call
are put back and the error raised); config_deployments() is a generator and `del` of a plain
attribute works on the Rust core. Least-busy pick reservations from
main now work on the Rust core: its selectors share one reservation
store, routed calls reserve and free their unstarted picks, and
select() and batch cancel do not reserve. On the Python core a dropped
upsert takes its name back off deployment_names, and
get_deployment_order reads every numeric spelling as one int so mixed
orders no longer raise TypeError.
Rulings on the remaining differences, the Python core changed where it
was the odd one out: sync picks and picks by deployment id apply the
team filter as async picks do, so another team's deployment is refused;
the fallback lists and default_litellm_params in RouterSettings are
copies, so an in-place edit is ignored and settings change only through
apply(); one health record with an unreadable timestamp drops out alone;
an empty fallback entry is skipped; a bool cooldown_time is refused at
construction, a bool or string cooldown_time on a deployment or a call
falls back to the router default, and a failure status that no HTTP
status can be is recorded as 500; an infinite or NaN cost and a whole
number wider than 64 bits (ptu_shares) are refused on both cores with
the same error (litellm/router_utils/deployment_numbers.py); and a
/v1/messages stream whose provider iterator raises cools the deployment
and retries like a non-stream call, with the Rust core waiting for that
cooldown before its retry picks.
Differences the proxy parity fuzz and its regression tests found on the
Rust core: a file upload or a batch on an uploaded file falls back only
within its own model group, as on the Python core, since the file lives
with one provider (the host sends same_group_fallbacks_only and the walk
skips targets in other groups); Decisions calls drop deployments whose
provider has no Decisions config, and deployments that refuse the audio
or videos a call carries, unless that would leave none, as the Python
core does, while the retry count still reads the whole group; a file
upload that fails on one order level is created on the next level's
deployments rather than the first level again; and a response that
cannot be converted (a text_completion body without a message) reports
only its own traceback, so a rate limit handled by the attempt before it
no longer turns the error into a 429, and both cores return 500.
* fix(auth): align cloud credential loading and expiry with the Python SDKs
* feat(auth): resolve consumer-scoped default identities for AWS and Azure
AwsAuthService::default_chain_credentials and AzureAuthService::identity_token give a consumer the SDK's own default identity without LiteLLM's Bedrock or Azure OpenAI settings. Moved here from #49510 so every shared auth crate change lives in one PR
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(auth): include identity hints in secret snapshots
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* feat(model-catalog): add regional_processing_uplift_multiplier_fedramp
* refactor(model-catalog): make the FedRAMP uplift its own fedramp_uplift_multiplier key
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(realtime): support latest OpenAI audio models
* fix(transcription): type SDK stream duration metadata explicitly
* fix(realtime): preserve transport settings and translation duration
* refactor(realtime): reuse SDK client initialization
* fix(audio): bill finalized streams and authorize transcription models
* refactor(audio): keep transcription pricing selection together
* fix(realtime): reconcile model metadata and test contracts
* fix(realtime): preserve provider duration and model capabilities
* fix(realtime): keep transcription validation compatible with cost map guard
* fix(realtime): address transcription and billing review findings
* fix(realtime): preserve input usage and provider routing
* fix(realtime): bill retained output and route Azure aliases
* fix(realtime): reject malformed provider durations
* fix(realtime): preserve batch cache rates and Azure v1 routes
* fix(transcription): reject nonfinite provider durations
* refactor(realtime): align annotations with current lint rules
* fix(realtime): preserve translation deployment pricing and input usage
* test(transcription): verify synchronous stream finalization
* fix(realtime): retain missing translation input usage at close
* fix(transcription): reject streams that bypass output guardrails
* fix(realtime): reconcile audio protocols billing and current CI
* fix(realtime): trust confirmed formats and authorize default models
* fix(realtime): preserve absent models and reject unsupported guardrails
* fix(realtime): reject translation policy pipelines
* chore(ci): rerun transient infrastructure timing failure
* fix(realtime): reconcile audio protocols with current upstream
* fix(audio): validate provider responses at SDK boundaries
* fix(realtime): preserve minimum OpenAI SDK compatibility
* fix(realtime): preserve deferred Live session authorization
* fix(realtime): preserve upstream authentication and transcription behavior
* fix(audio): scope provider duration precedence to configured models
* fix(typing): avoid Final declarations inside transcription cost loop
* refactor(audio): reuse transcription mapping and duration resolution
* fix(realtime): enforce model budgets and preserve translation routing
* fix(audio): log streamed transcripts whose done event reports detected languages
* fix(audio): pass stream, languages and keywords through unchecked for non-OpenAI transcription providers
Providers without a streaming transcription handler got UnsupportedParamsError
for stream=True, languages and keywords, where the merge base forwarded them as
provider-specific params. Only openai and azure validate them now.
* fix(audio): bill streamed gpt-transcribe from the provider's reported duration
The stream path computed cost after logging had replaced the transcription's
duration usage, so it fell back to the measured file length while the
unstreamed call billed the provider's seconds.
* fix(transcription): log streamed transcription usage from the provider response
The stream logging path built the spend payload from the response after
its duration usage was transformed away, so whisper-1 with stream=true and
streamed gpt-transcribe logged an empty usage object. Bill and log from the
provider's TranscriptionResponse in both the sync and async paths.
* fix(realtime): keep main's routing, protocol and transport behavior around the new translation routes
- Send OpenAI realtime and translation client_secrets and calls through litellm's httpx client again, so ssl_verify and custom CA settings apply and upstream errors keep main's shape
- Use the input transcription model as the session model only for transcription sessions
- Drop the new /openai/v1 translation aliases and record each lazy feature's own routes, so the /openai pass-through and the MCP routes resolve as on main
- Let a configured Azure realtime_protocol win for every model, with GA as the default only when none is set
- Return x-litellm headers on streamed transcriptions and mark the request done
- Refuse translation sessions before connecting upstream when realtime guardrails apply or the provider is not OpenAI or Azure
- Accept json and text for gpt-transcribe, and drop other formats under drop_params instead of failing
- Bill translation sessions on output audio seconds only, at the single published rate
- Document the 201 SDP answer on the translation calls routes and the transcription 200 response
* fix(realtime): restore main behavior for guardrails, Azure transcription, and realtime clients
Streamed transcription runs unstreamed when a deployment post_call guardrail applies, so the
guardrail still sees the transcript. Azure transcription maps v1, latest and preview to the
default dated version on the deployments route for every model. The SDK client secret keeps
session.model ahead of model, realtime WebSockets connect to api_base again, realtime metadata
keeps only guardrails, and managed agents need an explicit transcription model. Reverts the two
unrelated streamed Responses error codes and the duplicate cached_tokens_details field, and the
gpt-transcribe format check only reads the request provider's cost map entry.
* fix(realtime): keep main's transcription, client secret and route behavior where the PR changed it
Transcription reaches every provider as on main again, sends stream only when asked, forwards language and
languages together, keeps form key order, bills provider seconds only when above zero, honors base_model for the
new flags, keeps main's Azure api_version on the proxy path, and streams carry chat's headers, an error event on a
mid-stream failure, and a spend row when the stream ends early. Translation cost reads both per-second rates from
one source and spend rows carry the billed seconds.
Realtime transcription WebSockets keep model in the upstream URL outside OpenAI's hosts, client secrets forward
explicit nulls, accept conversation and delay, and answer 400 on a malformed body, the /openai translation paths
are back to main, translation refuses only for guardrails realtime sessions run, nested aliases resolve and later
session.update models are authorized, translation client secrets run per-model auth, beta clients bridged to GA
get beta-shaped session events, and transcription sessions send the deployment model in both shapes.
* fix(realtime): address review findings on transcription billing, streaming and realtime auth
Per-second pricing replaces token billing only when the priced entry sets no token rates, translation rates fall back per key to the cost map, transcription cache keys include languages and keywords, a stream request a post_call guardrail forces unstreamed comes back as redacted SSE, Azure beta clients are bridged only for GA-only deployments, nested transcription models on the realtime WebSocket are checked against the key, and translation spend carries key and team metadata
* fix(azure): pick the GA realtime protocol for deployments whose model_info.base_model is GA-only
* fix(realtime): preserve prior merge resolutions after rebase
* fix(realtime): enforce translation policies and normalize Azure transcription URLs
* fix(cookbook): stop translation once output idles after the upload ends
* fix(realtime): bill translation on input audio and drop pre-read stream cost headers
Translation spend now uses the input audio seconds the session sent, priced
through input_cost_per_second, instead of the translated output duration.
A streamed transcription no longer returns cost headers before the stream is
read, since the cost is only known after the last event. Also removes an
unused realtime guardrail helper.
* fix(realtime): drop top-level session nulls, bill dropped transcription streams, and bridge beta response fields
Client secret, translation and transcription session bodies drop top-level nulls again, as main did, while nested nulls such as turn_detection still pass through. Transcription client secrets send the deployment model as session.model. A streamed transcription the client drops still runs its success callbacks, and the stream ends with data: [DONE] like OpenAI's. The beta to GA bridge maps max_response_output_tokens, speed and noise reduction, remaps response.create, and drops temperature under drop_params. The Entra token for realtime HTTP routes is fetched off the event loop
* test(e2e): cover realtime translation, live transcription, client secrets, and streamed transcription
* fix(realtime): close review gaps in translation billing, transcription streams, and Azure GA
Bill translation sessions from the deployment's own per-second rates, add per-second prices for
Azure gpt-4o transcribe models, keep stream-scoped guardrails seeing the caller's stream flag, map
mid-stream transcription errors to LiteLLM exceptions with a [DONE] frame, map GA responses back to
the beta shape for bridged beta clients, honor the minting key's guardrail opt-out on translation
calls, restore the Azure transcription session URL, and flag GA-only Azure realtime models
* fix(realtime): bill a translation rate the deployment omits from the session model
* test(e2e): check translated speech is not English instead of exact French wording
* fix(realtime): keep main's behavior for non-bridged backends, stream error frames, gateway URLs and Azure transcribe prices
Remap beta events to GA only for bridged beta clients and translation sessions, bridge beta clients on GA-pinned Azure deployments, log error-only upstream sessions as failures, flag dated Azure GA realtime models, drop the unpublished Azure per-second transcribe rates, log a mid-stream transcription failure once, keep main's SSE error frame outside transcription, drop unsupported transcription params under drop_params, keep the caller's Azure auth headers, keep a gateway /v1 path, let key opt-outs release string-callback guardrails on translation, and keep the cookbook waiting while a long upload is still sending
* fix(realtime): keep the proxy's realtime session gate off translation sessions
A key that opts out of a legacy callback guardrail now gets a working translation session, since the transcription-guardrail gate no longer sends a realtime session.update the translation upstream rejects
* fix(realtime): keep websocket model limits, bill measured transcription length, route transcription sessions on the allowed model
A WebSocket opened without a query model now authorizes as an empty model, transcription cost prefers the measured upload length over the provider's reported seconds, transcription sessions overwrite nested models with the authorized top-level model again, and a non-boolean transcription stream value answers 400 or is dropped under drop_params.
* fix(ci): copy the rust bridge extension into the openapi docs gate's base worktree
* fix(model-catalog): accept the realtime translation endpoints in the cost map
* fix(cookbook): close the translation session and read until session.closed
* fix(realtime): treat an empty transcription stream as unset and pass the cost-map guard
An empty `stream` form value runs the transcription unstreamed again, as
OpenAI does, instead of answering 400. The four translate cost-map entries
drop the translation routes from supported_endpoints until the main
branch's catalog parser accepts them, and the nested transcription model
lookup uses the router registry entry point.
* test(proxy): drop the mid-stream error frame pin that main's error shape change replaced
* fix(transcription): bill uploaded seconds when a stream fails mid-transcript
* fix(proxy): bill uploaded seconds when a streamed transcription breaks mid-stream
* fix(audio): keep transcription's positional arguments and redact streamed error details
- Move the new languages, keywords and stream options to the end of transcription's parameters, so a prompt or format passed by position keeps its meaning
- Redact internal hostnames, paths and tracebacks from a streamed transcription's mid-stream error event, as chat streams already do
- Move the CA-bundle check for the realtime HTTP routes, which serves real TLS on loopback, from the unit suite to the SDK integration suite
* fix(audio): stream the guarded transcript to SDK callers that asked for a stream
* fix(logging): keep a streamed transcript out of callbacks that turned off message logging
A streamed transcription's assembled response reached success callbacks with its text intact under turn_off_message_logging or a callback's message_logging=False, because the shared streaming redactor had no transcription branch. It now redacts the text, as the unstreamed call already does
* fix(proxy): stream a transcription the deployment streamed when the client sent no stream field
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(model-catalog): declare the thinking_on_by_default key
The cost map guard runs main's catalog on a PR's cost map, so the key has to be declared on main before a PR can set it on the Claude 5 rows
* chore(model-catalog): keep the attribute list sorted and document thinking_on_by_default
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(count_tokens): count grok-4-1, grok-4.3 and DeepSeek V4.1 tool requests like their chat templates
Render grok-4-1, grok-4.3 and DeepSeek V4.1 tool schemas the way each chat
template does, and add a per-deployment token_count_tool_framing next to
token_count_framing for the hidden tool preamble a host bills. The Azure AI
Foundry grok-4-1, xAI grok-4.3 and Fireworks DeepSeek V4.1 cost map rows carry
the measured framing, overridable in model_info
* fix(count_tokens): count Azure grok-4-1 Responses requests with their own framing and use the tool framing in place of the base framing
* fix(count_tokens): keep operator framing on tool and Responses counts, frame bridged Responses like chat, follow base_model for tool rendering, count grok tool_choice
* fix(token_counter): count Azure Responses strict default, skip false strict and built-in-only tools, per-kind operator framing
* fix(token_counter): count an explicit null strict like false on Azure Responses
* fix(count_tokens): follow the chat completions bridge flag and Azure's strict default on azure/ deployments
Keep Azure grok-4-1 chat tool counts at or over the bill by raising its tool framing to 302
* fix(token_counter): keep strict false for non-grok templates, coerce the bridge flag, frame DeepSeek API counts
* fix(proxy): frame Responses counts for the api_base router defaults send inference to
* fix(proxy): keep the api_base helper's name so main's /v1/messages count caller resolves
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(model-catalog): declare the tool and Responses token count framing keys
* fix(model-catalog): read token count framings as strict integers
The cost map guard read the framing keys as lax counts, so a text "214"
or a true passed while litellm's reader drops both and falls back to the
plain framing. Read all four framing keys as plain u64, matching the reader
* fix(model-catalog): accept whole-float token count framings the way the Python reader does
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(anthropic): sync the beta header allowlist with what anthropic, vertex_ai, and azure_ai accept
* test(e2e): send the azure_ai beta check without the code execution tool
* test(anthropic): require a vertex_ai and azure_ai decision for every anthropic beta
* fix(anthropic): forward the betas Claude Code sends that anthropic, vertex_ai, and azure_ai accept
* test(e2e): send Vertex's header-only betas in the beta acceptance case
* fix(anthropic): keep forwarding Vertex computer-use betas and point beta tests at platforms that still lack them
A live probe found each Claude 4.5 and 4.6 model on Vertex needs one of the two
computer-use betas for the computer tool, so dropping both broke it. The e2e
acceptance test leaves those two out of its all-at-once Vertex call, and the
integration and unit tests that relied on a beta being dropped now use one
the platform still lacks.
* test(anthropic): point unsupported-beta tests at betas the platforms still drop
The Azure AI handler test used claude-code-20250219 as the beta Azure drops, and the
Rust parse test used it as an unknown value. This PR forwards it on Azure AI and maps it,
so both now use values that stay dropped or unknown.
* fix(anthropic): add computer-use-2025-11-24 for the computer_20251124 tool
The computer_20251124 tool fell through to the default computer-use-2024-10-22 beta, which the
provider rejects for that tool version. Map it to computer-use-2025-11-24, which the beta map now
forwards on anthropic and azure_ai
* fix(anthropic): send no computer use beta for an unknown computer tool version
An unknown computer tool version fell back to computer-use-2024-10-22, which the anthropic and azure_ai columns now forward. It now gets no beta, as every platform effectively sent before. Every computer tool version also skips defer_loading and allowed_callers, computer_20251124 included
* test(azure_ai): read the branch's beta map in the auto-injected beta wire test
* fix(anthropic): keep defer_loading and allowed_callers on the computer_20251124 chat tool
* test(anthropic): read the branch's beta map in the azure_ai safeguards beta wire test
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* docs(model-catalog): explain meter, band, channel and When
Add a Concepts section to the model-catalog AGENTS.md with worked key
breakdowns, doc comments on the undocumented litellm-cost price types,
and a cost AGENTS.md that points to the concepts.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(model-catalog): write concepts as nested lists
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(model-catalog): spell out band rules and channel fallbacks
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* fix(anthropic): map pause_turn and return refusal explanations on chat and responses
pause_turn now maps to finish_reason length (Python and Rust) and keeps the raw
stop_reason in provider_specific_fields. Responses reports a paused turn as
incomplete with max_output_tokens and shows the unrun server search as an
in_progress web_search_call instead of a client function_call.
A refusal's stop_details.explanation now fills message.refusal and the final
stream delta.refusal on chat, and becomes a refusal content part on Responses,
streamed or not.
* fix(anthropic): let a chat client resend a paused turn so Claude continues it
* fix(anthropic): replay a paused chat turn without its thinking blocks, and drop its unrun search before a new message
Chat cannot say where Claude wrote each thinking block in a paused web search turn, and
Anthropic answers 400 when one moves but accepts the turn without them. A new message after
a paused turn also 400s on a server_tool_use with no result, so that unrun search is dropped.
* fix(responses): send the streamed refusal part's type and text in content_part.added
* fix(anthropic): keep pause_turn and Claude refusal explanations out of the Messages adapter's text
A paused Claude turn now ends on stop_reason pause_turn instead of max_tokens, streamed or not. A Claude refusal keeps its explanation in stop_details with Claude's category and no longer repeats it as a text block, so a guardrail mask on a streamed refusal is delivered. A paused turn sent back with list content drops its thinking blocks like string content does
* fix(anthropic): replay a paused turn with one unrun search and no thinking on chat and Responses
* fix(responses): report a paused Claude turn as completed so Codex and the Agents SDK keep going
* fix(responses): mark a paused turn's message item completed
* fix(responses): mark every item of a paused turn completed
* fix(responses): keep a refusal's explanation when a refused turn is sent back
A refusal part in a Responses input message now goes to the model as text, so the assistant turn is no longer dropped from the history. The integration test now expects a paused Claude turn on Responses to report completed, and the paused-turn test helper is typed
* fix(responses): scan refusal parts in Responses guardrails
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* refactor(proxy): inject database repositories through per-repository dependencies
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(proxy): model Prisma client cleanup
* test(rust): include NativeDatabase in bridge API surface
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(vertex_ai): forward and auto-add the compact-2026-09-04 beta on Claude routes
* test(vertex_ai): prove the compact-2026-09-04 beta reaches the wire through both Claude handlers
* fix(vertex_ai): move a client-sent body compact-2026-09-04 beta to the request header
* refactor(vertex_ai): build the Claude beta set without mutation and inject the client in the messages wire test
* fix(vertex_ai): keep client body betas in the body and answer 400 on a malformed anthropic_beta
* fix(rust): allow the compact-2026-09-04 beta on vertex_ai in the Rust beta table
The beta header config now maps compact-2026-09-04 for vertex_ai, so the Rust
table that mirrors it must stop rejecting that beta there, and the Rust Vertex
messages route adds it for a compaction request like the Python route does.
* fix(vertex_ai): keep client body betas out of the Vertex body next to litellm betas, honor drop_params, and send the compaction beta on count_tokens
Litellm's own betas replace a client's body anthropic_beta again, as on main, so a body beta
Vertex rejects no longer turns a 200 into a 400. A malformed beta drops under drop_params and
names its field otherwise. inline-tools-2026-09-15 joins compact-2026-09-04 as header-only, since
Vertex rejects both in the body. Vertex count_tokens sends compact-2026-09-04 when the request
carries compaction.
* fix(logging): log and bill Anthropic messages compaction responses on the Rust route
The Rust route hands the served Anthropic body to the logger, which validates it
against AnthropicResponse. That model had no compaction content block and no
compaction stop reason, so a compaction reply raised and wrote no spend row.
* fix(vertex_ai,azure_ai): forward compact-2026-09-04 to Azure AI Foundry, merge Vertex body betas, and redact compaction summaries
Azure AI Foundry needs the compact-2026-09-04 header for compaction, a replayed compaction block, and count tokens, so the beta map, the native compaction providers, and the Rust beta table now allow it there. Vertex keeps a client's body betas next to the ones litellm adds, as the Anthropic route does, and turn_off_message_logging now redacts compaction summaries too
* fix(vertex_ai,azure_ai): auto-add thinking display and tool change betas, redact streamed compaction summaries
Folds the header changes from #44103 into this PR. Vertex AI and Azure AI Claude
routes now add thinking-display-updates-2026-08-18 when thinking.display is
"updates" and mid-conversation-tool-changes-2026-07-01 when a tool_addition
block is present, matching the Anthropic and Bedrock routes.
With turn_off_message_logging on, compaction summaries from streamed messages
and streamed chat completions are now redacted in spend logs and callback
payloads, like the non-streamed response already was.
* fix(vertex_ai,azure_ai,rust): add tool change betas on count tokens and Rust messages, bill Rust server tool replies
Vertex and Azure count tokens now add mid-conversation-tool-changes-2026-07-01 when the history carries a tool_addition or tool_removal block, through one shared helper with the compaction beta.
The Rust /v1/messages route adds thinking-display-updates-2026-08-18 and mid-conversation-tool-changes-2026-07-01 when a request uses them, on the providers that accept each beta.
Rust-route logging parses the served body with the Anthropic response parser, so web_search, pause_turn, mcp_tool_use and container_upload replies get a spend row.
Legacy enabled thinking keeps display updates on anthropic too, and the Vertex transform reads drop_params through drops_unsupported_params.
* style(rust): rewrap beta map test line with cargo fmt
* test(rust): expect the display beta on Databricks, which main's beta list now accepts
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>