* fix(proxy): forward the client's own metadata on Anthropic and custom pass-through routes
* fix(proxy): forward only a valid metadata.user_id to Anthropic Messages on pass-through routes
The whole-object forward sent keys Anthropic refuses and reached other
providers, so a request that used to pass could start failing with a 400.
Read the client's user_id after hooks, drop it unless it is a string with no
email or phone number, and send only that key when the upstream URL path ends
in /v1/messages. Share the validator with the translated route, whose email
check now matches any embedded address as Anthropic's does.
* fix(proxy): forward metadata.user_id only to Anthropic upstreams no guardrail scans
Limit the pass-through user_id to Messages create on api.anthropic.com or the
configured Anthropic base, mirror the checks Anthropic answers 400 for, and keep
the field off any request a pre-call guardrail inspects. The translated route
goes back to its own user_id filter.
* fix(proxy): judge user_id forwarding on the whole request and cap it for configured Anthropic bases
* refactor(proxy): pass the user_id check a plain mapping instead of an optional body
* fix(proxy): drop an unprintable metadata.user_id and warn when a refused id is dropped
Anthropic's whitespace is not Python's, so a control character before the at sign hid an email from the filter and turned a 200 into a 400. Fireworks and Moonshot also fail on newline ids. Requiring a printable id covers both, and a refused id now logs a warning that never carries the value.
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(vertex_ai): bill Claude web search per request on deployments priced in model_info
A vertex_ai deployment priced in its model_info is registered under its hashed id, so the
web search cost check on the model key never saw "claude" and billed Claude searches with the
Gemini grounding calculator: one search per call, or Gemini's $0.035 default with no search
price. The check now also reads the model the response names.
* fix(vertex_ai): bill token-priced Claude deployments at the served model's search price
A Vertex Claude deployment priced in model_info without a search price now bills each web search at the served model's cost map price, read at cost time and never written into the deployment entry. An explicit search price, zero included, still wins.
* fix(vertex_ai): price web search on the Vertex Claude Sonnet 4.5 and Haiku 4.5 rows
Google bills Claude web search at $10 per 1,000 searches on these models, so token-priced deployments of them no longer bill searches at 0
* fix(cost): bill served-model search price on token-only Anthropic and Azure AI Claude deployments
Token-only Claude deployments on anthropic and azure_ai billed web search at 0 because the deployment entry has no search price. They now fall back to the served model's cost map search price, as vertex_ai already does. Vertex routing reads the served model first, so a Gemini deployment whose id contains claude keeps per-prompt grounding
* fix(proxy): price streamed usage frames under the deployment's model so usage.cost matches the spend row
* test(proxy): give the Vertex streamRawPredict logging doubles the request's model, as a real pass-through Logging has
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* test(proxy): pin admin-owned team keys to their team's MCP servers and toolsets
A virtual key on a real team owned by a proxy_admin or proxy_admin_viewer, with no MCP list of its
own, currently reaches every MCP server in the registry and any toolset, ignoring its team. These
tests expect it to resolve only what its team grants, like the same key owned by an internal user,
while the admin dashboard session keeps the whole registry.
* fix(proxy)!: bind admin-owned team keys to their team's MCP servers
A virtual key on a real team now resolves MCP servers and toolsets through its team even when its
owner is a proxy_admin or proxy_admin_viewer, the way team model access already binds it. The
unscoped admin view stays for the master key, admin keys with no team, the admin dashboard
session, and admitted user subjects. The toolset grant checks on /toolset/{name}/mcp and on the
Responses litellm_proxy MCP tool use the same rule.
BREAKING CHANGE: a team key owned by a proxy admin or admin viewer no longer reaches MCP servers or
toolsets its team does not grant. Grant them to the team or use a key with no team.
* fix(proxy): answer 403 for a toolset the key cannot use on LLM routes
A litellm_proxy MCP tool on /v1/responses, /v1/chat/completions or /v1/messages that named a toolset
the key's team was never granted was skipped, which left the server filter empty and handed the model
every tool the key could reach. The denied name now stays in the filter, so the request gets the same
403 an unknown server name gets.
Admin credentials on a real team stay bound to it whatever carries them, a lite login session included,
the way the team already bounds their model access. The tests pin that and drop an unused patch.
* fix(proxy): keep a denied toolset out of the scoped key beside a granted one
* test(responses): cover toolset name fallbacks and align MCP admin docstrings
* test(mcp): cover no-team admins' toolset access on LLM routes and drop a dead admin-check clause
* fix(responses): keep a whole-gateway MCP entry serving the key's tools beside a denied toolset
* test(responses): check the tools a whole-gateway request lists beside a denied toolset
* test(responses): keep the toolset lookup real in the whole-gateway listing test
* test(responses): run the toolset gateway tests on a real MCP server manager
* fix(responses): keep denied toolset names out of the MCP server filter
A denied toolset named like a server the key holds was matched to that server
by the loose server-name lookup. Denied names now never reach the filter, and a
request naming only denied toolsets answers 403 directly, each name once
* fix(responses): let a held MCP access group outrank a same-named ungranted toolset and give denied toolsets the generic 403
* test(responses): pin access group and granted toolset precedence for LLM-route MCP names
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(a2a): bill completion-bridge agents at their own prices on /a2a send and stream
message/send on an agent served through the completion bridge (custom_llm_provider
agents such as LangGraph or databricks_agent) logged a $0, 0-token row, and
message/stream logged no row at all, so cost_per_query and per-token prices never
reached spend.
The bridge send path now records counted usage, the agent's pricing keys, and the
agent id on its logging object, the same way URL agents do. The bridge stream is
wrapped in the A2A streaming logger, which now reads the bridge's artifact event
for the reply text and gets the agent name from the proxy. Streamed calls also
count prompt tokens again, since the message text reader now reads pydantic
messages as dicts.
* fix(a2a): count chunked and Task replies once and skip agent prices on error replies
Bridge streams now follow the A2A append flag per artifact, so chunked
pydantic-ai, watsonx, and AgentCore replies count the whole reply once.
Send replies that are a Task count their artifact text. A JSON-RPC error
reply is logged at cost 0 instead of the agent's price.
* fix(a2a): keep one reply text per stream and bill every artifact and status reply
* fix(a2a): ignore a null error field on streams and run the stream per-token billing test
A stream event whose error field is null no longer zeroes the call's cost. The integration test for per-token stream billing now runs instead of skipping, and the URL agent SDK case skips when the optional a2a-sdk is not installed.
* fix(a2a): bill streams the client drops and skip the bridge's working placeholder
A stream the client closes or cancels mid-reply now logs its spend row at the agent's prices on both the bridge and URL paths, and a stream dropped before the agent sends anything still logs nothing. The bridge's "Processing request..." status no longer counts as reply text, so an empty bridge reply bills no completion tokens, as send does. Adds proxy-level tests that a bridge stream bills under the agent's card name.
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Strategies, pre-call checks, cooldown and fallback helpers, and the auto, complexity and quality
sub-routers now take a RoutingContext instead of the Router, and read the router only through
registry(), settings(), health() and the context's forwarding methods. Router-free helpers move
from router_utils to litellm_core_utils, outside callers of router state switch to entry points,
and the surface allowlist shrinks to the endpoint methods plus the entry points
* refactor(guardrails): record which policy pipeline steps and deployment-level hooks ran
* fix(guardrails): name only the request pipeline steps that ran in x-litellm-applied-guardrails
* fix(guardrails): drop the response pipeline steps that never ran from x-litellm-applied-guardrails
* test(guardrails): integration cells for request pipeline steps in x-litellm-applied-guardrails
* fix(guardrails): keep the agent cells on the shared proxy and shorten a long suppression line
* fix(guardrails): keep request steps when a response pipeline blocks before a later request pipeline
* refactor(proxy): inject gateway activity through a typed repository
* ci: keep dependency tests in their existing shard
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(guardrails): record which policy pipeline steps and deployment-level hooks ran
* fix(guardrails): name only the request pipeline steps that ran in x-litellm-applied-guardrails (#49263)
* fix(guardrails): name only the request pipeline steps that ran in x-litellm-applied-guardrails
* test(guardrails): integration cells for request pipeline steps in x-litellm-applied-guardrails
* fix(guardrails): keep the agent cells on the shared proxy and shorten a long suppression line
* refactor(proxy): inject database repositories through per-repository dependencies
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(proxy): model Prisma client cleanup
* test(rust): include NativeDatabase in bridge API surface
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(proxy): give database shutdown one lifespan owner
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(proxy): log database cleanup errors and bound shutdown waits
Database.aclose now logs a failure in either cleanup step instead of raising
out of the lifespan exit, which matches how main handled stop errors and keeps
setup_prisma_client from raising out of cleanup after a failed connect
Shutdown waits for an in-flight reconnect for a fixed five seconds instead of
the reconnect timeout plus grace, and joins engine wait threads with a fixed
limit instead of indefinitely. The threads are daemon threads, so a thread
still alive after the limit cannot keep the process from exiting
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* fix(batches): cost an unmanaged gs:// batch through the deployment that created it
The create now stores the router deployment id as model_id inside the
managed object row's file_object JSON and CheckBatchCost reads it before
falling back to the first deployment matching the base model in the
gs:// path. Rows without the field keep the old lookup
* fix(batches): tolerate a non-string or malformed file_object when reading the stored deployment
* fix(batches): read the stored creating deployment from the writer, not a lagging read replica
* fix(batches): resolve the writer database once so the type gate count does not grow
* fix(batches): decode a dict-shaped file_object when reading the stored deployment, accumulate wire samples, annotate test locals
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(sso): create the team_id_jwt_field team on UI SSO login when team_id_upsert is set
* fix(sso): keep the upserted team_id_jwt_field team out of result.team_ids so restricted_sso_group and the access-token group fallback are unchanged
* fix(sso): only upsert the team_id_jwt_field team on UI SSO login when enable_jwt_auth is on
The API JWT path only reaches team_id_upsert inside the enable_jwt_auth branch, so a deployment
with a leftover litellm_jwtauth block and JWT auth off never created teams. Gate the UI SSO upsert
the same way
* fix(sso): read enable_jwt_auth through the proxy_server module so the basedpyright gate stays at its ceiling
