The Vertex passthrough route resolved a router deployment only to rewrite the
upstream URL and dropped its model_info, so the standard logging payload and
the Prometheus litellm_deployment_success_responses_total counter carried
model_id="". Carry the deployment's model_info through request.state into the
passthrough logging metadata, where it overrides any client-supplied model_info.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
_handle_clientside_credential registered the per-request Deployment it built for
a client-supplied api_key/api_base via upsert_deployment, which added it to
self.model_list under the shared model_name. That made a request-scoped
credential a permanent, load-balanced deployment that any later caller of the
same model group could be routed onto, reaching the provider with someone
else's forwarded credential.
The per-request Deployment still gets its own stable id for cooldown and
logging identity; it is just never registered with the router.
Resolves LIT-7811
The failure logger skips fallback hops (has_logged_async_failure is already set), so
model_call_details.end_time still belongs to the previous hop and predates this hop's
api_call_start_time. The fallback cooldown guard measured a negative elapsed time and
cooled down deployments for caller-set timeouts.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
client_side_timeout records that the caller configured a timeout, not that
the timeout fired. A 408 the provider returns before that deadline is a
deployment failure and must still count toward cooldown.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The Usage page drains the daily activity endpoint page by page. A page that
threw was only logged to the console: the loading banner disappeared, the
partial totals stayed on screen looking final, and Export Data stayed
clickable, so the CSV handed to finance was silently short.
The hook now reports `failed`, PaginationStatusAlerts renders it as an error
banner naming how many pages actually loaded, and the export is blocked with
the reason on hover while the data on screen does not cover the range.
A 408 produced by a timeout the caller set (a timeout body field or an
x-litellm-timeout header, which the proxy marks as client_side_timeout)
says nothing about the deployment's health, yet the router's primary
failure callback counted it toward allowed_fails and cooled the
deployment down. The fallback path already skipped it.
The marker never reached that callback because get_litellm_params drops
kwargs outside OPTIONAL_KWARGS_KEYS, so it is listed there now, and
deployment_callback_on_failure returns before the failure counter when
is_caller_timeout_408 holds. A 408 from a timeout the deployment or the
provider set still counts and still cools the deployment down.
budget_duration and allowed_models ride on /team/member_add. tpm_limit and rpm_limit are sent through /team/member_update, the only endpoint that accepts them. Removing any of the four from config sends an explicit clear (null, or an empty list for allowed_models) since member_update is a merge-patch. The resource ID is set before the post-add limits call so a failure there taints the resource instead of orphaning the memberships
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Auth gate rejections are raised before add_litellm_data_to_request stamps the caller User-Agent and SpendLogsMetadata dropped the field, so failure spend logs and prometheus labels could not identify an abusive client. Stamp requester_ip_address and user_agent on the failure hook payload and carry user_agent through spend log metadata. Request scopes without a headers entry are tolerated.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
A stale RoutingPrismaWrapper.writer_unavailable flag made WriterPinnedClient hand back the routed wrapper, where query_raw is classified as a read, so the access-group UPDATE statements behind /key/regenerate, /key/generate with access_group_ids and model rename/delete went to the read replica and failed with SQLSTATE 25006. Route those raw statements through the underlying writer regardless of the flag; a raw SQL write has no replica fallback. WriterPinnedClient keeps yielding to the replica for degraded reads.
The model sync's backing-row count stays on the writer too: it runs right after the row delete/update on the writer and a lagging replica could still report the removed row, which would leave the group naming a model nobody serves.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
