LITELLM_LOG=ERROR still printed INFO lines from uvicorn (startup and access log) and from the litellm_proxy_extras migration logger, because neither read the variable. Forward the resolved level to uvicorn when LITELLM_LOG is set and no explicit log_config or JSON logging is in use, and let the extras logger take its level from LITELLM_LOG
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
A key inside a team with model_rpm_limit / model_tpm_limit in team metadata could not
override those limits for itself: the v3 limiter always added the team's per-model
descriptor next to the key's, so the tighter team limit won. The docs already say the
resolution order is key metadata > key model_max_budget > team metadata
get_key_own_model_rate_limit returns only what the key sets on itself, and the team
descriptor now carries only the metrics the key does not override, so an rpm-only
override still leaves the team tpm pool enforced
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
get_llm_provider() and Router._add_deployment() only knew the built-in
provider_list and JSON providers, so a provider registered through
litellm.custom_provider_map was rejected until custom_llm_setup() had
run inside the first completion() call. Both now check the map directly.
Resolves LIT-1742
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The Vertex passthrough route resolved a router deployment only to rewrite the
upstream URL and dropped its model_info, so the standard logging payload and
the Prometheus litellm_deployment_success_responses_total counter carried
model_id="". Carry the deployment's model_info through request.state into the
passthrough logging metadata, where it overrides any client-supplied model_info.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
_handle_clientside_credential registered the per-request Deployment it built for
a client-supplied api_key/api_base via upsert_deployment, which added it to
self.model_list under the shared model_name. That made a request-scoped
credential a permanent, load-balanced deployment that any later caller of the
same model group could be routed onto, reaching the provider with someone
else's forwarded credential.
The per-request Deployment still gets its own stable id for cooldown and
logging identity; it is just never registered with the router.
Resolves LIT-7811
The failure logger skips fallback hops (has_logged_async_failure is already set), so
model_call_details.end_time still belongs to the previous hop and predates this hop's
api_call_start_time. The fallback cooldown guard measured a negative elapsed time and
cooled down deployments for caller-set timeouts.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
client_side_timeout records that the caller configured a timeout, not that
the timeout fired. A 408 the provider returns before that deadline is a
deployment failure and must still count toward cooldown.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The Usage page drains the daily activity endpoint page by page. A page that
threw was only logged to the console: the loading banner disappeared, the
partial totals stayed on screen looking final, and Export Data stayed
clickable, so the CSV handed to finance was silently short.
The hook now reports `failed`, PaginationStatusAlerts renders it as an error
banner naming how many pages actually loaded, and the export is blocked with
the reason on hover while the data on screen does not cover the range.
A 408 produced by a timeout the caller set (a timeout body field or an
x-litellm-timeout header, which the proxy marks as client_side_timeout)
says nothing about the deployment's health, yet the router's primary
failure callback counted it toward allowed_fails and cooled the
deployment down. The fallback path already skipped it.
The marker never reached that callback because get_litellm_params drops
kwargs outside OPTIONAL_KWARGS_KEYS, so it is listed there now, and
deployment_callback_on_failure returns before the failure counter when
is_caller_timeout_408 holds. A 408 from a timeout the deployment or the
provider set still counts and still cools the deployment down.
budget_duration and allowed_models ride on /team/member_add. tpm_limit and rpm_limit are sent through /team/member_update, the only endpoint that accepts them. Removing any of the four from config sends an explicit clear (null, or an empty list for allowed_models) since member_update is a merge-patch. The resource ID is set before the post-add limits call so a failure there taints the resource instead of orphaning the memberships
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Auth gate rejections are raised before add_litellm_data_to_request stamps the caller User-Agent and SpendLogsMetadata dropped the field, so failure spend logs and prometheus labels could not identify an abusive client. Stamp requester_ip_address and user_agent on the failure hook payload and carry user_agent through spend log metadata. Request scopes without a headers entry are tolerated.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The gemini/ realtime test requires tpm and rpm, and Google publishes the
Live model limits only behind the AI Studio login, so the direct-API keys
cannot be sourced yet. The Vertex keys stay.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
