Move the WARNING that carries the internal denial detail out of the message formatter and into the auth exception handler. The denial exceptions now carry internal_message so access-group probes and fallback paths that catch and recover from the denial no longer log a false denial for an allowed request
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Add litellm_settings.model_access_denied_message, a template ({model} placeholder) returned to clients instead of the detailed "can only access models=[...]" text on key/team/user/org/project and team-member model access denials. The full denial reason is still written to the proxy logs at WARNING. Unset keeps the existing detailed message, status codes and error types are unchanged.
Expose the new setting and the existing expose_router_debug_in_errors flag in the Admin UI general settings (String editor, Boolean toggle with an explicit True default) and allow both as safe DB overrides so they persist and propagate across workers.
Resolves LIT-5283
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
get_llm_provider() and Router._add_deployment() only knew the built-in
provider_list and JSON providers, so a provider registered through
litellm.custom_provider_map was rejected until custom_llm_setup() had
run inside the first completion() call. Both now check the map directly.
Resolves LIT-1742
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The Vertex passthrough route resolved a router deployment only to rewrite the
upstream URL and dropped its model_info, so the standard logging payload and
the Prometheus litellm_deployment_success_responses_total counter carried
model_id="". Carry the deployment's model_info through request.state into the
passthrough logging metadata, where it overrides any client-supplied model_info.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
_handle_clientside_credential registered the per-request Deployment it built for
a client-supplied api_key/api_base via upsert_deployment, which added it to
self.model_list under the shared model_name. That made a request-scoped
credential a permanent, load-balanced deployment that any later caller of the
same model group could be routed onto, reaching the provider with someone
else's forwarded credential.
The per-request Deployment still gets its own stable id for cooldown and
logging identity; it is just never registered with the router.
Resolves LIT-7811
The failure logger skips fallback hops (has_logged_async_failure is already set), so
model_call_details.end_time still belongs to the previous hop and predates this hop's
api_call_start_time. The fallback cooldown guard measured a negative elapsed time and
cooled down deployments for caller-set timeouts.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
client_side_timeout records that the caller configured a timeout, not that
the timeout fired. A 408 the provider returns before that deadline is a
deployment failure and must still count toward cooldown.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
A 408 produced by a timeout the caller set (a timeout body field or an
x-litellm-timeout header, which the proxy marks as client_side_timeout)
says nothing about the deployment's health, yet the router's primary
failure callback counted it toward allowed_fails and cooled the
deployment down. The fallback path already skipped it.
The marker never reached that callback because get_litellm_params drops
kwargs outside OPTIONAL_KWARGS_KEYS, so it is listed there now, and
deployment_callback_on_failure returns before the failure counter when
is_caller_timeout_408 holds. A 408 from a timeout the deployment or the
provider set still counts and still cools the deployment down.
