Key and team `router_settings.model_group_alias` was accepted, persisted and
echoed back by `/key/info`, but never applied at request time, so the request
ran on the group the caller asked for. `route_request` forwards only the
settings the Router accepts as per-request kwargs, and `model_group_alias` is
not one of them: the Router resolves aliases from its own instance attribute,
which holds the global config map and is shared across requests.
Resolve the alias in the proxy instead, alongside the existing model-alias
rewrites and ahead of the pre-call hooks, so per-model limits and guardrails
key off the group that actually serves the request. Authorize the alias target
before the rewrite; model access was checked against the requested group, so a
key whose alias points at a group it cannot call gets the usual 403 rather than
being quietly served it.
Resolves LIT-4879
* fix(proxy): merge model-level guardrails before pre_call_hook
DB/UI-assigned guardrails (litellm_params.guardrails) only fire on
post_call paths today: _check_and_merge_model_level_guardrails is called
in utils.py:2234 + utils.py:2498 + common_request_processing.py:1665, but
never before pre_call_hook in common_request_processing.py:963. PR #23774
fixed the non-streaming post_call case; pre_call was left broken.
At the pre_call site, add_litellm_data_to_request strips client-supplied
metadata.model_info (pricing spoofing guard) and route_request hasn't run
yet, so model_info.id is unavailable. Extend the helper to fall back to
llm_router.get_deployment_by_model_group_name(model_alias) when model_id
is missing — that uses the O(1) model-name index already maintained by
the router.
Closes#29652
* fix(mcp): surface mcp_server_name in synthetic _convert_mcp_to_llm_format payload
Addresses veria-ai Medium finding + proxy-infra CI failure on this PR.
ParallelRequestLimiterV3 reads data["mcp_server_name"] for call_mcp_tool
hook payloads when applying key/team mcp_rpm_limit. _convert_mcp_to_llm_format
was omitting the field, so a key with mcp_rpm_limit could exceed it via the
MCP path.
Reads from kwargs.get("mcp_rate_limit_server_name") to match how
pre_call_tool_check resolves the alias-then-server-name fallback before
invoking hooks.
* fix(proxy): union guardrails across group deployments on alias fallback
Addresses second veria-ai Medium on #29654: the alias fallback called
get_deployment_by_model_group_name(), which returns ONE deployment.
A guardrail set on a non-first deployment would silently not run on
pre_call when the model_id is missing.
Switch to get_model_list(model_name=...) and take the UNION of
litellm_params.guardrails across all matching deployments (with dedup).
Trade-off documented in the comment: pre_call cannot know which
deployment route_request will select, so the conservative choice is to
apply any guardrail set on any eligible deployment.
Updated test stubs to use get_model_list. Added 3 new tests covering
union, dedup, and the all-empty case.
* test(model_level_guardrails): align integration test with get_model_list union API
* fix(proxy): ignore client-supplied model_info.id on pre_call merge + lint
Addresses 3rd veria-ai Medium on #29654:
add_litellm_data_to_request preserves client-supplied metadata.model_info
when the caller's key/team has allow_client_pricing_override. The pre_call
merge previously trusted that id, so a caller could spoof an unguarded
model_info.id while requesting a guarded alias and bypass guardrails.
New `trust_client_model_info: bool` param on the helper. The pre_call
call site passes False; post_call paths (existing) keep True.
Also fixes the ruff failure on the union loop: pulled the .get() into a
local + isinstance(list) check before iterating, so mypy stops complaining
about `object` not being iterable.
2 new regression tests covering spoof-and-bypass + default-trust behavior.
* fix(proxy): pass team_id to alias-lookup + restore scalar-string guardrail acceptance
Addresses two more reviewer findings on #29654:
veria-ai Medium: route_request resolves team-scoped public model names
with metadata.user_api_key_team_id. The pre_call alias fallback called
get_model_list(model_name=...) without the team_id, so team-scoped
deployments were invisible and their pre_call guardrails silently
skipped. Now reads team_id from metadata or litellm_metadata and passes
it to get_model_list.
greptile P1: the isinstance(deployment_guardrails, list) guard added for
mypy narrowing silently dropped bare-string guardrail values that the
existing post_call path used to truthy-accept. Restored by wrapping a
scalar string into a one-element list on both paths.
4 new tests: team_id passthrough (metadata + litellm_metadata), scalar
on post_call, scalar on alias-union. 36/36 tests pass.
* style: black formatting on _check_and_merge_model_level_guardrails team_id assignment
* chore: ruff format
* fix(lint): remove unused noqa PLR0915 directive
RUF100 flags the # noqa: PLR0915 on common_processing_pre_call_logic
because PLR0915 is not in this repo's enabled ruff rule set
(lint.extend-select in ruff.toml), so the directive suppresses nothing
and fails the lint job.
* refactor(proxy): hoist guardrail-merge import to module top
The pre_call guardrail-merge helper was imported inside
common_processing_pre_call_logic with a # noqa: PLC0415, which the
type-discipline gate counts as an unexplained suppression (LIT003). The
inline import's cyclic-import justification does not hold: this module
already imports from litellm.proxy.utils at top level, and utils.py does
not import common_request_processing at module load. Fold the helper
into the existing top-level import and drop the inline import, clearing
the suppression instead of budgeting for it.
---------
Co-authored-by: Yassin Kortam <yassin.kortam@gmail.com>
* fix(guardrails): run pre_call hook once for model-level guardrails
A CustomGuardrail attached to a deployment via litellm_params.guardrails
gets its async_pre_call_hook invoked twice per request: once by the proxy
pre-call loop and again by async_pre_call_deployment_hook after the router
spreads the model-level guardrails into the top-level request kwargs.
Record in request metadata that the proxy pre-call loop already ran a given
guardrail, and have the deployment hook skip it when the marker is present.
Direct-SDK usage never runs the proxy loop, so the deployment hook stays the
sole invocation there and still fires exactly once.
The marker key is stripped from untrusted caller metadata so a request body
cannot suppress a model-only guardrail by pre-seeding it.
* fix(guardrails): mark pre_call dedup on the post-hook request data
Record the exactly-once marker after async_pre_call_hook runs, on the data
object that flows downstream, rather than before it. A guardrail whose hook
returns a brand-new request dict (instead of mutating or spreading the one it
received) would otherwise discard the marker, letting the deployment hook
re-run the guardrail a second time.
Model-level guardrails (litellm_params.guardrails on a deployment) were
only merged into request metadata in the streaming post_call path
(async_post_call_streaming_hook) but not in the non-streaming path
(post_call_success_hook). This caused should_run_guardrail to skip the
guardrail because the guardrail name was never added to metadata.guardrails.
Add the same _check_and_merge_model_level_guardrails call to
post_call_success_hook before the guardrail callback loop, mirroring the
streaming path.
Fixes model-level guardrails silently not firing for non-streaming
post_call requests.
