mirror of
https://github.com/BerriAI/litellm.git
synced 2026-08-28 05:25:59 +00:00
2a55d23731
* fix(proxy): merge model-level guardrails before pre_call_hook DB/UI-assigned guardrails (litellm_params.guardrails) only fire on post_call paths today: _check_and_merge_model_level_guardrails is called in utils.py:2234 + utils.py:2498 + common_request_processing.py:1665, but never before pre_call_hook in common_request_processing.py:963. PR #23774 fixed the non-streaming post_call case; pre_call was left broken. At the pre_call site, add_litellm_data_to_request strips client-supplied metadata.model_info (pricing spoofing guard) and route_request hasn't run yet, so model_info.id is unavailable. Extend the helper to fall back to llm_router.get_deployment_by_model_group_name(model_alias) when model_id is missing — that uses the O(1) model-name index already maintained by the router. Closes #29652 * fix(mcp): surface mcp_server_name in synthetic _convert_mcp_to_llm_format payload Addresses veria-ai Medium finding + proxy-infra CI failure on this PR. ParallelRequestLimiterV3 reads data["mcp_server_name"] for call_mcp_tool hook payloads when applying key/team mcp_rpm_limit. _convert_mcp_to_llm_format was omitting the field, so a key with mcp_rpm_limit could exceed it via the MCP path. Reads from kwargs.get("mcp_rate_limit_server_name") to match how pre_call_tool_check resolves the alias-then-server-name fallback before invoking hooks. * fix(proxy): union guardrails across group deployments on alias fallback Addresses second veria-ai Medium on #29654: the alias fallback called get_deployment_by_model_group_name(), which returns ONE deployment. A guardrail set on a non-first deployment would silently not run on pre_call when the model_id is missing. Switch to get_model_list(model_name=...) and take the UNION of litellm_params.guardrails across all matching deployments (with dedup). Trade-off documented in the comment: pre_call cannot know which deployment route_request will select, so the conservative choice is to apply any guardrail set on any eligible deployment. Updated test stubs to use get_model_list. Added 3 new tests covering union, dedup, and the all-empty case. * test(model_level_guardrails): align integration test with get_model_list union API * fix(proxy): ignore client-supplied model_info.id on pre_call merge + lint Addresses 3rd veria-ai Medium on #29654: add_litellm_data_to_request preserves client-supplied metadata.model_info when the caller's key/team has allow_client_pricing_override. The pre_call merge previously trusted that id, so a caller could spoof an unguarded model_info.id while requesting a guarded alias and bypass guardrails. New `trust_client_model_info: bool` param on the helper. The pre_call call site passes False; post_call paths (existing) keep True. Also fixes the ruff failure on the union loop: pulled the .get() into a local + isinstance(list) check before iterating, so mypy stops complaining about `object` not being iterable. 2 new regression tests covering spoof-and-bypass + default-trust behavior. * fix(proxy): pass team_id to alias-lookup + restore scalar-string guardrail acceptance Addresses two more reviewer findings on #29654: veria-ai Medium: route_request resolves team-scoped public model names with metadata.user_api_key_team_id. The pre_call alias fallback called get_model_list(model_name=...) without the team_id, so team-scoped deployments were invisible and their pre_call guardrails silently skipped. Now reads team_id from metadata or litellm_metadata and passes it to get_model_list. greptile P1: the isinstance(deployment_guardrails, list) guard added for mypy narrowing silently dropped bare-string guardrail values that the existing post_call path used to truthy-accept. Restored by wrapping a scalar string into a one-element list on both paths. 4 new tests: team_id passthrough (metadata + litellm_metadata), scalar on post_call, scalar on alias-union. 36/36 tests pass. * style: black formatting on _check_and_merge_model_level_guardrails team_id assignment * chore: ruff format * fix(lint): remove unused noqa PLR0915 directive RUF100 flags the # noqa: PLR0915 on common_processing_pre_call_logic because PLR0915 is not in this repo's enabled ruff rule set (lint.extend-select in ruff.toml), so the directive suppresses nothing and fails the lint job. * refactor(proxy): hoist guardrail-merge import to module top The pre_call guardrail-merge helper was imported inside common_processing_pre_call_logic with a # noqa: PLC0415, which the type-discipline gate counts as an unexplained suppression (LIT003). The inline import's cyclic-import justification does not hold: this module already imports from litellm.proxy.utils at top level, and utils.py does not import common_request_processing at module load. Fold the helper into the existing top-level import and drop the inline import, clearing the suppression instead of budgeting for it. --------- Co-authored-by: Yassin Kortam <yassin.kortam@gmail.com> |
||
|---|---|---|
| .. | ||
| helpers | ||
| prisma_and_spend | ||
| proxy_logging | ||
| __init__.py | ||