mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-01 02:02:20 +00:00
fix(compact_20260112): enforce user/project/team-member model scopes on summary subrequest
Some checks are pending
Unit Tests: Proxy DB Operations / assert-shard-coverage (push) Waiting to run
Unit Tests: Proxy DB Operations / auth-checks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / budgets (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / custom-logging (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / db-and-spend (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / endpoints-and-responses (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / guardrails-hooks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / jwt-and-keys (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / key-generation (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / logging-misc (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-runtime (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-server-core (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / schema-migration (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-utils (push) Blocked by required conditions
Unit Tests: Security / security (push) Waiting to run
Some checks are pending
Unit Tests: Proxy DB Operations / assert-shard-coverage (push) Waiting to run
Unit Tests: Proxy DB Operations / auth-checks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / budgets (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / custom-logging (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / db-and-spend (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / endpoints-and-responses (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / guardrails-hooks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / jwt-and-keys (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / key-generation (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / logging-misc (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-runtime (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-server-core (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / schema-migration (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-utils (push) Blocked by required conditions
Unit Tests: Security / security (push) Waiting to run
The local gate previously only checked the parent key's and team's allowed-model lists. A caller restricted by a personal user, project, or per-team-member allowed_models scope could still trigger the configured summary model and receive its <summary> output as a compaction block, because llm_router.acompletion bypasses the proxy common_checks path. Extend _check_summary_model_access to also load the user_object, project_object, and team_membership and run the matching allowlist check at each scope before invoking the summary model.
This commit is contained in:
parent
d501bace1c
commit
acff8291e5
2 changed files with 298 additions and 12 deletions
|
|
@ -84,31 +84,49 @@ def _read_summary_max_tokens_setting() -> int:
|
|||
return COMPACT_SUMMARY_MAX_TOKENS
|
||||
|
||||
|
||||
def _check_summary_model_access(
|
||||
async def _check_summary_model_access( # noqa: PLR0915
|
||||
user_api_key_auth: Any,
|
||||
summary_model: str,
|
||||
llm_router: Any,
|
||||
) -> bool:
|
||||
"""Return True when the parent key/team is authorized to call ``summary_model``.
|
||||
"""Return True when every model-allowlist scope on the parent request is
|
||||
satisfied for ``summary_model``.
|
||||
|
||||
The summary subrequest does not pass through ``user_api_key_auth`` again,
|
||||
so without this gate a caller whose key/team is not allowed to use
|
||||
``context_management_summary_model`` could still get the proxy to invoke
|
||||
that model and return its ``<summary>`` output as a compaction block.
|
||||
so without this gate a caller whose configured scope at any of these
|
||||
levels excludes ``context_management_summary_model`` could still get the
|
||||
proxy to invoke that model and return its ``<summary>`` output as a
|
||||
compaction block. Mirrors the model-scope enforcement that
|
||||
``litellm.proxy.auth.common_checks`` runs for the client-requested model:
|
||||
key, team, user (personal), project, and team-member allowlists.
|
||||
|
||||
Returns True (allow) when ``user_api_key_auth`` is not present — SDK
|
||||
callers and tests run outside the proxy, where no key/team policy exists.
|
||||
Returns False when either the key-level or team-level allowlist denies
|
||||
the summary model (``ProxyException`` from ``_can_object_call_model``).
|
||||
Unexpected errors during the check (e.g. router internals raising) fail
|
||||
closed but are logged separately so operators can distinguish them from
|
||||
a real access-denied response.
|
||||
Returns False when any of the active allowlists denies the summary model
|
||||
(``ProxyException`` from ``_can_object_call_model`` / ``can_*_model``).
|
||||
Unexpected errors during an access check fail closed but are logged
|
||||
separately so operators can distinguish them from a real access-denied
|
||||
response. DB-lookup failures (object missing from cache or DB) skip the
|
||||
corresponding scope — matching ``common_checks``, which only enforces a
|
||||
scope when its backing object can be loaded.
|
||||
"""
|
||||
if user_api_key_auth is None:
|
||||
return True
|
||||
try:
|
||||
from litellm.proxy._types import ProxyException
|
||||
from litellm.proxy.auth.auth_checks import _can_object_call_model
|
||||
from litellm.proxy.auth.auth_checks import (
|
||||
_can_object_call_model,
|
||||
can_project_access_model,
|
||||
can_user_call_model,
|
||||
get_project_object,
|
||||
get_team_membership,
|
||||
get_user_object,
|
||||
)
|
||||
from litellm.proxy.proxy_server import (
|
||||
prisma_client,
|
||||
proxy_logging_obj,
|
||||
user_api_key_cache,
|
||||
)
|
||||
except Exception:
|
||||
return True
|
||||
|
||||
|
|
@ -116,6 +134,8 @@ def _check_summary_model_access(
|
|||
team_id = getattr(user_api_key_auth, "team_id", None)
|
||||
team_model_aliases = getattr(user_api_key_auth, "team_model_aliases", None)
|
||||
team_models = list(getattr(user_api_key_auth, "team_models", None) or [])
|
||||
user_id = getattr(user_api_key_auth, "user_id", None)
|
||||
project_id = getattr(user_api_key_auth, "project_id", None)
|
||||
|
||||
checks: Tuple[Tuple[Literal["key", "team"], List[str]], ...] = (
|
||||
("key", key_models),
|
||||
|
|
@ -145,6 +165,119 @@ def _check_summary_model_access(
|
|||
)
|
||||
return False
|
||||
|
||||
if user_id is not None and prisma_client is not None:
|
||||
try:
|
||||
user_obj = await get_user_object(
|
||||
user_id=user_id,
|
||||
prisma_client=prisma_client,
|
||||
user_api_key_cache=user_api_key_cache,
|
||||
user_id_upsert=False,
|
||||
proxy_logging_obj=proxy_logging_obj,
|
||||
)
|
||||
except Exception as e:
|
||||
verbose_logger.debug(
|
||||
"compact_20260112: user object lookup failed for "
|
||||
"summary_model=%s access check; skipping user-level scope: %s",
|
||||
summary_model,
|
||||
e,
|
||||
)
|
||||
user_obj = None
|
||||
if user_obj is not None:
|
||||
try:
|
||||
await can_user_call_model(
|
||||
model=summary_model,
|
||||
llm_router=llm_router,
|
||||
user_object=user_obj,
|
||||
)
|
||||
except ProxyException:
|
||||
return False
|
||||
except Exception as e:
|
||||
verbose_logger.warning(
|
||||
"compact_20260112: unexpected error during user-level "
|
||||
"access check for summary_model=%s; denying access: %s",
|
||||
summary_model,
|
||||
e,
|
||||
)
|
||||
return False
|
||||
|
||||
if project_id is not None and prisma_client is not None:
|
||||
try:
|
||||
project_obj = await get_project_object(
|
||||
project_id=project_id,
|
||||
prisma_client=prisma_client,
|
||||
user_api_key_cache=user_api_key_cache,
|
||||
proxy_logging_obj=proxy_logging_obj,
|
||||
)
|
||||
except Exception as e:
|
||||
verbose_logger.debug(
|
||||
"compact_20260112: project object lookup failed for "
|
||||
"summary_model=%s access check; skipping project-level scope: %s",
|
||||
summary_model,
|
||||
e,
|
||||
)
|
||||
project_obj = None
|
||||
if project_obj is not None and project_obj.models:
|
||||
try:
|
||||
can_project_access_model(
|
||||
model=summary_model,
|
||||
project_object=project_obj,
|
||||
llm_router=llm_router,
|
||||
)
|
||||
except ProxyException:
|
||||
return False
|
||||
except Exception as e:
|
||||
verbose_logger.warning(
|
||||
"compact_20260112: unexpected error during project-level "
|
||||
"access check for summary_model=%s; denying access: %s",
|
||||
summary_model,
|
||||
e,
|
||||
)
|
||||
return False
|
||||
|
||||
if user_id is not None and team_id is not None and prisma_client is not None:
|
||||
try:
|
||||
team_membership = await get_team_membership(
|
||||
user_id=user_id,
|
||||
team_id=team_id,
|
||||
prisma_client=prisma_client,
|
||||
user_api_key_cache=user_api_key_cache,
|
||||
proxy_logging_obj=proxy_logging_obj,
|
||||
)
|
||||
except Exception as e:
|
||||
verbose_logger.debug(
|
||||
"compact_20260112: team membership lookup failed for "
|
||||
"summary_model=%s access check; skipping member-level scope: %s",
|
||||
summary_model,
|
||||
e,
|
||||
)
|
||||
team_membership = None
|
||||
member_allowed_models = (
|
||||
team_membership.litellm_budget_table.allowed_models
|
||||
if team_membership is not None
|
||||
and team_membership.litellm_budget_table is not None
|
||||
else None
|
||||
)
|
||||
if member_allowed_models:
|
||||
try:
|
||||
_can_object_call_model(
|
||||
model=summary_model,
|
||||
llm_router=llm_router,
|
||||
models=list(member_allowed_models),
|
||||
team_model_aliases=team_model_aliases,
|
||||
team_id=team_id,
|
||||
object_type="team",
|
||||
)
|
||||
except ProxyException:
|
||||
return False
|
||||
except Exception as e:
|
||||
verbose_logger.warning(
|
||||
"compact_20260112: unexpected error during member-level "
|
||||
"access check for summary_model=%s; denying access: %s",
|
||||
summary_model,
|
||||
e,
|
||||
)
|
||||
return False
|
||||
|
||||
return True
|
||||
|
||||
|
||||
|
|
@ -774,7 +907,7 @@ async def apply_compact_20260112( # noqa: PLR0915
|
|||
# Phase C: summarize. ``augmented_system`` carries any prior compaction
|
||||
# summary so multi-round compaction does not lose accumulated history —
|
||||
# ``effective_messages`` only contains turns since the last compaction.
|
||||
if not _check_summary_model_access(
|
||||
if not await _check_summary_model_access(
|
||||
user_api_key_auth=user_api_key_auth,
|
||||
summary_model=summary_model,
|
||||
llm_router=llm_router,
|
||||
|
|
|
|||
|
|
@ -1295,6 +1295,159 @@ async def test_summary_model_allowed_when_no_user_api_key_auth():
|
|||
assert result.compaction_block is not None
|
||||
|
||||
|
||||
async def test_summary_model_denied_when_user_scope_excludes_it():
|
||||
"""Personal user allowed-models scope denies the summary model even when
|
||||
key/team allowlists permit it."""
|
||||
messages = _simple_messages()
|
||||
mock_call = AsyncMock(return_value=_make_mock_response("<summary>x</summary>"))
|
||||
|
||||
auth = _fake_user_api_key_auth(key_models=["all-proxy-models"])
|
||||
auth.user_id = "user-123"
|
||||
|
||||
class _User:
|
||||
user_id = "user-123"
|
||||
models = ["gpt-3.5-turbo"]
|
||||
organization_memberships = []
|
||||
|
||||
with (
|
||||
patch(
|
||||
"litellm.llms.anthropic.experimental_pass_through.context_management.editors.compact._read_summary_model_setting",
|
||||
return_value="claude-haiku-4-5",
|
||||
),
|
||||
patch("litellm.token_counter", return_value=200_000),
|
||||
patch(
|
||||
"litellm.llms.anthropic.experimental_pass_through.context_management.editors.compact._call_summary_model",
|
||||
mock_call,
|
||||
),
|
||||
patch(
|
||||
"litellm.proxy.auth.auth_checks.get_user_object",
|
||||
AsyncMock(return_value=_User()),
|
||||
),
|
||||
patch(
|
||||
"litellm.proxy.auth.auth_checks.get_team_membership",
|
||||
AsyncMock(return_value=None),
|
||||
),
|
||||
patch(
|
||||
"litellm.proxy.auth.auth_checks.get_project_object",
|
||||
AsyncMock(return_value=None),
|
||||
),
|
||||
patch("litellm.proxy.proxy_server.prisma_client", MagicMock()),
|
||||
):
|
||||
result = await apply_compact_20260112(
|
||||
model=MODEL,
|
||||
messages=messages,
|
||||
tools=None,
|
||||
system=None,
|
||||
edit_spec=_EDIT_SPEC_DEFAULT,
|
||||
user_api_key_auth=auth,
|
||||
)
|
||||
|
||||
mock_call.assert_not_awaited()
|
||||
assert result.applied_edits[0].get("error") == "summary_model_access_denied"
|
||||
|
||||
|
||||
async def test_summary_model_denied_when_project_scope_excludes_it():
|
||||
"""Project allowed-models scope denies the summary model even when
|
||||
key/team allowlists permit it."""
|
||||
messages = _simple_messages()
|
||||
mock_call = AsyncMock(return_value=_make_mock_response("<summary>x</summary>"))
|
||||
|
||||
auth = _fake_user_api_key_auth(key_models=["all-proxy-models"])
|
||||
auth.project_id = "project-1"
|
||||
|
||||
class _Project:
|
||||
project_id = "project-1"
|
||||
models = ["gpt-3.5-turbo"]
|
||||
|
||||
with (
|
||||
patch(
|
||||
"litellm.llms.anthropic.experimental_pass_through.context_management.editors.compact._read_summary_model_setting",
|
||||
return_value="claude-haiku-4-5",
|
||||
),
|
||||
patch("litellm.token_counter", return_value=200_000),
|
||||
patch(
|
||||
"litellm.llms.anthropic.experimental_pass_through.context_management.editors.compact._call_summary_model",
|
||||
mock_call,
|
||||
),
|
||||
patch(
|
||||
"litellm.proxy.auth.auth_checks.get_user_object",
|
||||
AsyncMock(return_value=None),
|
||||
),
|
||||
patch(
|
||||
"litellm.proxy.auth.auth_checks.get_team_membership",
|
||||
AsyncMock(return_value=None),
|
||||
),
|
||||
patch(
|
||||
"litellm.proxy.auth.auth_checks.get_project_object",
|
||||
AsyncMock(return_value=_Project()),
|
||||
),
|
||||
patch("litellm.proxy.proxy_server.prisma_client", MagicMock()),
|
||||
):
|
||||
result = await apply_compact_20260112(
|
||||
model=MODEL,
|
||||
messages=messages,
|
||||
tools=None,
|
||||
system=None,
|
||||
edit_spec=_EDIT_SPEC_DEFAULT,
|
||||
user_api_key_auth=auth,
|
||||
)
|
||||
|
||||
mock_call.assert_not_awaited()
|
||||
assert result.applied_edits[0].get("error") == "summary_model_access_denied"
|
||||
|
||||
|
||||
async def test_summary_model_denied_when_team_member_scope_excludes_it():
|
||||
"""Per-team-member allowed-models scope denies the summary model even
|
||||
when key/team allowlists permit it."""
|
||||
messages = _simple_messages()
|
||||
mock_call = AsyncMock(return_value=_make_mock_response("<summary>x</summary>"))
|
||||
|
||||
auth = _fake_user_api_key_auth(key_models=["all-proxy-models"], team_id="team-1")
|
||||
auth.user_id = "user-123"
|
||||
|
||||
class _Budget:
|
||||
allowed_models = ["gpt-3.5-turbo"]
|
||||
|
||||
class _Membership:
|
||||
litellm_budget_table = _Budget()
|
||||
|
||||
with (
|
||||
patch(
|
||||
"litellm.llms.anthropic.experimental_pass_through.context_management.editors.compact._read_summary_model_setting",
|
||||
return_value="claude-haiku-4-5",
|
||||
),
|
||||
patch("litellm.token_counter", return_value=200_000),
|
||||
patch(
|
||||
"litellm.llms.anthropic.experimental_pass_through.context_management.editors.compact._call_summary_model",
|
||||
mock_call,
|
||||
),
|
||||
patch(
|
||||
"litellm.proxy.auth.auth_checks.get_user_object",
|
||||
AsyncMock(return_value=None),
|
||||
),
|
||||
patch(
|
||||
"litellm.proxy.auth.auth_checks.get_team_membership",
|
||||
AsyncMock(return_value=_Membership()),
|
||||
),
|
||||
patch(
|
||||
"litellm.proxy.auth.auth_checks.get_project_object",
|
||||
AsyncMock(return_value=None),
|
||||
),
|
||||
patch("litellm.proxy.proxy_server.prisma_client", MagicMock()),
|
||||
):
|
||||
result = await apply_compact_20260112(
|
||||
model=MODEL,
|
||||
messages=messages,
|
||||
tools=None,
|
||||
system=None,
|
||||
edit_spec=_EDIT_SPEC_DEFAULT,
|
||||
user_api_key_auth=auth,
|
||||
)
|
||||
|
||||
mock_call.assert_not_awaited()
|
||||
assert result.applied_edits[0].get("error") == "summary_model_access_denied"
|
||||
|
||||
|
||||
async def test_summary_call_propagates_allowed_model_region():
|
||||
"""``allowed_model_region`` from ``user_api_key_auth`` is propagated to the
|
||||
summary subrequest as a top-level kwarg so the router applies the same
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue