From 50a3f10a9260e147c11501507042b916e325cedf Mon Sep 17 00:00:00 2001 From: Sameer Kankute Date: Fri, 22 May 2026 22:28:50 +0530 Subject: [PATCH 01/13] feat(proxy): persist allowlisted OIDC claims in CLI SSO poll (#28463) * feat(proxy): persist allowlisted OIDC claims in CLI SSO poll Map CLI_SSO_CLAIM_MAP sources into user metadata and return scalar attribution_metadata from /sso/cli/poll. Build SSOUserDefinedValues in cli_sso_callback so first-time CLI logins can upsert users. Add mock OIDC scripts and tests for claim extraction and poll exposure. Co-authored-by: Cursor * docs(proxy): document CLI SSO attribution_metadata in client README Co-authored-by: Cursor * Delete scripts/mock_oidc_server_for_cli_sso.py * Delete scripts/test_cli_sso_claims_e2e.py * fix(ui_sso): preserve claim types and avoid metadata. prefix stripping - Replace _update_dictionary with a local recursive merge so string OIDC claim values that happen to look numeric are not silently coerced to int/float when persisting CLI SSO attribution metadata. - Use a local dot-path resolver in _extract_sso_claim_value so that source claim paths beginning with 'metadata.' are not silently stripped by get_nested_value (which is designed for LiteLLM JWT metadata, not arbitrary OIDC claims). Co-authored-by: Yassin Kortam * Remove redundant metadata. prefix strip in _set_nested_metadata_value The _parse_cli_sso_claim_map already strips the metadata. prefix from dest keys before reaching the setter. The duplicate strip in _set_nested_metadata_value was a no-op in normal flow but could mis-place values for dest keys like metadata.metadata.foo. Co-authored-by: Yassin Kortam * Fix greptile * Fix ruff * Move CLI SSO user defined values build inside try/except for consistent error handling Co-authored-by: Yassin Kortam * fix(proxy): enforce restricted SSO group on CLI SSO callback Apply verify_user_in_restricted_sso_group before CLI session completion and user upsert, matching the UI SSO path. Re-raise ProxyException so restricted-group denials return 403 instead of 500. Co-authored-by: Cursor * fix(proxy): replace recursive CLI SSO metadata helpers with iterative merge Use stack-based flatten/merge to satisfy recursive_detector CI. Fix mypy types for UserApiKeyCache and user_id on CLI SSO session completion. Co-authored-by: Cursor * fix: resolve nested CustomOpenID extra_fields in CLI SSO claim extraction When GENERIC_USER_EXTRA_ATTRIBUTES captures a parent object (e.g. org_info), extra_fields stores it as {"org_info": {"department": "..."}}. A CLI claim map entry using a dotted path like org_info.department would silently fail because the lookup only checked the exact flat key. Fall back to dotted-path resolution on extra_fields before model_dump(). Co-authored-by: Yassin Kortam * fix(sso): update CLI SSO test for new received_response kwarg and remove redundant 'token' secret fragment Co-authored-by: Yassin Kortam --------- Co-authored-by: Cursor Co-authored-by: Yassin Kortam --- litellm/constants.py | 6 + litellm/proxy/client/README.md | 2 +- litellm/proxy/management_endpoints/ui_sso.py | 509 +++++++++++++++--- .../proxy/management_endpoints/test_ui_sso.py | 284 ++++++++++ 4 files changed, 717 insertions(+), 84 deletions(-) diff --git a/litellm/constants.py b/litellm/constants.py index e36746326cc..fb765c0226c 100644 --- a/litellm/constants.py +++ b/litellm/constants.py @@ -1443,6 +1443,12 @@ CLI_JWT_EXPIRATION_HOURS = int( or os.getenv("LITELLM_CLI_JWT_EXPIRATION_HOURS") or 24 ) +# Comma-separated allowlisted OIDC claim map for CLI SSO polling, e.g. +# "employment_type->acme_employment_type,org_info.department->department" +CLI_SSO_CLAIM_MAP = ( + os.getenv("CLI_SSO_CLAIM_MAP") or os.getenv("LITELLM_CLI_SSO_CLAIM_MAP") or "" +) +CLI_SSO_CLAIM_MAX_SCALAR_LENGTH = 1024 ########################### UI SESSION DURATION ########################### # Duration for UI login session (username/password, SSO, invitation links). Format: "30s", "30m", "24h", "7d" diff --git a/litellm/proxy/client/README.md b/litellm/proxy/client/README.md index adf562d69c5..9fbc6f2197d 100644 --- a/litellm/proxy/client/README.md +++ b/litellm/proxy/client/README.md @@ -350,7 +350,7 @@ The CLI provides three authentication commands: 4. **User Authentication**: User completes SSO authentication in browser 5. **Callback Processing**: SSO provider redirects back to proxy with state parameter 6. **User Code Verification**: Browser confirms the verification code shown in the CLI -7. **Polling**: CLI polls `/sso/cli/poll/{login_id}` with the polling secret header until the JWT is ready +7. **Polling**: CLI polls `/sso/cli/poll/{login_id}` with the polling secret header until the JWT is ready. When `CLI_SSO_CLAIM_MAP` is configured on the proxy, the poll response may include `attribution_metadata` (allowlisted scalar OIDC claims for client attribution). 8. **Token Storage**: CLI saves the authentication token to `~/.litellm/token.json` ### Benefits of This Approach diff --git a/litellm/proxy/management_endpoints/ui_sso.py b/litellm/proxy/management_endpoints/ui_sso.py index ff3bbf47389..d3e1099d968 100644 --- a/litellm/proxy/management_endpoints/ui_sso.py +++ b/litellm/proxy/management_endpoints/ui_sso.py @@ -43,6 +43,8 @@ from litellm.caching.dual_cache import DualCache from litellm._logging import verbose_proxy_logger from litellm._uuid import uuid from litellm.constants import ( + CLI_SSO_CLAIM_MAP, + CLI_SSO_CLAIM_MAX_SCALAR_LENGTH, CLI_SSO_SESSION_CACHE_KEY_PREFIX, CLI_SSO_SESSION_TTL_SECONDS, LITELLM_CLI_SOURCE_IDENTIFIER, @@ -140,6 +142,20 @@ _CLI_SSO_START_RATE_LIMIT_WINDOW_SECONDS = 60 _CLI_SSO_START_RATE_LIMIT_MAX_ATTEMPTS = 30 _CLI_SSO_USER_CODE_ALPHABET = "ABCDEFGHJKLMNPQRSTUVWXYZ23456789" _CLI_SSO_LOGIN_ID_RE = re.compile(r"^cli-[A-Za-z0-9_-]{12,124}$") +_CLI_SSO_SCALAR_TYPES = (str, int, float, bool) +_CLI_SSO_DEST_KEY_RE = re.compile(r"^[A-Za-z0-9_.-]+$") +_CLI_SSO_SECRET_KEY_FRAGMENTS = frozenset( + { + "access_token", + "api_key", + "client_secret", + "id_token", + "password", + "private_key", + "refresh_token", + "secret", + } +) def _hash_cli_sso_secret(secret: str) -> str: @@ -225,6 +241,239 @@ def _verify_cli_sso_poll_secret(flow: dict, poll_secret: Optional[str]) -> bool: return secrets.compare_digest(supplied_poll_secret_hash, expected_poll_secret_hash) +def _parse_cli_sso_claim_map() -> List[Tuple[str, str]]: + """ + Parse CLI_SSO_CLAIM_MAP / LITELLM_CLI_SSO_CLAIM_MAP. + + Format: comma-separated ``source_claim->metadata_key`` pairs, e.g. + ``employment_type->acme_employment_type,org_info.department->department``. + Destination keys may use an optional ``metadata.`` prefix; values are stored + on the LiteLLM user's ``metadata`` JSON column. + """ + claim_map_raw = CLI_SSO_CLAIM_MAP.strip() + if not claim_map_raw: + return [] + + parsed: List[Tuple[str, str]] = [] + for entry in claim_map_raw.split(","): + entry = entry.strip() + if not entry or "->" not in entry: + continue + source_claim, dest_key = entry.split("->", 1) + source_claim = source_claim.strip() + dest_key = dest_key.strip() + if dest_key.startswith("metadata."): + dest_key = dest_key[len("metadata.") :] + if source_claim and dest_key: + parsed.append((source_claim, dest_key)) + return parsed + + +def _is_safe_cli_sso_metadata_dest_key(dest_key: str) -> bool: + if not dest_key or not _CLI_SSO_DEST_KEY_RE.fullmatch(dest_key): + return False + lowered = dest_key.lower() + return not any(fragment in lowered for fragment in _CLI_SSO_SECRET_KEY_FRAGMENTS) + + +def _is_safe_cli_sso_scalar_claim_value(value: Any) -> bool: + if not isinstance(value, _CLI_SSO_SCALAR_TYPES): + return False + if isinstance(value, str): + if len(value) > CLI_SSO_CLAIM_MAX_SCALAR_LENGTH: + return False + if value.startswith("eyJ") and value.count(".") >= 2: + return False + return True + + +def _sso_result_to_dict(result: Union[CustomOpenID, OpenID, dict]) -> Dict[str, Any]: + if isinstance(result, dict): + return result + if hasattr(result, "model_dump"): + dumped = result.model_dump() + if isinstance(dumped, dict): + return cast(Dict[str, Any], dumped) + return {} + + +def _get_nested_claim_value(data: Dict[str, Any], claim_path: str) -> Any: + """Resolve a dot-notation claim path against an SSO result dict. + + Unlike ``get_nested_value``, this does not strip a leading ``metadata.`` + prefix, since OIDC claims may legitimately use ``metadata`` as a top-level + key. + """ + if not claim_path: + return None + if claim_path in data: + return data[claim_path] + placeholder = "\x00" + parts = claim_path.replace("\\.", placeholder).split(".") + parts = [p.replace(placeholder, ".") for p in parts] + current: Any = data + for part in parts: + if isinstance(current, dict) and part in current: + current = current[part] + else: + return None + return current + + +def _extract_sso_claim_value( + result: Union[CustomOpenID, OpenID, dict], claim_path: str +) -> Any: + extra_fields = getattr(result, "extra_fields", None) + if isinstance(extra_fields, dict): + if claim_path in extra_fields: + return extra_fields[claim_path] + nested = _get_nested_claim_value(extra_fields, claim_path) + if nested is not None: + return nested + + if isinstance(result, dict): + return _get_nested_claim_value(result, claim_path) + + result_dict = _sso_result_to_dict(result) + return _get_nested_claim_value(result_dict, claim_path) + + +def _set_nested_metadata_value( + metadata: Dict[str, Any], key_path: str, value: Any +) -> None: + placeholder = "\x00" + parts = key_path.replace("\\.", placeholder).split(".") + parts = [p.replace(placeholder, ".") for p in parts] + current: Any = metadata + for part in parts[:-1]: + existing = current.get(part) + if not isinstance(existing, dict): + existing = {} + current[part] = existing + current = existing + current[parts[-1]] = value + + +def _flatten_cli_sso_metadata_for_poll( + metadata: Dict[str, Any], +) -> Dict[str, Union[str, int, float, bool]]: + """Expose scalar attribution metadata as a flat dict for CLI poll responses.""" + flattened: Dict[str, Union[str, int, float, bool]] = {} + stack: List[Tuple[str, Any]] = [("", metadata)] + while stack: + prefix, value = stack.pop() + if isinstance(value, dict): + for key, nested in value.items(): + nested_prefix = f"{prefix}.{key}" if prefix else key + stack.append((nested_prefix, nested)) + elif _is_safe_cli_sso_scalar_claim_value(value): + flattened[prefix] = value + return flattened + + +def build_cli_sso_attribution_metadata( + result: Union[CustomOpenID, OpenID, dict], +) -> Dict[str, Any]: + """ + Build allowlisted, non-secret scalar attribution metadata from an SSO result. + + Sources are configured via CLI_SSO_CLAIM_MAP / LITELLM_CLI_SSO_CLAIM_MAP and + may include claims captured by GENERIC_USER_EXTRA_ATTRIBUTES on CustomOpenID. + """ + claim_map = _parse_cli_sso_claim_map() + if not claim_map: + return {} + + metadata: Dict[str, Any] = {} + for source_claim, dest_key in claim_map: + if not _is_safe_cli_sso_metadata_dest_key(dest_key): + verbose_proxy_logger.debug( + f"Skipping unsafe CLI SSO metadata destination key: {dest_key}" + ) + continue + + raw_value = _extract_sso_claim_value(result=result, claim_path=source_claim) + if not _is_safe_cli_sso_scalar_claim_value(raw_value): + continue + + _set_nested_metadata_value( + metadata=metadata, key_path=dest_key, value=raw_value + ) + + return metadata + + +def _merge_cli_sso_attribution_metadata( + existing_metadata: Dict[str, Any], attribution_metadata: Dict[str, Any] +) -> Dict[str, Any]: + """Merge attribution metadata into existing user metadata in-place. + + Preserves original value types (in particular, string claim values that + happen to look numeric are NOT coerced to ``int``/``float``). Nested dicts + are merged iteratively so attribution claims do not clobber unrelated keys + under the same parent. + """ + pending: List[Tuple[Dict[str, Any], Dict[str, Any]]] = [ + (existing_metadata, attribution_metadata) + ] + while pending: + target, source = pending.pop() + for key, value in source.items(): + if value is None: + continue + existing_value = target.get(key) + if isinstance(value, dict) and isinstance(existing_value, dict): + pending.append((existing_value, value)) + else: + target[key] = value + return existing_metadata + + +async def _persist_cli_sso_user_metadata( + prisma_client: PrismaClient, + user_id: str, + attribution_metadata: Dict[str, Any], +) -> None: + if not attribution_metadata: + return + + try: + user_row = await prisma_client.db.litellm_usertable.find_unique( + where={"user_id": user_id} + ) + existing_metadata: Dict[str, Any] = {} + if user_row is not None: + row_metadata = user_row.metadata + if isinstance(row_metadata, dict): + existing_metadata = deepcopy(row_metadata) + + merged_metadata = _merge_cli_sso_attribution_metadata( + existing_metadata=existing_metadata, + attribution_metadata=attribution_metadata, + ) + await prisma_client.db.litellm_usertable.update_many( + where={"user_id": user_id}, + data={"metadata": merged_metadata}, + ) + verbose_proxy_logger.info( + f"Persisted CLI SSO attribution metadata for user {user_id}: " + f"{list(_flatten_cli_sso_metadata_for_poll(attribution_metadata).keys())}" + ) + except Exception as e: + verbose_proxy_logger.error( + f"Failed to persist CLI SSO attribution metadata for user {user_id}: {e}" + ) + + +def _cli_poll_attribution_metadata_from_session( + session_data: Dict[str, Any], +) -> Dict[str, Union[str, int, float, bool]]: + stored = session_data.get("attribution_metadata") + if isinstance(stored, dict): + return _flatten_cli_sso_metadata_for_poll(stored) + return {} + + def _render_cli_sso_verification_page( verify_url: str, browser_complete_token: str ) -> str: @@ -1674,7 +1923,12 @@ async def auth_callback(request: Request, state: Optional[str] = None): # noqa: key_id = state_parts[1] if len(state_parts) > 1 else None verbose_proxy_logger.info("CLI SSO callback detected") - return await cli_sso_callback(request=request, key=key_id, result=result) + return await cli_sso_callback( + request=request, + key=key_id, + result=result, + received_response=received_response, + ) # Control-plane cross-origin: read return_to from cookie. # Starlette's cookie_parser already handles RFC 2109 unquoting. @@ -1692,15 +1946,144 @@ async def auth_callback(request: Request, state: Optional[str] = None): # noqa: ) +async def _build_cli_sso_user_defined_values( + result: Union[OpenID, dict], + parsed_openid_result: ParsedOpenIDResult, +) -> Optional[SSOUserDefinedValues]: + from litellm.proxy.proxy_server import user_custom_sso + + user_id = parsed_openid_result.get("user_id") + if user_custom_sso is not None: + if inspect.iscoroutinefunction(user_custom_sso): + return await user_custom_sso(result) # type: ignore + raise ValueError("user_custom_sso must be a coroutine function") + if user_id is None: + return None + return SSOUserDefinedValues( + models=[], + user_id=user_id, + user_email=parsed_openid_result.get("user_email"), + max_budget=litellm.max_internal_user_budget, + user_role=parsed_openid_result.get("user_role"), + budget_duration=litellm.internal_user_budget_duration, + ) + + +async def _fetch_cli_sso_team_details( + prisma_client: PrismaClient, + teams: List[str], +) -> List[Dict[str, Any]]: + team_details: List[Dict[str, Any]] = [] + try: + if teams: + prisma_teams = await prisma_client.db.litellm_teamtable.find_many( + where={"team_id": {"in": teams}} + ) + for team_row in prisma_teams: + team_dict = team_row.model_dump() + team_details.append( + { + "team_id": team_dict.get("team_id"), + "team_alias": team_dict.get("team_alias"), + } + ) + except Exception as e: + verbose_proxy_logger.error( + f"Error fetching team details for CLI SSO session: {e}" + ) + return team_details + + +async def _complete_cli_sso_callback_session( + *, + request: Request, + key: str, + flow: dict, + result: Union[OpenID, dict], + parsed_openid_result: ParsedOpenIDResult, + user_defined_values: Optional[SSOUserDefinedValues], + prisma_client: PrismaClient, + user_api_key_cache: UserApiKeyCache, + proxy_logging_obj: ProxyLogging, +): + from fastapi.responses import HTMLResponse + + user_id = parsed_openid_result.get("user_id") + user_email = parsed_openid_result.get("user_email") + user_info = await get_user_info_from_db( + result=result, + prisma_client=prisma_client, + user_api_key_cache=user_api_key_cache, + proxy_logging_obj=proxy_logging_obj, + user_email=user_email, + user_defined_values=user_defined_values, + alternate_user_id=user_id, + ) + if user_info is None: + raise HTTPException( + status_code=500, detail="Failed to retrieve user information from SSO" + ) + if not user_info.user_id: + raise HTTPException( + status_code=500, detail="Failed to retrieve user information from SSO" + ) + + teams: List[str] = [] + if hasattr(user_info, "teams") and user_info.teams: + teams = user_info.teams if isinstance(user_info.teams, list) else [] + + team_details = await _fetch_cli_sso_team_details( + prisma_client=prisma_client, teams=teams + ) + attribution_metadata = build_cli_sso_attribution_metadata(result=result) + if attribution_metadata: + await _persist_cli_sso_user_metadata( + prisma_client=prisma_client, + user_id=cast(str, user_info.user_id), + attribution_metadata=attribution_metadata, + ) + + flow["session_data"] = { + "user_id": cast(str, user_info.user_id), + "user_role": user_info.user_role, + "models": user_info.models if hasattr(user_info, "models") else [], + "user_email": user_email, + "teams": teams, + "team_details": team_details, + "attribution_metadata": attribution_metadata, + } + flow["sso_complete"] = True + browser_complete_token = secrets.token_urlsafe(32) + flow["browser_complete_token_hash"] = _hash_cli_sso_secret(browser_complete_token) + _set_cli_sso_flow(login_id=key, cache=user_api_key_cache, flow=flow) + + verbose_proxy_logger.info( + f"Stored CLI SSO session for user: {user_info.user_id}, teams: {teams}, num_teams: {len(teams)}" + ) + verify_url = get_custom_url( + request_base_url=str(request.base_url), + route=f"sso/cli/complete/{key}", + ) + return HTMLResponse( + content=_render_cli_sso_verification_page( + verify_url=verify_url, + browser_complete_token=browser_complete_token, + ), + status_code=200, + ) + + async def cli_sso_callback( request: Request, key: Optional[str] = None, result: Optional[Union[OpenID, dict]] = None, + received_response: Optional[dict] = None, ): """CLI SSO callback - stores session info for JWT generation on polling""" verbose_proxy_logger.info("CLI SSO callback") from litellm.proxy.proxy_server import ( + general_settings, prisma_client, proxy_logging_obj, user_api_key_cache, @@ -1722,92 +2105,40 @@ async def cli_sso_callback( # After None check, cast to non-None type for type checker result_non_none: Union[OpenID, dict] = cast(Union[OpenID, dict], result) - parsed_openid_result = SSOAuthenticationHandler._get_user_email_and_id_from_result( - result=result_non_none - ) - verbose_proxy_logger.debug(f"parsed_openid_result: {parsed_openid_result}") - try: - # Get full user info from DB - user_info = await get_user_info_from_db( + parsed_openid_result = ( + SSOAuthenticationHandler._get_user_email_and_id_from_result( + result=result_non_none, + generic_client_id=os.getenv("GENERIC_CLIENT_ID", None), + ) + ) + verbose_proxy_logger.debug(f"parsed_openid_result: {parsed_openid_result}") + user_defined_values = await _build_cli_sso_user_defined_values( result=result_non_none, + parsed_openid_result=parsed_openid_result, + ) + + SSOAuthenticationHandler.verify_user_in_restricted_sso_group( + general_settings=general_settings, + result=result_non_none, + received_response=received_response, + ) + + return await _complete_cli_sso_callback_session( + request=request, + key=cast(str, key), + flow=flow, + result=result_non_none, + parsed_openid_result=parsed_openid_result, + user_defined_values=user_defined_values, prisma_client=prisma_client, user_api_key_cache=user_api_key_cache, proxy_logging_obj=proxy_logging_obj, - user_email=parsed_openid_result.get("user_email"), - user_defined_values=None, - alternate_user_id=parsed_openid_result.get("user_id"), ) - - if user_info is None: - raise HTTPException( - status_code=500, detail="Failed to retrieve user information from SSO" - ) - - # Get all teams from user_info - CLI will let user select which one - teams: List[str] = [] - if hasattr(user_info, "teams") and user_info.teams: - teams = user_info.teams if isinstance(user_info.teams, list) else [] - - # Also fetch team aliases for a better CLI UX. We keep the original - # "teams" list of IDs for backwards compatibility and add an - # optional "team_details" field containing objects with both - # team_id and team_alias. - team_details: List[Dict[str, Any]] = [] - try: - if teams: - prisma_teams = await prisma_client.db.litellm_teamtable.find_many( - where={"team_id": {"in": teams}} - ) - for team_row in prisma_teams: - team_dict = team_row.model_dump() - team_details.append( - { - "team_id": team_dict.get("team_id"), - "team_alias": team_dict.get("team_alias"), - } - ) - except Exception as e: - # If anything goes wrong here, fall back gracefully without - # impacting the SSO flow. - verbose_proxy_logger.error( - f"Error fetching team details for CLI SSO session: {e}" - ) - - session_data = { - "user_id": user_info.user_id, - "user_role": user_info.user_role, - "models": user_info.models if hasattr(user_info, "models") else [], - "user_email": parsed_openid_result.get("user_email"), - "teams": teams, - # Optional rich metadata for clients that want nicer display - "team_details": team_details, - } - - flow["session_data"] = session_data - flow["sso_complete"] = True - browser_complete_token = secrets.token_urlsafe(32) - flow["browser_complete_token_hash"] = _hash_cli_sso_secret( - browser_complete_token - ) - _set_cli_sso_flow(login_id=cast(str, key), cache=user_api_key_cache, flow=flow) - - verbose_proxy_logger.info( - f"Stored CLI SSO session for user: {user_info.user_id}, teams: {teams}, num_teams: {len(teams)}" - ) - - from fastapi.responses import HTMLResponse - - verify_url = get_custom_url( - request_base_url=str(request.base_url), - route=f"sso/cli/complete/{key}", - ) - html_content = _render_cli_sso_verification_page( - verify_url=verify_url, - browser_complete_token=browser_complete_token, - ) - return HTMLResponse(content=html_content, status_code=200) - + except ProxyException: + raise + except HTTPException: + raise except Exception as e: verbose_proxy_logger.error(f"Error with CLI SSO callback: {e}") raise HTTPException( @@ -1873,13 +2204,19 @@ async def cli_poll_key( team_details_response = [ {"team_id": t, "team_alias": None} for t in user_teams ] - return { + poll_response: Dict[str, Any] = { "status": "ready", "user_id": user_id, "teams": user_teams, "team_details": team_details_response, "requires_team_selection": True, } + attribution_metadata = _cli_poll_attribution_metadata_from_session( + session_data + ) + if attribution_metadata: + poll_response["attribution_metadata"] = attribution_metadata + return poll_response # Validate team_id if provided if team_id is not None: @@ -1912,7 +2249,7 @@ async def cli_poll_key( verbose_proxy_logger.info( f"CLI JWT generated for user: {user_id}, team: {team_id}" ) - return { + poll_response = { "status": "ready", "key": jwt_token, "user_id": user_id, @@ -1922,6 +2259,12 @@ async def cli_poll_key( # present nicer information if needed. "team_details": user_team_details, } + attribution_metadata = _cli_poll_attribution_metadata_from_session( + session_data + ) + if attribution_metadata: + poll_response["attribution_metadata"] = attribution_metadata + return poll_response else: return {"status": "pending"} diff --git a/tests/test_litellm/proxy/management_endpoints/test_ui_sso.py b/tests/test_litellm/proxy/management_endpoints/test_ui_sso.py index 23216542f35..a72633b726f 100644 --- a/tests/test_litellm/proxy/management_endpoints/test_ui_sso.py +++ b/tests/test_litellm/proxy/management_endpoints/test_ui_sso.py @@ -2438,6 +2438,7 @@ class TestCLIKeyRegenerationFlow: request=mock_request, key="cli-new-session-key-456", result=mock_result, + received_response=None, ) def test_get_redirect_url_does_not_include_existing_key_in_url(self): @@ -5552,6 +5553,289 @@ def test_generic_response_convertor_extra_attributes_missing_field(monkeypatch): assert result.extra_fields["another_missing"] is None +class TestCliSsoAttributionMetadata: + """CLI SSO allowlisted OIDC claim persistence and poll exposure.""" + + def test_parse_cli_sso_claim_map(self, monkeypatch): + from litellm.proxy.management_endpoints import ui_sso + + monkeypatch.setattr( + ui_sso, + "CLI_SSO_CLAIM_MAP", + "employment_type->metadata.acme_employment_type, org_info.department -> department", + ) + assert ui_sso._parse_cli_sso_claim_map() == [ + ("employment_type", "acme_employment_type"), + ("org_info.department", "department"), + ] + + def test_build_cli_sso_attribution_metadata_filters_non_scalars(self, monkeypatch): + from litellm.proxy.management_endpoints import ui_sso + from litellm.proxy.management_endpoints.types import CustomOpenID + + monkeypatch.setattr( + ui_sso, + "CLI_SSO_CLAIM_MAP", + "employment_type->acme_employment_type,access_token->should_drop,group->groups", + ) + + result = CustomOpenID( + id="user-1", + email="user@example.com", + display_name="User", + provider="generic", + team_ids=[], + extra_fields={ + "employment_type": "full_time", + "access_token": "eyJhbGciOiJIUzI1NiJ9.payload.signature", + "group": ["team-a", "team-b"], + }, + ) + + metadata = ui_sso.build_cli_sso_attribution_metadata(result=result) + assert metadata == {"acme_employment_type": "full_time"} + + def test_build_cli_sso_attribution_metadata_from_oidc_dict(self, monkeypatch): + from litellm.proxy.management_endpoints import ui_sso + + monkeypatch.setattr( + ui_sso, + "CLI_SSO_CLAIM_MAP", + "org_info.department->department", + ) + + metadata = ui_sso.build_cli_sso_attribution_metadata( + result={ + "sub": "user-1", + "email": "user@example.com", + "org_info": {"department": "Engineering"}, + } + ) + assert metadata == {"department": "Engineering"} + + @pytest.mark.asyncio + async def test_cli_sso_callback_passes_user_defined_values_for_new_users(self): + """First CLI SSO login must supply SSOUserDefinedValues so upsert can create the user.""" + from litellm.proxy._types import LiteLLM_UserTable + from litellm.proxy.management_endpoints import ui_sso + from litellm.proxy.management_endpoints.types import CustomOpenID + + mock_request = MagicMock(spec=Request) + mock_request.base_url = "http://internal-proxy.local/" + session_key = "cli-session-new-user" + mock_user_info = LiteLLM_UserTable( + user_id="cli-test-user", + user_role="internal_user", + teams=[], + models=[], + ) + mock_sso_result = CustomOpenID( + id="cli-test-user", + email="cli-test@example.com", + display_name="cli-test-user", + provider="generic", + team_ids=[], + ) + mock_cache = MagicMock() + mock_cache.get_cache.return_value = { + "poll_secret_hash": "poll-secret-hash", + "user_code_hash": "user-code-hash", + "sso_complete": False, + "user_code_verified": False, + "session_data": None, + } + get_user_info_mock = AsyncMock(return_value=mock_user_info) + + with ( + patch( + "litellm.proxy.management_endpoints.ui_sso.get_user_info_from_db", + get_user_info_mock, + ), + patch("litellm.proxy.proxy_server.prisma_client", MagicMock()), + patch("litellm.proxy.proxy_server.user_api_key_cache", mock_cache), + patch("litellm.proxy.proxy_server.user_custom_sso", None), + ): + await ui_sso.cli_sso_callback( + request=mock_request, + key=session_key, + result=mock_sso_result, + ) + + get_user_info_mock.assert_awaited_once() + assert get_user_info_mock.call_args.kwargs["user_defined_values"] is not None + assert ( + get_user_info_mock.call_args.kwargs["user_defined_values"]["user_id"] + == "cli-test-user" + ) + + @pytest.mark.asyncio + async def test_cli_sso_callback_rejects_restricted_sso_group(self): + """CLI SSO must enforce restricted_sso_group before upserting the user.""" + from litellm.proxy._types import ProxyException + from litellm.proxy.management_endpoints import ui_sso + from litellm.proxy.management_endpoints.types import CustomOpenID + + mock_request = MagicMock(spec=Request) + mock_request.base_url = "http://internal-proxy.local/" + mock_cache = MagicMock() + mock_cache.get_cache.return_value = { + "poll_secret_hash": "poll-secret-hash", + "user_code_hash": "user-code-hash", + "sso_complete": False, + "user_code_verified": False, + "session_data": None, + } + mock_sso_result = CustomOpenID( + id="cli-test-user", + email="cli-test@example.com", + display_name="cli-test-user", + provider="generic", + team_ids=["other-group"], + ) + + with ( + patch( + "litellm.proxy.management_endpoints.ui_sso.get_user_info_from_db", + new=AsyncMock(), + ) as get_user_info_mock, + patch("litellm.proxy.proxy_server.prisma_client", MagicMock()), + patch("litellm.proxy.proxy_server.user_api_key_cache", mock_cache), + patch("litellm.proxy.proxy_server.user_custom_sso", None), + patch( + "litellm.proxy.proxy_server.general_settings", + { + "ui_access_mode": { + "type": "restricted_sso_group", + "restricted_sso_group": "required-group", + } + }, + ), + ): + with pytest.raises(ProxyException): + await ui_sso.cli_sso_callback( + request=mock_request, + key="cli-session-restricted", + result=mock_sso_result, + received_response={"groups": ["other-group"]}, + ) + + get_user_info_mock.assert_not_awaited() + + @pytest.mark.asyncio + async def test_cli_sso_callback_persists_attribution_metadata(self, monkeypatch): + from litellm.proxy._types import LiteLLM_UserTable + from litellm.proxy.management_endpoints import ui_sso + + monkeypatch.setattr( + ui_sso, + "CLI_SSO_CLAIM_MAP", + "employment_type->acme_employment_type", + ) + + mock_request = MagicMock(spec=Request) + mock_request.base_url = "http://internal-proxy.local/" + session_key = "cli-session-4567890" + mock_user_info = LiteLLM_UserTable( + user_id="test-user-123", + user_role="internal_user", + teams=["team1"], + models=["gpt-4"], + ) + mock_sso_result = { + "user_email": "test@example.com", + "user_id": "test-user-123", + "employment_type": "contractor", + } + mock_cache = MagicMock() + mock_cache.get_cache.return_value = { + "poll_secret_hash": "poll-secret-hash", + "user_code_hash": "user-code-hash", + "sso_complete": False, + "user_code_verified": False, + "session_data": None, + } + mock_prisma = MagicMock() + mock_prisma.db.litellm_usertable.find_unique = AsyncMock( + return_value=MagicMock(metadata={"auth_provider": "generic"}) + ) + mock_prisma.db.litellm_usertable.update_many = AsyncMock() + + with ( + patch.dict( + os.environ, + { + "PROXY_BASE_URL": "https://test.example.com", + "SERVER_ROOT_PATH": "", + }, + ), + patch( + "litellm.proxy.management_endpoints.ui_sso.get_user_info_from_db", + return_value=mock_user_info, + ), + patch("litellm.proxy.proxy_server.prisma_client", mock_prisma), + patch("litellm.proxy.proxy_server.user_api_key_cache", mock_cache), + patch("litellm.proxy.proxy_server.user_custom_sso", None), + patch( + "litellm.proxy.common_utils.html_forms.cli_sso_success.render_cli_sso_success_page", + return_value="Success", + ), + ): + await ui_sso.cli_sso_callback( + request=mock_request, + key=session_key, + result=mock_sso_result, + ) + + flow_data = mock_cache.set_cache.call_args.kwargs["value"] + assert flow_data["session_data"]["attribution_metadata"] == { + "acme_employment_type": "contractor" + } + mock_prisma.db.litellm_usertable.update_many.assert_awaited_once() + update_data = mock_prisma.db.litellm_usertable.update_many.call_args.kwargs[ + "data" + ] + assert update_data["metadata"]["acme_employment_type"] == "contractor" + assert update_data["metadata"]["auth_provider"] == "generic" + + @pytest.mark.asyncio + async def test_cli_poll_key_returns_attribution_metadata(self, monkeypatch): + from litellm.proxy.management_endpoints.ui_sso import ( + _hash_cli_sso_secret, + cli_poll_key, + ) + + session_key = "cli-session-789123" + session_data = { + "user_id": "test-user-456", + "user_role": "internal_user", + "teams": ["team-a", "team-b"], + "models": ["gpt-4"], + "attribution_metadata": { + "acme_employment_type": "full_time", + "org": {"cost_center": "CC-42"}, + }, + } + mock_cache = MagicMock() + mock_cache.get_cache.return_value = { + "poll_secret_hash": _hash_cli_sso_secret("poll-secret"), + "sso_complete": True, + "user_code_verified": True, + "session_data": session_data, + } + + with patch("litellm.proxy.proxy_server.user_api_key_cache", mock_cache): + result = await cli_poll_key( + key_id=session_key, + team_id=None, + x_litellm_cli_poll_secret="poll-secret", + ) + + assert result["attribution_metadata"] == { + "acme_employment_type": "full_time", + "org.cost_center": "CC-42", + } + + class TestValidateReturnTo: """Tests for SSOAuthenticationHandler._validate_return_to""" From 21a21e01f7e5793032d131dfb72796bd4bf1b8c9 Mon Sep 17 00:00:00 2001 From: Sameer Kankute Date: Fri, 22 May 2026 22:33:36 +0530 Subject: [PATCH 02/13] fix(responses): use OpenAI SSEDecoder for Responses API streaming (#28566) * fix(responses): use OpenAI SSEDecoder for Responses API streaming httpx aiter_lines() uses str.splitlines(), which splits on U+2028 inside JSON payloads and silently drops response.completed (no spend log). Use openai._streaming.SSEDecoder (bytes.splitlines before decode) instead. Co-authored-by: Cursor * fix(responses): drop redundant SSE prefix strip after SSEDecoder switch SSEDecoder already strips the 'data:' field prefix from each event, so the extra call to _strip_sse_data_from_chunk on sse.data was redundant and could incorrectly mangle payloads whose actual content starts with 'data:'. Co-authored-by: Yassin Kortam --------- Co-authored-by: Cursor Co-authored-by: Yassin Kortam --- litellm/responses/streaming_iterator.py | 23 +++--- ...t_base_responses_api_streaming_iterator.py | 78 +++++++++++++++---- 2 files changed, 77 insertions(+), 24 deletions(-) diff --git a/litellm/responses/streaming_iterator.py b/litellm/responses/streaming_iterator.py index da8da1b486f..c4e72cb7dc5 100644 --- a/litellm/responses/streaming_iterator.py +++ b/litellm/responses/streaming_iterator.py @@ -9,6 +9,7 @@ from functools import lru_cache from typing import Any, Dict, List, Literal, Optional import httpx +from openai._streaming import SSEDecoder import litellm from litellm.constants import ( @@ -27,7 +28,7 @@ from litellm.llms.base_llm.responses.transformation import BaseResponsesAPIConfi from litellm.responses.utils import ResponsesAPIRequestUtils from litellm.types.llms.openai import ResponsesAPIStreamEvents from litellm.types.utils import CallTypes -from litellm.utils import CustomStreamWrapper, async_post_call_success_deployment_hook +from litellm.utils import async_post_call_success_deployment_hook @lru_cache(maxsize=1) @@ -120,10 +121,10 @@ class BaseResponsesAPIStreamingIterator: if not chunk: return None - # Handle SSE format (data: {...}) - chunk = CustomStreamWrapper._strip_sse_data_from_chunk(chunk) - if chunk is None: - return None + # NOTE: ``SSEDecoder`` already strips the SSE ``data:`` field prefix, so + # the value passed in here is the raw field content. Do not re-run + # ``_strip_sse_data_from_chunk`` on it — doing so would incorrectly mangle + # payloads whose actual JSON value happens to start with ``data:``. # Handle "[DONE]" marker if chunk == STREAM_SSE_DONE_STRING: @@ -634,7 +635,7 @@ class ResponsesAPIStreamingIterator(BaseResponsesAPIStreamingIterator): request_data, call_type, ) - self.stream_iterator = response.aiter_lines() + self.stream_iterator = SSEDecoder().aiter_bytes(response.aiter_bytes()) def __aiter__(self): return self @@ -645,13 +646,13 @@ class ResponsesAPIStreamingIterator(BaseResponsesAPIStreamingIterator): while True: # Get the next chunk from the stream try: - chunk = await self.stream_iterator.__anext__() + sse = await self.stream_iterator.__anext__() except StopAsyncIteration: self.finished = True raise StopAsyncIteration self._check_max_streaming_duration() - result = self._process_chunk(chunk) + result = self._process_chunk(sse.data) if self.finished: raise StopAsyncIteration @@ -708,7 +709,7 @@ class SyncResponsesAPIStreamingIterator(BaseResponsesAPIStreamingIterator): request_data, call_type, ) - self.stream_iterator = response.iter_lines() + self.stream_iterator = SSEDecoder().iter_bytes(response.iter_bytes()) def __iter__(self): return self @@ -719,13 +720,13 @@ class SyncResponsesAPIStreamingIterator(BaseResponsesAPIStreamingIterator): while True: # Get the next chunk from the stream try: - chunk = next(self.stream_iterator) + sse = next(self.stream_iterator) except StopIteration: self.finished = True raise StopIteration self._check_max_streaming_duration() - result = self._process_chunk(chunk) + result = self._process_chunk(sse.data) if self.finished: raise StopIteration diff --git a/tests/llm_responses_api_testing/test_base_responses_api_streaming_iterator.py b/tests/llm_responses_api_testing/test_base_responses_api_streaming_iterator.py index e2c50810cc2..37fcc602d37 100644 --- a/tests/llm_responses_api_testing/test_base_responses_api_streaming_iterator.py +++ b/tests/llm_responses_api_testing/test_base_responses_api_streaming_iterator.py @@ -41,6 +41,62 @@ from litellm.types.llms.openai import ( class TestBaseResponsesAPIStreamingIterator: """Test cases for BaseResponsesAPIStreamingIterator""" + @pytest.mark.asyncio + async def test_responses_streaming_iterator_parses_u2028_in_sse_json(self): + """ + U+2028 inside JSON must not split the SSE event. httpx aiter_lines uses + str.splitlines() and drops response.completed; OpenAI SSEDecoder does not. + """ + from litellm.responses.streaming_iterator import ResponsesAPIStreamingIterator + + u2028 = "\u2028" + payload = json.dumps( + { + "type": "response.completed", + "response": {"instructions": f"eligible{u2028}promo"}, + } + ) + sse_bytes = f"data: {payload}\n\n".encode("utf-8") + + async def mock_aiter_bytes(): + yield sse_bytes + + mock_response = Mock() + mock_response.headers = {} + mock_response.aiter_bytes = mock_aiter_bytes + + mock_logging_obj = Mock(spec=LiteLLMLoggingObj) + mock_logging_obj.model_call_details = {"litellm_params": {}} + mock_config = Mock(spec=BaseResponsesAPIConfig) + + mock_responses_api_response = Mock(spec=ResponsesAPIResponse) + mock_responses_api_response.id = "resp_u2028" + mock_completed_event = Mock(spec=ResponseCompletedEvent) + mock_completed_event.type = ResponsesAPIStreamEvents.RESPONSE_COMPLETED + mock_completed_event.response = mock_responses_api_response + mock_config.transform_streaming_response.return_value = mock_completed_event + + iterator = ResponsesAPIStreamingIterator( + response=mock_response, + model="gpt-5.5", + responses_api_provider_config=mock_config, + logging_obj=mock_logging_obj, + litellm_metadata={"model_info": {"id": "model_123"}}, + custom_llm_provider="openai", + ) + + chunks = [] + with ( + patch("asyncio.create_task"), + patch("litellm.responses.streaming_iterator.executor"), + ): + async for chunk in iterator: + chunks.append(chunk) + + assert len(chunks) == 1 + assert chunks[0].type == ResponsesAPIStreamEvents.RESPONSE_COMPLETED + assert iterator.completed_response is not None + def test_process_chunk_with_response_completed_event(self): """ Test that _process_chunk correctly processes a ResponseCompletedEvent @@ -270,7 +326,7 @@ class TestBaseResponsesAPIStreamingIterator: # Mock dependencies mock_response = Mock() mock_response.headers = {} - mock_response.aiter_lines = Mock() + mock_response.aiter_bytes = Mock() mock_logging_obj = Mock(spec=LiteLLMLoggingObj) mock_logging_obj.model_call_details = {"litellm_params": {}} mock_logging_obj.async_success_handler = Mock() @@ -334,12 +390,10 @@ class TestBaseResponsesAPIStreamingIterator: mock_response = Mock() mock_response.headers = {} - # Create an async iterator that raises StopAsyncIteration after yielding one chunk - async def mock_aiter_lines(): - yield 'data: {"type": "response.output_text.delta", "delta": "test"}' - # Normal end of stream - raise StopAsyncIteration + async def mock_aiter_bytes(): + yield b'data: {"type": "response.output_text.delta", "delta": "test"}\n\n' - mock_response.aiter_lines = mock_aiter_lines + mock_response.aiter_bytes = mock_aiter_bytes mock_logging_obj = Mock(spec=LiteLLMLoggingObj) mock_logging_obj.model_call_details = {"litellm_params": {}} @@ -396,12 +450,10 @@ class TestBaseResponsesAPIStreamingIterator: mock_response = Mock() mock_response.headers = {} - # Create a sync iterator that raises StopIteration after yielding one chunk - def mock_iter_lines(): - yield 'data: {"type": "response.output_text.delta", "delta": "test"}' - # Normal end of stream - raise StopIteration + def mock_iter_bytes(): + yield b'data: {"type": "response.output_text.delta", "delta": "test"}\n\n' - mock_response.iter_lines = mock_iter_lines + mock_response.iter_bytes = mock_iter_bytes mock_logging_obj = Mock(spec=LiteLLMLoggingObj) mock_logging_obj.model_call_details = {"litellm_params": {}} @@ -450,7 +502,7 @@ class TestBaseResponsesAPIStreamingIterator: mock_response = Mock() mock_response.headers = {} - mock_response.aiter_lines = Mock() + mock_response.aiter_bytes = Mock() mock_logging_obj = Mock(spec=LiteLLMLoggingObj) mock_logging_obj.model_call_details = {"litellm_params": {}} mock_logging_obj.async_failure_handler = Mock() @@ -532,7 +584,7 @@ class TestBaseResponsesAPIStreamingIterator: mock_response = Mock() mock_response.headers = {} - mock_response.aiter_lines = Mock() + mock_response.aiter_bytes = Mock() mock_logging_obj = Mock(spec=LiteLLMLoggingObj) mock_logging_obj.model_call_details = {"litellm_params": {}} mock_logging_obj.async_failure_handler = Mock() From e9f0eddbd1d8f0e4053aaf822ff023e5711563b1 Mon Sep 17 00:00:00 2001 From: Sameer Kankute Date: Fri, 22 May 2026 22:34:23 +0530 Subject: [PATCH 03/13] Litellm oss staging 2 (#28582) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * fix(anthropic): handle empty streaming tool calls (#28549) Co-authored-by: shin-berri Co-authored-by: yuneng-jiang * [Feature][Bug Fix] Decouple Azure OpenAI Deployment ID from model name via base_model to fix gpt5 model routing (#28490) * feat(azure): decouple deployment ID from model name via base_model Azure OpenAI deployments have arbitrary names (deployment IDs) that may not match the underlying model. Previously, model-type detection (o-series, gpt-5, etc.) relied on substring matching against the deployment name, causing misrouted configs and rejected params when deployment names were non-standard (e.g. 'my-deployment-id' for gpt-5.2). This change extends the existing base_model field to drive model-type detection, config selection, supported param resolution, and param mapping throughout the Azure call path: - _get_azure_config() uses base_model for is_o_series/is_gpt_5 checks - get_provider_chat_config() threads base_model for Azure - get_supported_openai_params() accepts and uses base_model - get_optional_params() accepts base_model and passes it to all Azure config method calls (get_supported_openai_params, map_openai_params) - azure.py completion handler uses base_model for GPT-5 detection - Config internal methods (e.g. is_model_gpt_5_2_model) now receive base_model so features like logprobs are correctly enabled Fully backward compatible - when base_model is unset, behavior is identical. Existing o_series/ and gpt5_series/ prefix workarounds continue to work. Usage in proxy config: model_list: - model_name: my-gpt5 litellm_params: model: azure/my-deployment-id model_info: base_model: azure/gpt-5.2 Fixes: non-standard deployment names like 'prefix-gpt-5.2' rejecting logprobs/top_logprobs despite the underlying model supporting them. * Addressing Greptile comments. * gemini-3.1-flash-lite pricing (#27933) * feat(model_prices): add gemini-3.1-flash-lite pricing with standard/batch/flex/priority tiers * fix pricing * add service tier --------- Co-authored-by: shin-berri * fix(openai-responses): strip Anthropic cache_control from Responses API requests (#28431) Squash-merged by litellm-agent from cwang-otto's PR. * Treat None litellm_provider as wildcard in _check_provider_match (#28523) Squash-merged by litellm-agent from adityasingh2400's PR. * fix greptile * fix: use _azure_detection_model in default Azure branch of get_supported_openai_params Co-authored-by: Yassin Kortam * fix(openai-responses): strip cache_control on compact endpoint as well Co-authored-by: Yassin Kortam --------- Co-authored-by: Felipe Garé <90070734+FelipeRodriguesGare@users.noreply.github.com> Co-authored-by: shin-berri Co-authored-by: yuneng-jiang Co-authored-by: withomasmicrosoft Co-authored-by: mubashir1osmani Co-authored-by: cwang-otto Co-authored-by: Aditya Singh <60082699+adityasingh2400@users.noreply.github.com> Co-authored-by: Cursor Agent Co-authored-by: Yassin Kortam --- .../get_supported_openai_params.py | 33 ++- .../adapters/transformation.py | 2 +- litellm/llms/azure/azure.py | 4 +- .../llms/openai/responses/transformation.py | 52 +++- litellm/main.py | 17 +- litellm/utils.py | 76 +++-- model_prices_and_context_window.json | 67 +++-- ...al_pass_through_adapters_transformation.py | 45 +++ .../chat/test_azure_base_model_routing.py | 274 ++++++++++++++++++ .../test_openai_responses_transformation.py | 84 ++++++ .../test_register_model_custom_pricing.py | 161 ++++++++++ tests/test_litellm/test_utils.py | 28 ++ 12 files changed, 792 insertions(+), 51 deletions(-) create mode 100644 tests/test_litellm/llms/azure/chat/test_azure_base_model_routing.py diff --git a/litellm/litellm_core_utils/get_supported_openai_params.py b/litellm/litellm_core_utils/get_supported_openai_params.py index 9d8bd7523db..b8cdc8210fc 100644 --- a/litellm/litellm_core_utils/get_supported_openai_params.py +++ b/litellm/litellm_core_utils/get_supported_openai_params.py @@ -11,6 +11,7 @@ def get_supported_openai_params( # noqa: PLR0915 request_type: Literal[ "chat_completion", "embeddings", "transcription" ] = "chat_completion", + base_model: Optional[str] = None, ) -> Optional[list]: """ Returns the supported openai params for a given model + provider @@ -20,6 +21,11 @@ def get_supported_openai_params( # noqa: PLR0915 get_supported_openai_params(model="anthropic.claude-3", custom_llm_provider="bedrock") ``` + Args: + base_model: For Azure, the true underlying model (e.g. ``"azure/gpt-5.2"``) + when the deployment name differs. Used for model-type detection so that + non-standard deployment names route to the correct config. + Returns: - List if custom_llm_provider is mapped - None if unmapped @@ -32,17 +38,21 @@ def get_supported_openai_params( # noqa: PLR0915 if custom_llm_provider in LlmProvidersSet: provider_config = litellm.ProviderConfigManager.get_provider_chat_config( - model=model, provider=LlmProviders(custom_llm_provider) + model=model, + provider=LlmProviders(custom_llm_provider), + base_model=base_model, ) elif custom_llm_provider.split("/")[0] in LlmProvidersSet: provider_config = litellm.ProviderConfigManager.get_provider_chat_config( - model=model, provider=LlmProviders(custom_llm_provider.split("/")[0]) + model=model, + provider=LlmProviders(custom_llm_provider.split("/")[0]), + base_model=base_model, ) else: provider_config = None if provider_config and request_type == "chat_completion": - return provider_config.get_supported_openai_params(model=model) + return provider_config.get_supported_openai_params(model=base_model or model) if custom_llm_provider == "bedrock": return litellm.AmazonConverseConfig().get_supported_openai_params(model=model) @@ -130,16 +140,23 @@ def get_supported_openai_params( # noqa: PLR0915 model=model ) elif custom_llm_provider == "azure": - if litellm.AzureOpenAIO1Config().is_o_series_model(model=model): + _azure_detection_model = base_model or model + if litellm.AzureOpenAIO1Config().is_o_series_model( + model=_azure_detection_model + ): return litellm.AzureOpenAIO1Config().get_supported_openai_params( - model=model + model=_azure_detection_model ) - elif litellm.AzureOpenAIGPT5Config.is_model_gpt_5_model(model=model): + elif litellm.AzureOpenAIGPT5Config.is_model_gpt_5_model( + model=_azure_detection_model + ): return litellm.AzureOpenAIGPT5Config().get_supported_openai_params( - model=model + model=_azure_detection_model ) else: - return litellm.AzureOpenAIConfig().get_supported_openai_params(model=model) + return litellm.AzureOpenAIConfig().get_supported_openai_params( + model=_azure_detection_model + ) elif custom_llm_provider == "openrouter": return litellm.OpenrouterConfig().get_supported_openai_params(model=model) elif custom_llm_provider == "vercel_ai_gateway": diff --git a/litellm/llms/anthropic/experimental_pass_through/adapters/transformation.py b/litellm/llms/anthropic/experimental_pass_through/adapters/transformation.py index 0e198daf089..51a1e739a0f 100644 --- a/litellm/llms/anthropic/experimental_pass_through/adapters/transformation.py +++ b/litellm/llms/anthropic/experimental_pass_through/adapters/transformation.py @@ -1476,7 +1476,7 @@ class LiteLLMAnthropicMessagesAdapter: for choice in choices: if choice.delta.content is not None and len(choice.delta.content) > 0: text += choice.delta.content - if choice.delta.tool_calls is not None: + if choice.delta.tool_calls: partial_json = "" for tool in choice.delta.tool_calls: if ( diff --git a/litellm/llms/azure/azure.py b/litellm/llms/azure/azure.py index 9291269d153..734b8ecef16 100644 --- a/litellm/llms/azure/azure.py +++ b/litellm/llms/azure/azure.py @@ -239,7 +239,9 @@ class AzureChatCompletion(BaseAzureLLM, BaseLLM): ) data = {"model": None, "messages": messages, **optional_params} - elif litellm.AzureOpenAIGPT5Config.is_model_gpt_5_model(model=model): + elif litellm.AzureOpenAIGPT5Config.is_model_gpt_5_model( + model=litellm_params.get("base_model") or model + ): data = litellm.AzureOpenAIGPT5Config().transform_request( model=model, messages=messages, diff --git a/litellm/llms/openai/responses/transformation.py b/litellm/llms/openai/responses/transformation.py index b7d5340d8d4..5043d25ee37 100644 --- a/litellm/llms/openai/responses/transformation.py +++ b/litellm/llms/openai/responses/transformation.py @@ -126,9 +126,21 @@ class OpenAIResponsesAPIConfig(BaseResponsesAPIConfig): litellm_params: GenericLiteLLMParams, headers: dict, ) -> Dict: - """No transform applied since inputs are in OpenAI spec already""" + """Strip Anthropic-only `cache_control` markers before sending to OpenAI. + + OpenAI's Responses API rejects unknown fields on input content blocks + with HTTP 400 ("Unknown parameter: 'input[0].content[0].cache_control'"). + Chat Completions strips these in + `remove_cache_control_flag_from_messages_and_tools`; mirror that here. + """ input = self._validate_input_param(input) + tools = response_api_optional_request_params.get("tools") + input, tools = self.remove_cache_control_flag_from_input_and_tools( + model=model, input=input, tools=tools + ) + if tools is not None: + response_api_optional_request_params["tools"] = tools final_request_params = dict( ResponsesAPIRequestParams( model=model, input=input, **response_api_optional_request_params @@ -137,6 +149,38 @@ class OpenAIResponsesAPIConfig(BaseResponsesAPIConfig): return final_request_params + def remove_cache_control_flag_from_input_and_tools( + self, + model: str, # allows overrides to selectively run this + input: Union[str, ResponseInputParam], + tools: Optional[List[ALL_RESPONSES_API_TOOL_PARAMS]] = None, + ) -> Tuple[ + Union[str, ResponseInputParam], + Optional[List[ALL_RESPONSES_API_TOOL_PARAMS]], + ]: + """Sibling of `remove_cache_control_flag_from_messages_and_tools` on + the chat path. Strips Anthropic-only `cache_control` markers from + Responses API input content blocks and tools. + + `filter_value_from_dict` mutates each dict in place, so the same + objects are returned. + """ + from litellm.litellm_core_utils.prompt_templates.common_utils import ( + filter_value_from_dict, + ) + + if isinstance(input, list): + for item in input: + if isinstance(item, dict): + filter_value_from_dict(cast(dict, item), "cache_control") + + if tools is not None: + for tool in tools: + if isinstance(tool, dict): + filter_value_from_dict(cast(dict, tool), "cache_control") + + return input, tools + def _validate_input_param( self, input: Union[str, ResponseInputParam] ) -> Union[str, ResponseInputParam]: @@ -604,6 +648,12 @@ class OpenAIResponsesAPIConfig(BaseResponsesAPIConfig): url = str(parsed_url.copy_with(path=compact_path)) input = self._validate_input_param(input) + tools = response_api_optional_request_params.get("tools") + input, tools = self.remove_cache_control_flag_from_input_and_tools( + model=model, input=input, tools=tools + ) + if tools is not None: + response_api_optional_request_params["tools"] = tools data = dict( ResponsesAPIRequestParams( model=model, input=input, **response_api_optional_request_params diff --git a/litellm/main.py b/litellm/main.py index b5364f8ba17..e17a5ad9a48 100644 --- a/litellm/main.py +++ b/litellm/main.py @@ -1491,7 +1491,9 @@ def completion( # type: ignore # noqa: PLR0915 provider.value for provider in LlmProviders ]: provider_config = ProviderConfigManager.get_provider_chat_config( - model=model, provider=LlmProviders(custom_llm_provider) + model=model, + provider=LlmProviders(custom_llm_provider), + base_model=base_model, ) if provider_config is not None: @@ -1550,6 +1552,7 @@ def completion( # type: ignore # noqa: PLR0915 "safety_identifier": safety_identifier, "service_tier": service_tier, "allowed_openai_params": kwargs.get("allowed_openai_params"), + "base_model": base_model, } optional_params = get_optional_params( **optional_param_args, **non_default_params @@ -1670,6 +1673,10 @@ def completion( # type: ignore # noqa: PLR0915 reasoning_summary=_reasoning_summary_for_bridge, ) + # Use base_model (the true underlying model) for Azure model-type + # detection when the deployment name differs from the model name. + _azure_detection_model = base_model or model + if responses_api_model_info.get("mode") == "responses": from litellm.completion_extras import responses_api_bridge @@ -1713,7 +1720,9 @@ def completion( # type: ignore # noqa: PLR0915 and OpenAIGPT5Config.is_model_gpt_5_model(model) ) or ( custom_llm_provider == "azure" - and litellm.AzureOpenAIGPT5Config.is_model_gpt_5_model(model) + and litellm.AzureOpenAIGPT5Config.is_model_gpt_5_model( + _azure_detection_model + ) ): optional_params, _ = strip_reasoning_summary_aliases_from_optional_params( optional_params @@ -1766,7 +1775,9 @@ def completion( # type: ignore # noqa: PLR0915 if max_retries is not None: optional_params["max_retries"] = max_retries - if litellm.AzureOpenAIO1Config().is_o_series_model(model=model): + if litellm.AzureOpenAIO1Config().is_o_series_model( + model=_azure_detection_model + ): ## LOAD CONFIG - if set config = litellm.AzureOpenAIO1Config.get_config() for k, v in config.items(): diff --git a/litellm/utils.py b/litellm/utils.py index 2487d39bd0d..18ee811f0f1 100644 --- a/litellm/utils.py +++ b/litellm/utils.py @@ -2933,6 +2933,13 @@ def register_model(model_cost: Union[str, dict]): # noqa: PLR0915 except Exception: existing_model = {} model_cost_key = key + # ``get_model_info`` returns ``litellm_provider: None`` when the + # provider is unknown (e.g. custom deployments registered via + # ``Router.add_deployment``). Persisting that None into + # ``litellm.model_cost`` causes ``_check_provider_match`` to drop + # custom pricing on subsequent cost lookups. + if existing_model.get("litellm_provider") is None: + existing_model.pop("litellm_provider", None) ## override / add new keys to the existing model cost dictionary updated_dictionary = _update_dictionary(existing_model, value) litellm.model_cost.setdefault(model_cost_key, {}).update(updated_dictionary) @@ -4019,16 +4026,23 @@ def get_optional_params( # noqa: PLR0915 thinking: Optional[AnthropicThinkingParam] = None, web_search_options: Optional[OpenAIWebSearchOptions] = None, safety_identifier: Optional[str] = None, + base_model: Optional[str] = None, **kwargs, ): passed_params = locals().copy() special_params = passed_params.pop("kwargs") + # Remove base_model from passed_params so it doesn't interfere with + # non_default_params / _check_valid_arg — it's a routing hint, not an + # OpenAI param. + passed_params.pop("base_model", None) provider_config: Optional[BaseConfig] = None if custom_llm_provider is not None and custom_llm_provider in [ provider.value for provider in LlmProviders ]: provider_config = ProviderConfigManager.get_provider_chat_config( - model=model, provider=LlmProviders(custom_llm_provider) + model=model, + provider=LlmProviders(custom_llm_provider), + base_model=base_model, ) non_default_params = pre_process_non_default_params( passed_params=passed_params, @@ -4091,7 +4105,7 @@ def get_optional_params( # noqa: PLR0915 sys.modules[__name__], "get_supported_openai_params" ) supported_params = get_supported_openai_params( - model=model, custom_llm_provider=custom_llm_provider + model=model, custom_llm_provider=custom_llm_provider, base_model=base_model ) if supported_params is None: supported_params = get_supported_openai_params( @@ -4702,22 +4716,27 @@ def get_optional_params( # noqa: PLR0915 ), ) elif custom_llm_provider == "azure": - if litellm.AzureOpenAIO1Config().is_o_series_model(model=model): + _azure_detection_model = base_model or model + if litellm.AzureOpenAIO1Config().is_o_series_model( + model=_azure_detection_model + ): optional_params = litellm.AzureOpenAIO1Config().map_openai_params( non_default_params=non_default_params, optional_params=optional_params, - model=model, + model=_azure_detection_model, drop_params=( drop_params if drop_params is not None and isinstance(drop_params, bool) else False ), ) - elif litellm.AzureOpenAIGPT5Config.is_model_gpt_5_model(model=model): + elif litellm.AzureOpenAIGPT5Config.is_model_gpt_5_model( + model=_azure_detection_model + ): optional_params = litellm.AzureOpenAIGPT5Config().map_openai_params( non_default_params=non_default_params, optional_params=optional_params, - model=model, + model=_azure_detection_model, drop_params=( drop_params if drop_params is not None and isinstance(drop_params, bool) @@ -4739,7 +4758,7 @@ def get_optional_params( # noqa: PLR0915 optional_params = litellm.AzureOpenAIConfig().map_openai_params( non_default_params=non_default_params, optional_params=optional_params, - model=model, + model=_azure_detection_model, api_version=api_version, # type: ignore drop_params=( drop_params @@ -5510,9 +5529,15 @@ def _get_model_info_from_model_cost(key: str) -> dict: def _check_provider_match(model_info: dict, custom_llm_provider: Optional[str]) -> bool: """ Check if the model info provider matches the custom provider. + + A missing ``litellm_provider`` key and a ``litellm_provider`` set to + ``None`` both mean "no specific provider constraint" and are treated + as a wildcard match. ``register_model`` may persist ``None`` here via + ``get_model_info`` when a deployment is registered without a provider, + so normalising the two cases keeps custom pricing applied consistently. """ if custom_llm_provider and ( - "litellm_provider" in model_info + model_info.get("litellm_provider") is not None and model_info["litellm_provider"] != custom_llm_provider ): if custom_llm_provider == "vertex_ai" and model_info[ @@ -8124,10 +8149,8 @@ class ProviderConfigManager: # Format: (factory_function, needs_model_parameter: bool) LlmProviders.OPENAI: (lambda: litellm.OpenAIGPTConfig(), False), LlmProviders.ANTHROPIC: (lambda: litellm.AnthropicConfig(), False), - LlmProviders.AZURE: ( - lambda model: ProviderConfigManager._get_azure_config(model), - True, - ), + # AZURE is handled as a special case in get_provider_chat_config() + # so that base_model can be threaded through for model-type detection. LlmProviders.AZURE_AI: ( lambda model: ProviderConfigManager._get_azure_ai_config(model), True, @@ -8267,11 +8290,19 @@ class ProviderConfigManager: } @staticmethod - def _get_azure_config(model: str) -> BaseConfig: - """Get Azure config based on model type.""" - if litellm.AzureOpenAIO1Config().is_o_series_model(model=model): + def _get_azure_config(model: str, base_model: Optional[str] = None) -> BaseConfig: + """Get Azure config based on model type. + + When *base_model* is provided (e.g. ``"azure/gpt-5.2"``), it is used + for model-type detection instead of *model* (the deployment name). + This allows non-standard deployment names like ``"azure/foo"`` to be + routed through the correct config when the user specifies the true + underlying model via ``base_model``. + """ + detection_model = base_model or model + if litellm.AzureOpenAIO1Config().is_o_series_model(model=detection_model): return litellm.AzureOpenAIO1Config() - if litellm.AzureOpenAIGPT5Config.is_model_gpt_5_model(model=model): + if litellm.AzureOpenAIGPT5Config.is_model_gpt_5_model(model=detection_model): return litellm.AzureOpenAIGPT5Config() return litellm.AzureOpenAIConfig() @@ -8329,13 +8360,18 @@ class ProviderConfigManager: @staticmethod def get_provider_chat_config( # noqa: PLR0915 - model: str, provider: LlmProviders + model: str, + provider: LlmProviders, + base_model: Optional[str] = None, ) -> Optional[BaseConfig]: """ Returns the provider config for a given provider. Uses O(1) dictionary lookup for fast provider resolution. Python classes take priority over JSON (they have custom overrides). + + For Azure, *base_model* (when set) drives model-type detection so that + non-standard deployment names still route to the correct config. """ # Handle OpenAI special cases (O-series and GPT-5 models) if provider == LlmProviders.OPENAI: @@ -8344,6 +8380,12 @@ class ProviderConfigManager: if litellm.OpenAIGPT5Config.is_model_gpt_5_model(model=model): return litellm.OpenAIGPT5Config() + # Handle Azure before the generic map so base_model can be threaded through + if provider == LlmProviders.AZURE: + return ProviderConfigManager._get_azure_config( + model=model, base_model=base_model + ) + # Initialize provider config map lazily (avoids circular imports) if ProviderConfigManager._PROVIDER_CONFIG_MAP is None: ProviderConfigManager._PROVIDER_CONFIG_MAP = ( diff --git a/model_prices_and_context_window.json b/model_prices_and_context_window.json index 31a5993a240..2140493ec4a 100644 --- a/model_prices_and_context_window.json +++ b/model_prices_and_context_window.json @@ -15006,10 +15006,16 @@ "supports_service_tier": true }, "gemini-3.1-flash-lite": { - "cache_read_input_token_cost": 4.5e-08, - "cache_read_input_token_cost_per_audio_token": 9e-08, - "input_cost_per_audio_token": 9e-07, - "input_cost_per_token": 4.5e-07, + "cache_read_input_token_cost": 2.5e-08, + "cache_read_input_token_cost_batches": 1.25e-08, + "cache_read_input_token_cost_flex": 1.25e-08, + "cache_read_input_token_cost_per_audio_token": 5e-08, + "cache_read_input_token_cost_priority": 4.5e-08, + "input_cost_per_audio_token": 5e-07, + "input_cost_per_token": 2.5e-07, + "input_cost_per_token_batches": 1.25e-07, + "input_cost_per_token_flex": 1.25e-07, + "input_cost_per_token_priority": 4.5e-07, "litellm_provider": "vertex_ai-language-models", "max_audio_length_hours": 8.4, "max_audio_per_prompt": 1, @@ -15021,9 +15027,12 @@ "max_video_length": 1, "max_videos_per_prompt": 10, "mode": "chat", - "output_cost_per_reasoning_token": 2.7e-06, - "output_cost_per_token": 2.7e-06, - "source": "https://cloud.google.com/vertex-ai/generative-ai/pricing#gemini-models", + "output_cost_per_reasoning_token": 1.5e-06, + "output_cost_per_token": 1.5e-06, + "output_cost_per_token_batches": 7.5e-07, + "output_cost_per_token_flex": 7.5e-07, + "output_cost_per_token_priority": 2.7e-06, + "source": "https://ai.google.dev/gemini-api/docs/pricing#gemini-3.1-flash-lite", "supported_endpoints": [ "/v1/chat/completions", "/v1/completions", @@ -17128,10 +17137,16 @@ "supports_service_tier": true }, "gemini/gemini-3.1-flash-lite": { - "cache_read_input_token_cost": 4.5e-08, - "cache_read_input_token_cost_per_audio_token": 9e-08, - "input_cost_per_audio_token": 9e-07, - "input_cost_per_token": 4.5e-07, + "cache_read_input_token_cost": 2.5e-08, + "cache_read_input_token_cost_batches": 1.25e-08, + "cache_read_input_token_cost_flex": 1.25e-08, + "cache_read_input_token_cost_per_audio_token": 5e-08, + "cache_read_input_token_cost_priority": 4.5e-08, + "input_cost_per_audio_token": 5e-07, + "input_cost_per_token": 2.5e-07, + "input_cost_per_token_batches": 1.25e-07, + "input_cost_per_token_flex": 1.25e-07, + "input_cost_per_token_priority": 4.5e-07, "litellm_provider": "gemini", "max_audio_length_hours": 8.4, "max_audio_per_prompt": 1, @@ -17143,10 +17158,13 @@ "max_video_length": 1, "max_videos_per_prompt": 10, "mode": "chat", - "output_cost_per_reasoning_token": 2.7e-06, - "output_cost_per_token": 2.7e-06, + "output_cost_per_reasoning_token": 1.5e-06, + "output_cost_per_token": 1.5e-06, + "output_cost_per_token_batches": 7.5e-07, + "output_cost_per_token_flex": 7.5e-07, + "output_cost_per_token_priority": 2.7e-06, "rpm": 15, - "source": "https://ai.google.dev/gemini-api/docs/pricing", + "source": "https://ai.google.dev/gemini-api/docs/pricing#gemini-3.1-flash-lite", "supported_endpoints": [ "/v1/chat/completions", "/v1/completions", @@ -33932,10 +33950,16 @@ "web_search_billing_unit": "per_query" }, "vertex_ai/gemini-3.1-flash-lite": { - "cache_read_input_token_cost": 4.5e-08, - "cache_read_input_token_cost_per_audio_token": 9e-08, - "input_cost_per_audio_token": 9e-07, - "input_cost_per_token": 4.5e-07, + "cache_read_input_token_cost": 2.5e-08, + "cache_read_input_token_cost_batches": 1.25e-08, + "cache_read_input_token_cost_flex": 1.25e-08, + "cache_read_input_token_cost_per_audio_token": 5e-08, + "cache_read_input_token_cost_priority": 4.5e-08, + "input_cost_per_audio_token": 5e-07, + "input_cost_per_token": 2.5e-07, + "input_cost_per_token_batches": 1.25e-07, + "input_cost_per_token_flex": 1.25e-07, + "input_cost_per_token_priority": 4.5e-07, "litellm_provider": "vertex_ai-language-models", "max_audio_length_hours": 8.4, "max_audio_per_prompt": 1, @@ -33947,8 +33971,11 @@ "max_video_length": 1, "max_videos_per_prompt": 10, "mode": "chat", - "output_cost_per_reasoning_token": 2.7e-06, - "output_cost_per_token": 2.7e-06, + "output_cost_per_reasoning_token": 1.5e-06, + "output_cost_per_token": 1.5e-06, + "output_cost_per_token_batches": 7.5e-07, + "output_cost_per_token_flex": 7.5e-07, + "output_cost_per_token_priority": 2.7e-06, "source": "https://cloud.google.com/vertex-ai/generative-ai/pricing#gemini-models", "supported_endpoints": [ "/v1/chat/completions", diff --git a/tests/test_litellm/llms/anthropic/experimental_pass_through/adapters/test_anthropic_experimental_pass_through_adapters_transformation.py b/tests/test_litellm/llms/anthropic/experimental_pass_through/adapters/test_anthropic_experimental_pass_through_adapters_transformation.py index 11465e6f718..44530fecebd 100644 --- a/tests/test_litellm/llms/anthropic/experimental_pass_through/adapters/test_anthropic_experimental_pass_through_adapters_transformation.py +++ b/tests/test_litellm/llms/anthropic/experimental_pass_through/adapters/test_anthropic_experimental_pass_through_adapters_transformation.py @@ -1203,6 +1203,51 @@ def test_streaming_chunk_with_both_text_and_tool_calls_issue_18238(): assert content_block_start["id"] == "toolu_bdrk_013xRVejhv3ybmLEGCoZib2b" +def test_streaming_chunk_with_text_and_empty_tool_calls_returns_text_delta(): + """ + Some OpenAI-compatible providers emit `tool_calls: []` on regular text chunks. + + Empty tool_calls should be treated as no tool call so the Anthropic adapter + does not shadow text with an empty input_json_delta. + """ + choices = [ + StreamingChoices( + finish_reason=None, + index=0, + delta=Delta( + provider_specific_fields=None, + content="Hello from vLLM", + role="assistant", + function_call=None, + tool_calls=[], + audio=None, + ), + logprobs=None, + ) + ] + + adapter = LiteLLMAnthropicMessagesAdapter() + + ( + type_of_content, + content_block_delta, + ) = adapter._translate_streaming_openai_chunk_to_anthropic(choices=choices) + + assert type_of_content == "text_delta" + assert content_block_delta["type"] == "text_delta" + assert content_block_delta["text"] == "Hello from vLLM" + + ( + block_type, + content_block_start, + ) = adapter._translate_streaming_openai_chunk_to_anthropic_content_block( + choices=choices + ) + + assert block_type == "text" + assert content_block_start == {"type": "text", "text": ""} + + # ============================================================================ # Cache Control Transformation Tests # ============================================================================ diff --git a/tests/test_litellm/llms/azure/chat/test_azure_base_model_routing.py b/tests/test_litellm/llms/azure/chat/test_azure_base_model_routing.py new file mode 100644 index 00000000000..1e8e23c38ca --- /dev/null +++ b/tests/test_litellm/llms/azure/chat/test_azure_base_model_routing.py @@ -0,0 +1,274 @@ +"""Tests for decoupling Azure deployment IDs from underlying model names. + +When users name their Azure deployment something non-standard (e.g. "my-deployment-id"), +setting ``base_model`` should drive model-type detection (o-series, gpt-5, +etc.) so the correct config, supported params, and param mapping are used. +""" + +import pytest + +import litellm +from litellm.llms.azure.chat.gpt_5_transformation import AzureOpenAIGPT5Config +from litellm.llms.azure.chat.o_series_transformation import AzureOpenAIO1Config +from litellm.utils import ProviderConfigManager, get_optional_params + + +# --------------------------------------------------------------------------- +# _get_azure_config — routes to the correct config based on base_model +# --------------------------------------------------------------------------- +class TestGetAzureConfigWithBaseModel: + """ProviderConfigManager._get_azure_config should use base_model for detection.""" + + def test_should_return_gpt5_config_when_base_model_is_gpt5(self): + config = ProviderConfigManager._get_azure_config( + model="my-deployment-id", base_model="azure/gpt-5.2" + ) + assert isinstance(config, AzureOpenAIGPT5Config) + + def test_should_return_o_series_config_when_base_model_is_o_series(self): + config = ProviderConfigManager._get_azure_config( + model="my-deployment-id", base_model="azure/o4-mini" + ) + assert isinstance(config, AzureOpenAIO1Config) + + def test_should_return_default_config_when_base_model_is_regular(self): + config = ProviderConfigManager._get_azure_config( + model="my-deployment-id", base_model="azure/gpt-4o" + ) + assert type(config).__name__ == "AzureOpenAIConfig" + + def test_should_fallback_to_model_when_base_model_is_none(self): + config = ProviderConfigManager._get_azure_config( + model="gpt-5.2", base_model=None + ) + assert isinstance(config, AzureOpenAIGPT5Config) + + def test_should_return_default_config_when_both_are_non_standard(self): + config = ProviderConfigManager._get_azure_config( + model="my-deployment-id", base_model=None + ) + assert type(config).__name__ == "AzureOpenAIConfig" + + +# --------------------------------------------------------------------------- +# get_provider_chat_config — threads base_model through for Azure +# --------------------------------------------------------------------------- +class TestGetProviderChatConfigWithBaseModel: + """get_provider_chat_config should pass base_model to Azure config selection.""" + + def test_should_return_gpt5_config_for_custom_deployment_with_base_model(self): + from litellm.types.utils import LlmProviders + + config = ProviderConfigManager.get_provider_chat_config( + model="my-deployment-id", + provider=LlmProviders.AZURE, + base_model="azure/gpt-5", + ) + assert isinstance(config, AzureOpenAIGPT5Config) + + def test_should_return_o_series_config_for_custom_deployment_with_base_model(self): + from litellm.types.utils import LlmProviders + + config = ProviderConfigManager.get_provider_chat_config( + model="my-other-deployment", + provider=LlmProviders.AZURE, + base_model="azure/o3-mini", + ) + assert isinstance(config, AzureOpenAIO1Config) + + +# --------------------------------------------------------------------------- +# get_supported_openai_params — base_model drives Azure param detection +# --------------------------------------------------------------------------- +class TestGetSupportedOpenAIParamsWithBaseModel: + """get_supported_openai_params should use base_model for Azure detection.""" + + def test_should_return_gpt5_params_for_custom_deployment_with_gpt5_base_model( + self, + ): + params = litellm.get_supported_openai_params( + model="my-deployment-id", + custom_llm_provider="azure", + base_model="azure/gpt-5", + ) + assert params is not None + assert "reasoning_effort" in params + # gpt-5 maps max_tokens -> max_completion_tokens, verifying we got GPT-5 config + assert "max_completion_tokens" in params + + def test_should_return_o_series_params_for_custom_deployment_with_o_series_base_model( + self, + ): + params = litellm.get_supported_openai_params( + model="my-other-deployment", + custom_llm_provider="azure", + base_model="azure/o4-mini", + ) + assert params is not None + assert "reasoning_effort" in params + + def test_should_return_regular_params_when_no_base_model(self): + """When base_model is not set and model is non-standard, default Azure config.""" + params = litellm.get_supported_openai_params( + model="my-deployment-id", + custom_llm_provider="azure", + ) + assert params is not None + # Default Azure config supports temperature + assert "temperature" in params + + +# --------------------------------------------------------------------------- +# get_optional_params — base_model drives Azure param mapping +# --------------------------------------------------------------------------- +class TestGetOptionalParamsWithBaseModel: + """get_optional_params should use base_model for Azure model-type detection.""" + + def test_should_map_max_tokens_for_custom_deployment_with_gpt5_base_model(self): + """A non-standard deployment name + gpt-5 base_model should map max_tokens -> max_completion_tokens.""" + params = get_optional_params( + model="my-deployment-id", + custom_llm_provider="azure", + max_tokens=100, + base_model="azure/gpt-5", + ) + assert params.get("max_completion_tokens") == 100 + assert "max_tokens" not in params + + def test_should_keep_max_tokens_for_custom_deployment_without_base_model(self): + """A non-standard deployment name without base_model should use default Azure config.""" + params = get_optional_params( + model="my-deployment-id", + custom_llm_provider="azure", + max_tokens=100, + api_version="2024-05-01-preview", + ) + # Default AzureOpenAIConfig keeps max_tokens as-is (or maps based on api_version) + assert "max_tokens" in params or "max_completion_tokens" in params + + def test_should_support_reasoning_effort_for_custom_deployment_with_o_series_base_model( + self, + ): + """A non-standard deployment name + o-series base_model should accept reasoning_effort.""" + params = get_optional_params( + model="my-other-deployment", + custom_llm_provider="azure", + reasoning_effort="low", + base_model="azure/o4-mini", + ) + assert params.get("reasoning_effort") == "low" + + def test_should_reject_temperature_for_custom_deployment_with_gpt5_base_model( + self, + ): + """A non-standard deployment + gpt-5 base_model should reject temperature.""" + with pytest.raises(litellm.UnsupportedParamsError): + get_optional_params( + model="my-deployment-id", + custom_llm_provider="azure", + temperature=0.5, + base_model="azure/gpt-5", + ) + + +# --------------------------------------------------------------------------- +# Backward compatibility — existing patterns still work +# --------------------------------------------------------------------------- +class TestBackwardCompatibility: + """Existing model-name-based and prefix-based patterns must keep working.""" + + def test_should_detect_gpt5_from_model_name(self): + config = ProviderConfigManager._get_azure_config(model="gpt-5.2") + assert isinstance(config, AzureOpenAIGPT5Config) + + def test_should_detect_gpt5_from_gpt5_series_prefix(self): + config = ProviderConfigManager._get_azure_config( + model="gpt5_series/my-deployment" + ) + assert isinstance(config, AzureOpenAIGPT5Config) + + def test_should_detect_o_series_from_model_name(self): + config = ProviderConfigManager._get_azure_config(model="o4-mini") + assert isinstance(config, AzureOpenAIO1Config) + + def test_should_detect_o_series_from_o_series_prefix(self): + config = ProviderConfigManager._get_azure_config(model="o_series/my-deployment") + assert isinstance(config, AzureOpenAIO1Config) + + def test_should_handle_gpt5_chat_model_correctly(self): + """gpt-5-chat models should NOT be routed to GPT-5 config.""" + config = ProviderConfigManager._get_azure_config(model="gpt-5-chat") + assert type(config).__name__ == "AzureOpenAIConfig" + + def test_base_model_overrides_model_detection(self): + """base_model should take priority over model for type detection.""" + # model looks like o-series, but base_model says gpt-5 + config = ProviderConfigManager._get_azure_config( + model="o3-mini", base_model="azure/gpt-5.2" + ) + assert isinstance(config, AzureOpenAIGPT5Config) + + +# --------------------------------------------------------------------------- +# Deep config method awareness — base_model flows into config internals +# --------------------------------------------------------------------------- +class TestBaseModelFlowsIntoConfigInternals: + """base_model should be used by config internal methods (e.g. is_model_gpt_5_2_model).""" + + def test_should_support_logprobs_for_prefixed_deployment_with_gpt52_base_model( + self, + ): + """Deployment 'my-gpt-5.2' with base_model='azure/gpt-5.2' should support logprobs.""" + params = litellm.get_supported_openai_params( + model="gpt5_series/my-gpt-5.2", + custom_llm_provider="azure", + base_model="azure/gpt-5.2", + ) + assert params is not None + assert "logprobs" in params + assert "top_logprobs" in params + + def test_should_support_logprobs_for_plain_deployment_with_gpt52_base_model(self): + """Deployment 'my-deployment-id' with base_model='azure/gpt-5.2' should support logprobs.""" + params = litellm.get_supported_openai_params( + model="my-deployment-id", + custom_llm_provider="azure", + base_model="azure/gpt-5.2", + ) + assert params is not None + assert "logprobs" in params + assert "top_logprobs" in params + + def test_should_not_support_logprobs_for_gpt5_base_model(self): + """Deployment with base_model='azure/gpt-5' (not 5.2) should NOT support logprobs.""" + params = litellm.get_supported_openai_params( + model="my-deployment-id", + custom_llm_provider="azure", + base_model="azure/gpt-5", + ) + assert params is not None + assert "logprobs" not in params + assert "top_logprobs" not in params + + def test_should_pass_logprobs_through_get_optional_params(self): + """logprobs should pass validation in get_optional_params when base_model is gpt-5.2.""" + params = get_optional_params( + model="gpt5_series/my-gpt-5.2", + custom_llm_provider="azure", + logprobs=True, + top_logprobs=5, + base_model="azure/gpt-5.2", + ) + assert params.get("logprobs") is True + assert params.get("top_logprobs") == 5 + + def test_should_map_max_tokens_for_prefixed_deployment_with_gpt5_base_model(self): + """my-gpt-5.2 with base_model should correctly map max_tokens -> max_completion_tokens.""" + params = get_optional_params( + model="gpt5_series/my-gpt-5.2", + custom_llm_provider="azure", + max_tokens=200, + base_model="azure/gpt-5.2", + ) + assert params.get("max_completion_tokens") == 200 + assert "max_tokens" not in params diff --git a/tests/test_litellm/llms/openai/responses/test_openai_responses_transformation.py b/tests/test_litellm/llms/openai/responses/test_openai_responses_transformation.py index acb9fa9b64c..4b2e9471fb7 100644 --- a/tests/test_litellm/llms/openai/responses/test_openai_responses_transformation.py +++ b/tests/test_litellm/llms/openai/responses/test_openai_responses_transformation.py @@ -86,6 +86,90 @@ class TestOpenAIResponsesAPIConfig: self.validate_responses_api_request_params(result, expected_fields) + def test_transform_strips_cache_control_from_input_content_blocks(self): + """`cache_control` markers (Anthropic-only) must be stripped from + Responses API input content blocks before sending to OpenAI. + + OpenAI rejects unknown params on input content blocks with HTTP 400: + "Unknown parameter: 'input[0].content[0].cache_control'" + Chat Completions strips these via + `remove_cache_control_flag_from_messages_and_tools`; the Responses + path must do the same. + """ + input_with_cache_control = [ + { + "role": "user", + "content": [ + { + "type": "input_text", + "text": "Hello", + "cache_control": {"type": "ephemeral"}, + } + ], + } + ] + + result = self.config.transform_responses_api_request( + model=self.model, + input=input_with_cache_control, + response_api_optional_request_params={}, + litellm_params={}, + headers={}, + ) + + assert "cache_control" not in result["input"][0]["content"][0] + assert result["input"][0]["content"][0]["type"] == "input_text" + assert result["input"][0]["content"][0]["text"] == "Hello" + + def test_transform_strips_cache_control_from_tools(self): + """`cache_control` markers must also be stripped from tools for + symmetry with the Chat Completions path. OpenAI currently accepts + cache_control on tools silently but stripping keeps the wire payload + clean and matches `remove_cache_control_flag_from_messages_and_tools`. + """ + tools_with_cache_control = [ + { + "type": "function", + "name": "get_weather", + "description": "Get the weather", + "parameters": { + "type": "object", + "properties": {"city": {"type": "string"}}, + }, + "cache_control": {"type": "ephemeral"}, + } + ] + + result = self.config.transform_responses_api_request( + model=self.model, + input="hi", + response_api_optional_request_params={"tools": tools_with_cache_control}, + litellm_params={}, + headers={}, + ) + + assert "cache_control" not in result["tools"][0] + assert result["tools"][0]["name"] == "get_weather" + + def test_transform_preserves_input_without_cache_control(self): + """Inputs without cache_control must pass through unmodified.""" + input_clean = [ + { + "role": "user", + "content": [{"type": "input_text", "text": "Hello"}], + } + ] + + result = self.config.transform_responses_api_request( + model=self.model, + input=input_clean, + response_api_optional_request_params={}, + litellm_params={}, + headers={}, + ) + + assert result["input"] == input_clean + def test_transform_streaming_response(self): """Test streaming response transformation""" # Test with a text delta event diff --git a/tests/test_litellm/test_register_model_custom_pricing.py b/tests/test_litellm/test_register_model_custom_pricing.py index 1efd698fb64..719cb8eecd2 100644 --- a/tests/test_litellm/test_register_model_custom_pricing.py +++ b/tests/test_litellm/test_register_model_custom_pricing.py @@ -190,3 +190,164 @@ def test_build_custom_pricing_entry_time_based(): assert entry["litellm_provider"] == "openai" assert entry["input_cost_per_second"] == 0.01 assert entry["output_cost_per_second"] == 0.02 + + +def test_register_model_strips_none_litellm_provider(): + """``get_model_info`` returns ``litellm_provider: None`` for deployments + registered without a provider (e.g. ``Router.add_deployment`` flows). + ``register_model`` must not persist that None into ``model_cost``, + otherwise ``_check_provider_match`` will drop custom pricing on + subsequent cost lookups. + + Regression test for https://github.com/BerriAI/litellm/issues/28336. + """ + from litellm.utils import _check_provider_match + + model_key = "test-custom-pricing-no-provider-28336" + litellm.model_cost.pop(model_key, None) + + try: + litellm.register_model( + { + model_key: { + "input_cost_per_token": 0.001, + "output_cost_per_token": 0.002, + } + } + ) + + registered = litellm.model_cost.get(model_key) + assert registered is not None, f"{model_key} should be in model_cost" + # The key may be absent entirely, but if present it must not be None. + assert ( + "litellm_provider" not in registered + or registered["litellm_provider"] is not None + ) + # Downstream consumers must accept this entry for any provider, + # mirroring what the cost calculator does. + assert _check_provider_match(registered, "openai") is True + assert _check_provider_match(registered, "anthropic") is True + finally: + litellm.model_cost.pop(model_key, None) + + +def test_register_model_strips_none_litellm_provider_from_get_model_info(monkeypatch): + """Directly exercise the strip in ``register_model``. + + The companion test above hits the ``except Exception`` branch where + ``existing_model`` is an empty dict, so the ``pop`` is a no-op. This + test patches ``get_model_info`` to return the failure mode the strip + was added to handle, namely a populated dict whose ``litellm_provider`` + is ``None``. Without the strip, the merged entry in + ``litellm.model_cost`` would carry ``litellm_provider: None`` and + ``_check_provider_match`` would drop custom pricing. + + Regression test for https://github.com/BerriAI/litellm/issues/28336. + """ + from litellm import utils as litellm_utils + from litellm.utils import _check_provider_match + + model_key = "test-strip-none-provider-from-get-model-info-28336" + litellm.model_cost.pop(model_key, None) + + def _fake_get_model_info(model, *args, **kwargs): + assert model == model_key + return { + "key": model_key, + "litellm_provider": None, + "mode": "chat", + "max_tokens": 4096, + } + + # ``register_model`` calls ``get_model_info.cache_clear`` via + # ``_invalidate_model_cost_lowercase_map``, so the replacement must + # expose a no-op ``cache_clear`` attribute. + _fake_get_model_info.cache_clear = lambda: None + monkeypatch.setattr(litellm_utils, "get_model_info", _fake_get_model_info) + + try: + litellm.register_model( + { + model_key: { + "input_cost_per_token": 0.001, + "output_cost_per_token": 0.002, + } + } + ) + + registered = litellm.model_cost.get(model_key) + assert registered is not None, f"{model_key} should be in model_cost" + # The strip must have removed the None-valued provider that + # ``get_model_info`` returned. The key may be absent entirely, but + # it must never be present with value ``None``. + assert "litellm_provider" not in registered or ( + registered["litellm_provider"] is not None + ), ( + "register_model failed to strip litellm_provider=None returned " + f"by get_model_info, got {registered.get('litellm_provider')!r}" + ) + # Metadata from the patched ``get_model_info`` must still flow + # through, so we know the strip did not nuke the rest of the entry. + assert registered.get("mode") == "chat" + assert registered.get("max_tokens") == 4096 + # And custom pricing from the registration call must be preserved. + assert registered.get("input_cost_per_token") == 0.001 + assert registered.get("output_cost_per_token") == 0.002 + # Downstream _check_provider_match must accept any provider for + # this entry, mirroring the cost calculator path. + assert _check_provider_match(registered, "openai") is True + assert _check_provider_match(registered, "anthropic") is True + finally: + litellm.model_cost.pop(model_key, None) + + +def test_register_model_router_add_deployment_custom_pricing_applies(): + """End-to-end regression for https://github.com/BerriAI/litellm/issues/28336. + + ``Router.add_deployment`` registers custom pricing without passing + ``litellm_provider``. Cost calculation must still pick up the custom + pricing instead of falling back to the default provider price. + """ + from litellm import Router + + model_key = "router-add-deployment-custom-pricing-28336" + deployment_model = f"openai/{model_key}" + litellm.model_cost.pop(model_key, None) + litellm.model_cost.pop(deployment_model, None) + + router = Router( + model_list=[ + { + "model_name": model_key, + "litellm_params": { + "model": deployment_model, + "api_key": "fake-key-for-registration", + "input_cost_per_token": 0.00042, + "output_cost_per_token": 0.00084, + }, + "model_info": {"id": "deployment-28336"}, + } + ] + ) + + try: + # ``add_deployment`` runs as part of ``Router.__init__``; the + # registered entry must not block ``_check_provider_match`` for + # the deployment's provider. + from litellm.utils import _check_provider_match + + registered_keys = [ + k for k in (deployment_model, model_key) if k in litellm.model_cost + ] + assert registered_keys, ( + "Router.add_deployment did not register custom pricing for " + f"{model_key} / {deployment_model}" + ) + for k in registered_keys: + assert _check_provider_match(litellm.model_cost[k], "openai") is True, ( + f"custom pricing for {k} was dropped by _check_provider_match" + ) + finally: + litellm.model_cost.pop(model_key, None) + litellm.model_cost.pop(deployment_model, None) + del router diff --git a/tests/test_litellm/test_utils.py b/tests/test_litellm/test_utils.py index de286aede93..0efb3083139 100644 --- a/tests/test_litellm/test_utils.py +++ b/tests/test_litellm/test_utils.py @@ -1140,6 +1140,34 @@ def test_check_provider_match(): assert litellm.utils._check_provider_match(model_info, "openai") is False +def test_check_provider_match_none_value_matches_any_provider(): + """ + A ``litellm_provider`` of None must be treated the same as a missing + key: both mean "no provider constraint" and should match any + ``custom_llm_provider``. + + Regression test for https://github.com/BerriAI/litellm/issues/28336. + Before the fix, ``register_model`` persisted ``litellm_provider: None`` + via ``get_model_info`` for deployments registered without a provider + (e.g. ``Router.add_deployment``), which caused ``_check_provider_match`` + to drop custom pricing intermittently. + """ + # Missing key already returned True; None must behave identically. + assert litellm.utils._check_provider_match({}, "openai") is True + assert ( + litellm.utils._check_provider_match({"litellm_provider": None}, "openai") + is True + ) + assert ( + litellm.utils._check_provider_match({"litellm_provider": None}, "anthropic") + is True + ) + # When custom_llm_provider is also None nothing constrains the match. + assert ( + litellm.utils._check_provider_match({"litellm_provider": None}, None) is True + ) + + def test_get_provider_rerank_config(): """ Test the get_provider_rerank_config function for various providers From b0b25ae4b9bf9aec6805cb001707c18c0cf5c0fd Mon Sep 17 00:00:00 2001 From: Mateo Wang <277851410+mateo-berri@users.noreply.github.com> Date: Fri, 22 May 2026 10:40:59 -0700 Subject: [PATCH 04/13] Include team alias in CLI JWT token (#28621) --- litellm/proxy/auth/auth_checks.py | 6 +++++- litellm/proxy/management_endpoints/ui_sso.py | 13 ++++++++++++- .../test_litellm/proxy/auth/test_auth_checks.py | 17 +++++++++++++++++ .../proxy/management_endpoints/test_ui_sso.py | 6 ++++++ 4 files changed, 40 insertions(+), 2 deletions(-) diff --git a/litellm/proxy/auth/auth_checks.py b/litellm/proxy/auth/auth_checks.py index 09bb8057203..14f198e0f12 100644 --- a/litellm/proxy/auth/auth_checks.py +++ b/litellm/proxy/auth/auth_checks.py @@ -2353,7 +2353,9 @@ class ExperimentalUIJWTToken: @staticmethod def get_cli_jwt_auth_token( - user_info: LiteLLM_UserTable, team_id: Optional[str] = None + user_info: LiteLLM_UserTable, + team_id: Optional[str] = None, + team_alias: Optional[str] = None, ) -> str: """ Generate a JWT token for CLI authentication with configurable expiration. @@ -2364,6 +2366,7 @@ class ExperimentalUIJWTToken: Args: user_info: User information from the database team_id: Team ID for the user (optional, uses user's team if available) + team_alias: Team alias for the selected team, if available Returns: Encrypted JWT token string @@ -2397,6 +2400,7 @@ class ExperimentalUIJWTToken: expires=expires, user_id=user_info.user_id, team_id=_team_id, + team_alias=team_alias, models=user_info.models, max_parallel_requests=None, user_role=LitellmUserRoles(user_info.user_role), diff --git a/litellm/proxy/management_endpoints/ui_sso.py b/litellm/proxy/management_endpoints/ui_sso.py index d3e1099d968..d6082899c02 100644 --- a/litellm/proxy/management_endpoints/ui_sso.py +++ b/litellm/proxy/management_endpoints/ui_sso.py @@ -2229,6 +2229,17 @@ async def cli_poll_key( # If no team_id provided and user has 0 or 1 team, use first team (or None) team_id = user_teams[0] if len(user_teams) > 0 else None + team_alias = None + if team_id and isinstance(user_team_details, list): + team_alias = next( + ( + team.get("team_alias") + for team in user_team_details + if team.get("team_id") == team_id + ), + None, + ) + # Create user object for JWT generation user_info = LiteLLM_UserTable( user_id=user_id, @@ -2240,7 +2251,7 @@ async def cli_poll_key( # Generate CLI JWT on-demand (expiration configurable via LITELLM_CLI_JWT_EXPIRATION_HOURS) # Pass selected team_id to ensure JWT has correct team jwt_token = ExperimentalUIJWTToken.get_cli_jwt_auth_token( - user_info=user_info, team_id=team_id + user_info=user_info, team_id=team_id, team_alias=team_alias ) # Delete cache entry (single-use) diff --git a/tests/test_litellm/proxy/auth/test_auth_checks.py b/tests/test_litellm/proxy/auth/test_auth_checks.py index 35a3bd7f657..116ba83f42e 100644 --- a/tests/test_litellm/proxy/auth/test_auth_checks.py +++ b/tests/test_litellm/proxy/auth/test_auth_checks.py @@ -127,6 +127,23 @@ def test_get_experimental_ui_login_jwt_auth_token_valid(valid_sso_user_defined_v assert expires <= now + timedelta(minutes=10, seconds=2) +def test_get_cli_jwt_auth_token_includes_team_alias(valid_sso_user_defined_values): + token = ExperimentalUIJWTToken.get_cli_jwt_auth_token( + valid_sso_user_defined_values, + team_id="team-123", + team_alias="test-team", + ) + + decrypted_token = decrypt_value_helper( + token, key="ui_hash_key", exception_type="debug" + ) + assert decrypted_token is not None + token_data = json.loads(decrypted_token) + + assert token_data["team_id"] == "team-123" + assert token_data["team_alias"] == "test-team" + + def test_get_experimental_ui_login_jwt_auth_token_uses_10_min_expiry( valid_sso_user_defined_values, ): diff --git a/tests/test_litellm/proxy/management_endpoints/test_ui_sso.py b/tests/test_litellm/proxy/management_endpoints/test_ui_sso.py index a72633b726f..c763e9c0e98 100644 --- a/tests/test_litellm/proxy/management_endpoints/test_ui_sso.py +++ b/tests/test_litellm/proxy/management_endpoints/test_ui_sso.py @@ -2497,6 +2497,11 @@ class TestCLIKeyRegenerationFlow: "user_id": "test-user-789", "user_role": "internal_user", "teams": ["team-a", "team-b", "team-c"], + "team_details": [ + {"team_id": "team-a", "team_alias": "Team A"}, + {"team_id": "team-b", "team_alias": "Team B"}, + {"team_id": "team-c", "team_alias": "Team C"}, + ], "models": ["gpt-4"], "user_email": "test@example.com", } @@ -2551,6 +2556,7 @@ class TestCLIKeyRegenerationFlow: mock_get_jwt.assert_called_once() jwt_call_args = mock_get_jwt.call_args assert jwt_call_args.kwargs["team_id"] == selected_team + assert jwt_call_args.kwargs["team_alias"] == "Team B" # Verify session was deleted after JWT generation mock_cache.delete_cache.assert_called_once() From 985574b6be662dbde8abcdf034e30d3b1da4cf9f Mon Sep 17 00:00:00 2001 From: yuneng-jiang Date: Fri, 22 May 2026 11:22:38 -0700 Subject: [PATCH 05/13] fix(check_licenses): read PEP 639 license-expression metadata (#28529) The dependency license checker only read the legacy free-text `info.license` field from PyPI. Packages that adopt PEP 639 publish their license as an SPDX expression in `info.license_expression` and leave the legacy field null, so the checker reported "Unknown license" and failed CI for every newly-bumped PEP 639 dependency. `get_package_license_from_pypi` now resolves the license in order: `license_expression`, then legacy `license`, then the `License :: OSI Approved :: ...` trove classifiers. `is_license_acceptable` splits compound SPDX expressions on the uppercase OR/AND operators (case-sensitive, so the lowercase `-or-later` inside an identifier is not mistaken for an operator) and strips `WITH ` suffixes, requiring every component to be acceptable. Free-text license blobs are detected and fall back to the original whole-string matching. The `black` and `pydantic-settings` entries in liccheck.ini that existed solely to work around this now resolve correctly on their own and have been removed. --- tests/code_coverage_tests/check_licenses.py | 82 +++++++- tests/code_coverage_tests/liccheck.ini | 2 - tests/test_litellm/test_check_licenses.py | 211 ++++++++++++++++++++ 3 files changed, 288 insertions(+), 7 deletions(-) create mode 100644 tests/test_litellm/test_check_licenses.py diff --git a/tests/code_coverage_tests/check_licenses.py b/tests/code_coverage_tests/check_licenses.py index 668aefa8024..5fb2b495c24 100644 --- a/tests/code_coverage_tests/check_licenses.py +++ b/tests/code_coverage_tests/check_licenses.py @@ -31,6 +31,13 @@ DEFAULT_TRANSITIVE_PIN_PACKAGES = ( "wheel", ) +# SPDX license expressions (PEP 639 "License-Expression") join identifiers with +# the uppercase operators OR / AND / WITH. The split is case-sensitive: the +# lowercase "-or-later" inside an identifier such as "GPL-2.0-or-later" is part +# of the identifier, not an operator. +_SPDX_OPERATOR_SPLIT = re.compile(r"\s+(?:OR|AND)\s+") +_SPDX_WITH_SUFFIX = re.compile(r"\s+WITH\s+.*", re.DOTALL) + @dataclass class PackageLicense: @@ -109,21 +116,86 @@ class LicenseChecker: def get_package_license_from_pypi( self, package_name: str, version: str ) -> Optional[str]: - """Fetch license information for a package from PyPI.""" + """Fetch license information for a package from PyPI. + + Prefers the PEP 639 SPDX expression (``info.license_expression``), + falls back to the legacy free-text ``info.license`` field, and as a + last resort derives the license from the ``License :: OSI Approved :: + ...`` trove classifiers. + """ try: url = f"https://pypi.org/pypi/{package_name}/{version}/json" response = requests.get(url, timeout=10) response.raise_for_status() - data = response.json() - return data.get("info", {}).get("license") + info = response.json().get("info", {}) or {} + return ( + info.get("license_expression") + or info.get("license") + or self._license_from_classifiers(info.get("classifiers") or []) + ) except Exception as e: print( f"Warning: Failed to fetch license for {package_name} {version}: {str(e)}" ) return None - def is_license_acceptable(self, license_str: str) -> Tuple[bool, str]: - """Check if a license is acceptable based on configured lists.""" + @staticmethod + def _license_from_classifiers(classifiers: List[str]) -> Optional[str]: + """Derive a license name from the ``License :: OSI Approved :: ...`` trove classifiers.""" + prefix = "License :: OSI Approved :: " + for classifier in classifiers: + if classifier.startswith(prefix): + license_name = classifier[len(prefix) :].strip() + if license_name: + return license_name + return None + + @staticmethod + def _split_spdx_expression(license_str: str) -> Optional[List[str]]: + """Split an SPDX license expression into its component identifiers. + + Returns ``None`` when the string is not a recognizable SPDX expression + (for example a free-text license blob), so callers fall back to + whole-string matching. + """ + if "OR" not in license_str and "AND" not in license_str: + return None + + components: List[str] = [] + normalized = license_str.replace("(", " ").replace(")", " ") + for part in _SPDX_OPERATOR_SPLIT.split(normalized): + # Drop any "WITH " suffix: the exception qualifies the + # preceding license, it is not itself a license to authorize. + identifier = _SPDX_WITH_SUFFIX.sub("", part).strip() + if not identifier: + continue + # SPDX short-form identifiers are single whitespace-free tokens; a + # component with internal whitespace means this is free text. + if any(char.isspace() for char in identifier): + return None + components.append(identifier) + + return components if len(components) > 1 else None + + def is_license_acceptable(self, license_str: Optional[str]) -> Tuple[bool, str]: + """Check if a license (or compound SPDX expression) is acceptable.""" + if not license_str: + return False, "Unknown license" + + components = self._split_spdx_expression(license_str) + if components is None: + return self._is_single_license_acceptable(license_str) + + # Compound SPDX expression: conservatively require every component to + # be acceptable on its own (the safe direction for a CI gate). + for component in components: + is_acceptable, reason = self._is_single_license_acceptable(component) + if not is_acceptable: + return False, f"{reason} (in SPDX expression '{license_str}')" + return True, f"All SPDX components authorized: {', '.join(components)}" + + def _is_single_license_acceptable(self, license_str: str) -> Tuple[bool, str]: + """Check if a single license identifier is acceptable based on configured lists.""" if not license_str: return False, "Unknown license" diff --git a/tests/code_coverage_tests/liccheck.ini b/tests/code_coverage_tests/liccheck.ini index 0d1a6f0b045..5a09403c570 100644 --- a/tests/code_coverage_tests/liccheck.ini +++ b/tests/code_coverage_tests/liccheck.ini @@ -90,7 +90,6 @@ jinja2: >=3.1.4 # BSD 3-Clause License litellm-proxy-extras: >=0.1.1 # MIT License litellm-enterprise: >=0.1.1 # LiteLLM Enterprise License a2a-sdk: >=0.3.22 # Apache 2.0 license -pydantic-settings: >=2.14.1 # MIT License manually verified (uses PEP 639 License-Expression: MIT, not the legacy License field, so liccheck reports it as unknown) anyio: >=4.5.0 # Unknown license httpx-aiohttp: >=0.1.4 # Unknown license backoff: >=2.2.1 # Unknown license @@ -156,7 +155,6 @@ pytest: >=9.0.3 # MIT license pytest-postgresql: >=7.0.2 # LGPLv3+ license pytest-xdist: >=3.8.0 # MIT License ruff: >=0.15.3 # MIT License -black: >=26.3.1 # MIT License manually verified (uses PEP 639 License-Expression: MIT, not the legacy License field, so liccheck reports it as unknown) types-requests: >=2.32.4.20260107 # Apache 2.0 license (typeshed) types-pyyaml: >=6.0.12.20250915 # Apache 2.0 license (typeshed) fakeredis: >=2.34.1 # BSD license diff --git a/tests/test_litellm/test_check_licenses.py b/tests/test_litellm/test_check_licenses.py new file mode 100644 index 00000000000..4d72f185a25 --- /dev/null +++ b/tests/test_litellm/test_check_licenses.py @@ -0,0 +1,211 @@ +"""Tests for the dependency license checker at tests/code_coverage_tests/check_licenses.py. + +Focus: PEP 639 license metadata. Packages that adopt PEP 639 publish their +license as an SPDX expression in ``info.license_expression`` and often leave the +legacy ``info.license`` field null, so the checker must read the new field (and +fall back to trove classifiers) instead of reporting "Unknown license". + +PyPI HTTP responses are mocked — these tests never hit the network. +""" + +import os +import sys +from pathlib import Path + +_CODE_COVERAGE_DIR = os.path.join( + os.path.dirname(os.path.abspath(__file__)), "..", "code_coverage_tests" +) +sys.path.insert(0, _CODE_COVERAGE_DIR) + +import check_licenses # noqa: E402 + +_LICCHECK_INI = Path(_CODE_COVERAGE_DIR) / "liccheck.ini" + + +class _FakeResponse: + def __init__(self, payload): + self._payload = payload + + def raise_for_status(self): + return None + + def json(self): + return self._payload + + +def _make_checker(): + return check_licenses.LicenseChecker(config_file=_LICCHECK_INI) + + +def _patch_pypi(monkeypatch, info): + """Make PyPI return a JSON response with the given ``info`` block.""" + + def _fake_get(url, timeout=None): + return _FakeResponse({"info": info}) + + monkeypatch.setattr(check_licenses.requests, "get", _fake_get) + + +# -------------------------------------------------------------------------- +# get_package_license_from_pypi: license metadata resolution +# -------------------------------------------------------------------------- + + +def test_get_license_prefers_license_expression(monkeypatch): + """(a) PEP 639 packages publish the SPDX expression in license_expression.""" + _patch_pypi( + monkeypatch, + {"license_expression": "MIT", "license": None, "classifiers": []}, + ) + checker = _make_checker() + assert checker.get_package_license_from_pypi("black", "26.3.1") == "MIT" + + +def test_license_expression_wins_when_both_present(monkeypatch): + """license_expression takes precedence over the legacy license field.""" + _patch_pypi( + monkeypatch, + {"license_expression": "Apache-2.0", "license": "stale free text"}, + ) + checker = _make_checker() + assert checker.get_package_license_from_pypi("pkg", "1.0.0") == "Apache-2.0" + + +def test_get_license_falls_back_to_legacy_license(monkeypatch): + """(b) Pre-PEP-639 packages only set the legacy free-text license field.""" + _patch_pypi( + monkeypatch, + {"license_expression": None, "license": "MIT License", "classifiers": []}, + ) + checker = _make_checker() + assert checker.get_package_license_from_pypi("pkg", "1.0.0") == "MIT License" + + +def test_get_license_falls_back_to_classifiers(monkeypatch): + """(c) Some packages express the license only through trove classifiers.""" + _patch_pypi( + monkeypatch, + { + "license_expression": None, + "license": None, + "classifiers": [ + "Programming Language :: Python :: 3", + "License :: OSI Approved :: Apache Software License", + ], + }, + ) + checker = _make_checker() + assert ( + checker.get_package_license_from_pypi("pkg", "1.0.0") + == "Apache Software License" + ) + + +def test_get_license_returns_none_when_unset(monkeypatch): + """(d) With no license metadata at all the license stays unknown.""" + _patch_pypi( + monkeypatch, + {"license_expression": None, "license": None, "classifiers": []}, + ) + checker = _make_checker() + assert checker.get_package_license_from_pypi("pkg", "1.0.0") is None + + +def test_get_license_returns_none_on_request_failure(monkeypatch): + """Network/HTTP failures are swallowed and reported as unknown.""" + + def _boom(url, timeout=None): + raise RuntimeError("network down") + + monkeypatch.setattr(check_licenses.requests, "get", _boom) + checker = _make_checker() + assert checker.get_package_license_from_pypi("pkg", "1.0.0") is None + + +# -------------------------------------------------------------------------- +# is_license_acceptable: SPDX identifiers and compound expressions +# -------------------------------------------------------------------------- + + +def test_spdx_identifiers_are_authorized(): + """Plain SPDX identifiers match the legacy-spelled authorized list as-is.""" + checker = _make_checker() + for identifier in ("MIT", "Apache-2.0", "BSD-3-Clause"): + is_ok, reason = checker.is_license_acceptable(identifier) + assert is_ok is True, f"{identifier}: {reason}" + + +def test_spdx_compound_or_expression_is_authorized(): + checker = _make_checker() + is_ok, reason = checker.is_license_acceptable("MIT OR Apache-2.0") + assert is_ok is True, reason + + +def test_spdx_with_exception_in_compound_is_authorized(): + """The 'WITH ' suffix is stripped; the base license is checked.""" + checker = _make_checker() + is_ok, reason = checker.is_license_acceptable( + "Apache-2.0 WITH LLVM-exception OR MIT" + ) + assert is_ok is True, reason + + +def test_spdx_gpl3_is_rejected(): + """GPL-3.0 spellings must fail — they match no authorized license.""" + checker = _make_checker() + for expr in ("GPL-3.0-only", "GPL-3.0-or-later"): + is_ok, reason = checker.is_license_acceptable(expr) + assert is_ok is False, f"{expr} unexpectedly accepted: {reason}" + + +def test_spdx_compound_with_copyleft_component_is_rejected(): + """A permissive-OR-copyleft expression is conservatively rejected.""" + checker = _make_checker() + is_ok, _ = checker.is_license_acceptable("MIT OR GPL-3.0-only") + assert is_ok is False + + +def test_or_later_identifier_is_not_split_as_operator(): + """The lowercase '-or-later' inside an identifier is not the SPDX OR operator.""" + assert ( + check_licenses.LicenseChecker._split_spdx_expression("GPL-2.0-or-later") is None + ) + + +def test_free_text_license_is_not_treated_as_spdx(): + """Free-text license blobs fall back to whole-string substring matching.""" + free_text = "MIT License AND additional redistribution permissions" + assert check_licenses.LicenseChecker._split_spdx_expression(free_text) is None + checker = _make_checker() + assert checker.is_license_acceptable(free_text)[0] is True + + +def test_unknown_license_is_reported(): + checker = _make_checker() + is_ok, reason = checker.is_license_acceptable(None) + assert is_ok is False + assert reason == "Unknown license" + + +# -------------------------------------------------------------------------- +# check_package: end-to-end resolution + acceptability +# -------------------------------------------------------------------------- + + +def test_check_package_accepts_pep639_package(monkeypatch): + """A PEP 639 package whose license lives only in license_expression passes.""" + _patch_pypi( + monkeypatch, + {"license_expression": "MIT", "license": None, "classifiers": []}, + ) + checker = _make_checker() + assert checker.check_package("some-pep639-pkg", "1.0.0") is True + + +def test_check_package_rejects_package_without_license(monkeypatch): + _patch_pypi( + monkeypatch, + {"license_expression": None, "license": None, "classifiers": []}, + ) + checker = _make_checker() + assert checker.check_package("mystery-pkg", "1.0.0") is False From f62ae93e13ce411ebc3f5879a3bd2e30fad993e4 Mon Sep 17 00:00:00 2001 From: yuneng-jiang Date: Fri, 22 May 2026 11:24:41 -0700 Subject: [PATCH 06/13] test(proxy): behavior-pinning matrix for tier-2/3 key + team management endpoints (#28620) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * test(proxy): add create_scratch_actor harness helper Adds create_scratch_actor() to the management behavior-suite conftest and extends create_scratch_team() with team_member_permissions / models kwargs, needed by the PR3 team-key-permission and team-model matrices. The new helper mints a scratch-prefixed user + verification token (+ org memberships), all reclaimed by the existing scratch-prefix teardown. * test(proxy): pin /key block, unblock, health, aliases behavior Adds behavior-pinning matrices for POST /key/block, POST /key/unblock, POST /key/health, and GET /key/aliases. Pins that the management-route gate 401s ORG_ADMIN-role callers before _check_key_admin_access runs, the block/unblock round-trip on the blocked column, missing-key 404, and the _apply_non_admin_alias_scope visibility rules for /key/aliases. * test(proxy): pin /key/bulk_update + /team/key/bulk_update behavior Adds behavior-pinning matrices for POST /key/bulk_update (PROXY_ADMIN-only; ORG_ADMIN stopped 401 at the route gate, INTERNAL_USER-role 403 at the handler) and POST /team/key/bulk_update (team-member-permission gate keyed on KEY_UPDATE). Pins batch semantics: empty/over-cap 400, per-key failure isolation into failed_updates, all_keys_in_team broadcast, and no-keys 404. Adds an optional key_alias arg to create_scratch_key for multi-key scenarios. * test(proxy): pin /key SA-generate, v2-info, reset-spend behavior Adds behavior-pinning matrices for POST /key/service-account/generate (team-membership + team-member-permission gating; SA keys carry no user_id), POST /v2/key/info (per-key _can_user_query_key_info silently drops invisible keys), and POST /key/{key}/reset_spend (PROXY_ADMIN or team admin only; missing key 404, reset-value 400). Pins that ORG_ADMIN-role callers are stopped 401 at the management-route gate on the two non-info routes. * test(proxy): close PR1/PR2 key-side deferred coverage gaps Closes the four key-side gaps deferred from PR1/PR2: - 404 on missing key for /key/update and /key/delete (not 401/403) - denied /key/update leaves max_budget/tpm_limit/rpm_limit untouched - /key/regenerate enforces litellm.upperbound_key_generate_params (#26340) - /key/list key_alias substring vs exact (admin-only) + team_id filter, and a non-admin filtering a foreign team is 403 * test(proxy): pin /team block, unblock, available, filter/ui, members/me Adds behavior-pinning matrices for POST /team/block + /team/unblock (management-route gate fronts _verify_team_access; reachable only by PROXY_ADMIN and an org admin of the team's own org), GET /team/available (default empty path), GET /team/filter/ui (route-gated PROXY-ADMIN-only despite the handler having no gate), and GET /team/{team_id}/members/me (caller resolves its own membership; non-member 404, no-user_id key 400). * test(proxy): pin /team model add/delete + permissions endpoints Adds behavior-pinning matrices for POST /team/model/add + /team/model/delete (route-gated PROXY-ADMIN-only; missing team 404), GET /team/permissions_list + POST /team/permissions_update (self-managed; proxy/team/org admin pass), and POST /team/permissions_bulk_update (PROXY_ADMIN-only). Pins the deliberate divergence that the available-team self-join grants read access via permissions_list but never write access via permissions_update. * test(proxy): pin /team delete, bulk_member_add, v2/list, daily/activity Adds behavior-pinning matrices for POST /team/delete (per-team _verify_team_access; batch aborts whole on a missing id), POST /team/bulk_member_add (route-gated PROXY-ADMIN-only; empty/over-cap 400), GET /v2/team/list (_enforce_list_team_v2_access — bare query 401s regular users, org-scoped for org admins) and GET /team/daily/activity (non-member team_ids filter 404, the VERIA-43 fix). * test(proxy): add route-coverage gate + close team org-relocation gap Adds test_route_coverage.py (PR3.M1): parses every @router route literal from the two management-endpoint source files and asserts each is exercised by >=1 behavior-suite scenario — a permanent regression guard for future routes. Closes the last PR1/PR2 deferred gap: the /team/update org-relocation allowed branch, exercised by a dual-org-admin minted via create_scratch_actor. test_team_model uses literal route URLs so the coverage parser resolves them. * test(proxy): bound plain route params to one path segment in coverage gate Plain path params ({team_id}) now compile to [^/?]+ instead of [^?]+, so a parameter cannot span '/'. Starlette ':path' params still match across '/'. Keeps the route-coverage guard from falsely reporting a future multi-segment route as covered. All 37 routes remain covered. --- tests/proxy_behavior/management/conftest.py | 77 ++++++- .../management/test_key_aliases.py | 119 ++++++++++ .../management/test_key_block_unblock.py | 159 +++++++++++++ .../management/test_key_bulk_update.py | 123 ++++++++++ .../management/test_key_delete.py | 12 + .../management/test_key_health.py | 24 ++ .../management/test_key_info_v2.py | 82 +++++++ .../management/test_key_list.py | 110 ++++++++- .../management/test_key_regenerate.py | 48 ++++ .../management/test_key_reset_spend.py | 136 +++++++++++ .../test_key_service_account_generate.py | 98 ++++++++ .../management/test_key_update.py | 84 +++++++ .../management/test_route_coverage.py | 91 ++++++++ .../management/test_scratch_teardown.py | 42 +++- .../management/test_team_available.py | 21 ++ .../management/test_team_block_unblock.py | 114 +++++++++ .../management/test_team_bulk_member_add.py | 105 +++++++++ .../management/test_team_daily_activity.py | 63 +++++ .../management/test_team_delete.py | 78 +++++++ .../management/test_team_filter_ui.py | 39 ++++ .../management/test_team_key_bulk_update.py | 217 ++++++++++++++++++ .../management/test_team_list_v2.py | 141 ++++++++++++ .../management/test_team_member_me.py | 83 +++++++ .../management/test_team_model.py | 78 +++++++ .../management/test_team_permissions.py | 170 ++++++++++++++ .../management/test_team_update.py | 39 +++- 26 files changed, 2342 insertions(+), 11 deletions(-) create mode 100644 tests/proxy_behavior/management/test_key_aliases.py create mode 100644 tests/proxy_behavior/management/test_key_block_unblock.py create mode 100644 tests/proxy_behavior/management/test_key_bulk_update.py create mode 100644 tests/proxy_behavior/management/test_key_health.py create mode 100644 tests/proxy_behavior/management/test_key_info_v2.py create mode 100644 tests/proxy_behavior/management/test_key_reset_spend.py create mode 100644 tests/proxy_behavior/management/test_key_service_account_generate.py create mode 100644 tests/proxy_behavior/management/test_route_coverage.py create mode 100644 tests/proxy_behavior/management/test_team_available.py create mode 100644 tests/proxy_behavior/management/test_team_block_unblock.py create mode 100644 tests/proxy_behavior/management/test_team_bulk_member_add.py create mode 100644 tests/proxy_behavior/management/test_team_daily_activity.py create mode 100644 tests/proxy_behavior/management/test_team_delete.py create mode 100644 tests/proxy_behavior/management/test_team_filter_ui.py create mode 100644 tests/proxy_behavior/management/test_team_key_bulk_update.py create mode 100644 tests/proxy_behavior/management/test_team_list_v2.py create mode 100644 tests/proxy_behavior/management/test_team_member_me.py create mode 100644 tests/proxy_behavior/management/test_team_model.py create mode 100644 tests/proxy_behavior/management/test_team_permissions.py diff --git a/tests/proxy_behavior/management/conftest.py b/tests/proxy_behavior/management/conftest.py index 3432f4ad6cf..fa0bef86280 100644 --- a/tests/proxy_behavior/management/conftest.py +++ b/tests/proxy_behavior/management/conftest.py @@ -11,6 +11,7 @@ import pytest_asyncio import yaml from prisma import Json +from litellm.proxy.utils import hash_token MASTER_KEY = "sk-1234" SCRATCH_PREFIX = "scratch-" @@ -106,12 +107,19 @@ async def create_scratch_key( user_id: str, team_id: Optional[str] = None, organization_id: Optional[str] = None, + key_alias: Optional[str] = None, ) -> str: """Seed a scratch-tagged key via /key/generate; returns its cleartext. Shared by the write-scenario matrices (key update/regenerate/delete). + key_alias defaults to scratch_prefix; pass a distinct scratch-prefixed + alias when a single scenario needs more than one key (/key/generate + enforces unique aliases). """ - body: Dict[str, Any] = {"key_alias": scratch_prefix, "user_id": user_id} + body: Dict[str, Any] = { + "key_alias": key_alias or scratch_prefix, + "user_id": user_id, + } if team_id is not None: body["team_id"] = team_id if organization_id is not None: @@ -132,6 +140,8 @@ async def create_scratch_team( organization_id: Optional[str] = None, admin_user_ids: Optional[list] = None, member_user_ids: Optional[list] = None, + team_member_permissions: Optional[list] = None, + models: Optional[list] = None, ) -> str: """Raw-seed a scratch-tagged team row; returns its team_id. @@ -142,6 +152,9 @@ async def create_scratch_team( members_with_roles JSON, so a raw-seeded team exercises them exactly as a /team/new-created team would. team_id must start with the scratch prefix so the `scratch` fixture reclaims the row. + + team_member_permissions / models seed the matching raw columns — needed + by the team-key-permission and team-model matrices. """ admin_user_ids = list(admin_user_ids or []) member_user_ids = list(member_user_ids or []) @@ -157,10 +170,72 @@ async def create_scratch_team( } if organization_id is not None: data["organization_id"] = organization_id + if team_member_permissions is not None: + data["team_member_permissions"] = team_member_permissions + if models is not None: + data["models"] = models await prisma.db.litellm_teamtable.create(data=data) return team_id +@dataclass(frozen=True) +class SeededActor: + user_id: str + cleartext: str + hashed: str + + +async def create_scratch_actor( + prisma, + scratch_prefix: str, + *, + user_role: str, + org_admin_of: tuple = (), + organization_id: Optional[str] = None, + suffix: str = "actor", +) -> SeededActor: + """Mint a scratch-prefixed user + verification token (+ org memberships). + + Reclaimed by the existing `scratch` teardown, which sweeps + litellm_usertable, litellm_verificationtoken, and + litellm_organizationmembership by scratch prefix — no bespoke cleanup + needed. Does NOT write litellm_teammembership against world teams: the + teardown reclaims that table only by team_id prefix, so a scratch actor + needing team membership must join a scratch team instead. The cleartext + is hashed with the real hash_token so the key authenticates end-to-end; + models=[] satisfies LiteLLM_VerificationTokenView. + """ + user_id = f"{scratch_prefix}-{suffix}" + cleartext = "sk-" + uuid.uuid4().hex + hashed = hash_token(cleartext) + await prisma.db.litellm_usertable.create( + data={ + "user_id": user_id, + "user_role": user_role, + "organization_id": organization_id, + } + ) + token_data: Dict[str, Any] = { + "token": hashed, + "key_name": f"{scratch_prefix}-{suffix}-key", + "key_alias": f"{scratch_prefix}-{suffix}-alias", + "user_id": user_id, + "models": [], + } + if organization_id is not None: + token_data["organization_id"] = organization_id + await prisma.db.litellm_verificationtoken.create(data=token_data) + for org_id in org_admin_of: + await prisma.db.litellm_organizationmembership.create( + data={ + "user_id": user_id, + "organization_id": org_id, + "user_role": "org_admin", + } + ) + return SeededActor(user_id=user_id, cleartext=cleartext, hashed=hashed) + + @pytest_asyncio.fixture async def scratch(prisma): handle = Scratch(prefix=f"{SCRATCH_PREFIX}{uuid.uuid4().hex[:12]}") diff --git a/tests/proxy_behavior/management/test_key_aliases.py b/tests/proxy_behavior/management/test_key_aliases.py new file mode 100644 index 00000000000..38ce5cdfaf3 --- /dev/null +++ b/tests/proxy_behavior/management/test_key_aliases.py @@ -0,0 +1,119 @@ +import uuid +from typing import FrozenSet + +import pytest + +from litellm.proxy.utils import hash_token + +from .actors import TEAM_ALPHA, TEAM_BETA, Actor + +pytestmark = pytest.mark.asyncio(loop_scope="session") + + +# GET /key/aliases scopes non-admins via _apply_non_admin_alias_scope: a +# non-admin sees an alias only if it owns the key (user_id match) or the key +# belongs to one of its teams. PROXY_ADMIN sees every alias. The seeded keys: +# own — owned by INTERNAL_USER, no team -> user_id scope only +# alpha — owned by OWNER, team TEAM_ALPHA -> team scope for alpha members +# beta — owned by CROSS_ORG_USER, TEAM_BETA +async def _seed_alias_keys(prisma, prefix: str, world) -> dict: + spec = { + "own": (Actor.INTERNAL_USER, None), + "alpha": (Actor.OWNER, TEAM_ALPHA), + "beta": (Actor.CROSS_ORG_USER, TEAM_BETA), + } + out = {} + for tag, (owner, team_id) in spec.items(): + alias = f"{prefix}-{tag}" + data = { + "token": hash_token("sk-" + uuid.uuid4().hex), + "key_name": f"{prefix}-{tag}-key", + "key_alias": alias, + "user_id": world.keys[owner].user_id, + "models": [], + } + if team_id is not None: + data["team_id"] = team_id + await prisma.db.litellm_verificationtoken.create(data=data) + out[tag] = alias + return out + + +async def _fetch_aliases(proxy_client, caller_cleartext: str, query: str) -> set: + resp = await proxy_client.get( + f"/key/aliases?{query}&size=100", + headers={"Authorization": f"Bearer {caller_cleartext}"}, + ) + assert resp.status_code == 200, resp.text + return set(resp.json()["aliases"]) + + +# ORG_ADMIN-role callers are stopped 401 by the management-route gate before +# the handler runs — /key/aliases carries no org context. Every other actor +# reaches the handler and is scoped by _apply_non_admin_alias_scope. +_VISIBILITY = { + Actor.PROXY_ADMIN: (200, frozenset({"own", "alpha", "beta"})), + Actor.ORG_ADMIN: (401, None), + Actor.TEAM_ADMIN: (200, frozenset({"alpha"})), + Actor.INTERNAL_USER: (200, frozenset({"own", "alpha"})), + Actor.OWNER: (200, frozenset({"alpha"})), + Actor.UNRELATED_SAME_ORG: (200, frozenset({"alpha"})), + Actor.CROSS_ORG_USER: (200, frozenset({"beta"})), + Actor.SERVICE_ACCOUNT: (200, frozenset({"alpha"})), + Actor.ORG_B_ADMIN: (401, None), +} + + +@pytest.mark.parametrize( + "actor,expected_status,expected_tags", + [(a, s, t) for a, (s, t) in _VISIBILITY.items()], + ids=[a.value for a in _VISIBILITY], +) +async def test_key_aliases_visibility( + actor: Actor, + expected_status: int, + expected_tags: FrozenSet[str], + proxy_client, + prisma, + scratch, + world, +): + aliases = await _seed_alias_keys(prisma, scratch.prefix, world) + known = {v: k for k, v in aliases.items()} + + resp = await proxy_client.get( + f"/key/aliases?search={scratch.prefix}&size=100", + headers={"Authorization": f"Bearer {world.keys[actor].cleartext}"}, + ) + assert ( + resp.status_code == expected_status + ), f"{actor.value}: {resp.status_code} {resp.text}" + if expected_status != 200: + return + + visible = {known[a] for a in resp.json()["aliases"] if a in known} + assert visible == set( + expected_tags + ), f"{actor.value}: expected {sorted(expected_tags)}, got {sorted(visible)}" + + +async def test_key_aliases_team_id_filter(proxy_client, prisma, scratch, world): + """team_id filter narrows the result to keys of that team.""" + aliases = await _seed_alias_keys(prisma, scratch.prefix, world) + returned = await _fetch_aliases( + proxy_client, + world.keys[Actor.PROXY_ADMIN].cleartext, + f"search={scratch.prefix}&team_id={TEAM_ALPHA}", + ) + assert returned & set(aliases.values()) == {aliases["alpha"]} + + +async def test_key_aliases_search_filter(proxy_client, prisma, scratch, world): + """search is a case-insensitive substring match on key_alias.""" + aliases = await _seed_alias_keys(prisma, scratch.prefix, world) + returned = await _fetch_aliases( + proxy_client, + world.keys[Actor.PROXY_ADMIN].cleartext, + f"search={aliases['beta']}", + ) + assert returned & set(aliases.values()) == {aliases["beta"]} diff --git a/tests/proxy_behavior/management/test_key_block_unblock.py b/tests/proxy_behavior/management/test_key_block_unblock.py new file mode 100644 index 00000000000..37aa0c0219a --- /dev/null +++ b/tests/proxy_behavior/management/test_key_block_unblock.py @@ -0,0 +1,159 @@ +import uuid + +import pytest + +from litellm.proxy.utils import hash_token + +from .actors import TEAM_ALPHA, TEAM_BETA, Actor +from .conftest import create_scratch_key + +pytestmark = pytest.mark.asyncio(loop_scope="session") + + +# POST /key/block + /key/unblock. PROXY_ADMIN bypasses. ORG_ADMIN-role callers +# are stopped 401 by the management-route gate BEFORE the handler runs — the +# body carries no organization_id, so the gate has no org context and falls +# back to proxy-admin-only. The handler's own _check_key_admin_access org-admin +# branch is therefore unreachable via these routes. INTERNAL_USER-role callers +# do reach _check_key_admin_access: a team admin of the key's team passes (200); +# everyone else (incl. a teamless "self" key with no team to admin) is 403. +_SCENARIOS = [ + ("self/proxy_admin", Actor.PROXY_ADMIN, "self", 200), + ("self/org_admin", Actor.ORG_ADMIN, "self", 401), + ("self/team_admin", Actor.TEAM_ADMIN, "self", 403), + ("self/internal_user", Actor.INTERNAL_USER, "self", 403), + ("self/cross_org_user", Actor.CROSS_ORG_USER, "self", 403), + ("owner/proxy_admin", Actor.PROXY_ADMIN, "owner", 200), + ("owner/org_admin", Actor.ORG_ADMIN, "owner", 401), + ("owner/team_admin", Actor.TEAM_ADMIN, "owner", 200), + ("owner/internal_user", Actor.INTERNAL_USER, "owner", 403), + ("owner/owner", Actor.OWNER, "owner", 403), + ("owner/unrelated_same_org", Actor.UNRELATED_SAME_ORG, "owner", 403), + ("owner/cross_org_user", Actor.CROSS_ORG_USER, "owner", 403), + ("owner/service_account", Actor.SERVICE_ACCOUNT, "owner", 403), + ("owner/org_b_admin", Actor.ORG_B_ADMIN, "owner", 401), + ("cross_org/proxy_admin", Actor.PROXY_ADMIN, "cross_org", 200), + ("cross_org/org_admin", Actor.ORG_ADMIN, "cross_org", 401), + ("cross_org/team_admin", Actor.TEAM_ADMIN, "cross_org", 403), + ("cross_org/cross_org_user", Actor.CROSS_ORG_USER, "cross_org", 403), + ("cross_org/org_b_admin", Actor.ORG_B_ADMIN, "cross_org", 401), +] + + +async def _seed_target(proxy_client, seeder, scratch_prefix, world, shape, caller): + if shape == "self": + return await create_scratch_key( + proxy_client, seeder, scratch_prefix, user_id=caller.user_id + ) + if shape == "owner": + return await create_scratch_key( + proxy_client, + seeder, + scratch_prefix, + user_id=world.keys[Actor.OWNER].user_id, + team_id=TEAM_ALPHA, + ) + if shape == "cross_org": + return await create_scratch_key( + proxy_client, + seeder, + scratch_prefix, + user_id=world.keys[Actor.CROSS_ORG_USER].user_id, + team_id=TEAM_BETA, + ) + pytest.fail(f"unknown shape={shape}") # pragma: no cover + + +@pytest.mark.parametrize("route", ["block", "unblock"]) +@pytest.mark.parametrize( + "actor,shape,expected_status", + [(a, sh, s) for (_id, a, sh, s) in _SCENARIOS], + ids=[s[0] for s in _SCENARIOS], +) +async def test_key_block_unblock_authz_matrix( + route: str, + actor: Actor, + shape: str, + expected_status: int, + proxy_client, + prisma, + scratch, + world, +): + caller = world.keys[actor] + seeder = world.keys[Actor.PROXY_ADMIN].cleartext + target_cleartext = await _seed_target( + proxy_client, seeder, scratch.prefix, world, shape, caller + ) + target_hashed = hash_token(target_cleartext) + + # /unblock starts from a blocked row so a 200 is observable as True->False. + if route == "unblock": + await prisma.db.litellm_verificationtoken.update( + where={"token": target_hashed}, data={"blocked": True} + ) + + resp = await proxy_client.post( + f"/key/{route}", + headers={"Authorization": f"Bearer {caller.cleartext}"}, + json={"key": target_cleartext}, + ) + assert ( + resp.status_code == expected_status + ), f"{route} {actor.value} {shape}: {resp.status_code} {resp.text}" + + row = await prisma.db.litellm_verificationtoken.find_unique( + where={"token": target_hashed} + ) + assert row is not None + # A never-blocked key reads back blocked=None; treat that as not-blocked. + if expected_status == 200: + assert bool(row.blocked) is (route == "block") + else: + # A denial leaves the blocked column at its pre-request value. + assert bool(row.blocked) is (route == "unblock"), "denied but blocked mutated" + + +async def test_key_block_unblock_round_trip(proxy_client, prisma, scratch, world): + """PROXY_ADMIN block then unblock flips the blocked column True then False.""" + admin = world.keys[Actor.PROXY_ADMIN] + target = await create_scratch_key( + proxy_client, admin.cleartext, scratch.prefix, user_id=admin.user_id + ) + hashed = hash_token(target) + headers = {"Authorization": f"Bearer {admin.cleartext}"} + + blocked = await proxy_client.post( + "/key/block", headers=headers, json={"key": target} + ) + assert blocked.status_code == 200, blocked.text + row = await prisma.db.litellm_verificationtoken.find_unique(where={"token": hashed}) + assert row is not None and row.blocked is True + + unblocked = await proxy_client.post( + "/key/unblock", headers=headers, json={"key": target} + ) + assert unblocked.status_code == 200, unblocked.text + row = await prisma.db.litellm_verificationtoken.find_unique(where={"token": hashed}) + assert row is not None and row.blocked is False + + +@pytest.mark.parametrize("route", ["block", "unblock"]) +@pytest.mark.parametrize( + "actor", [Actor.PROXY_ADMIN, Actor.TEAM_ADMIN], ids=["proxy_admin", "team_admin"] +) +async def test_key_block_unblock_missing_key_returns_404( + route: str, actor: Actor, proxy_client, world +): + """A well-formed but unseeded key is 404 — not 401/403 — for both the + PROXY_ADMIN existence check and the non-admin _check_key_admin_access path.""" + caller = world.keys[actor] + missing = "sk-" + uuid.uuid4().hex + resp = await proxy_client.post( + f"/key/{route}", + headers={"Authorization": f"Bearer {caller.cleartext}"}, + json={"key": missing}, + ) + assert ( + resp.status_code == 404 + ), f"{route} {actor.value}: {resp.status_code} {resp.text}" diff --git a/tests/proxy_behavior/management/test_key_bulk_update.py b/tests/proxy_behavior/management/test_key_bulk_update.py new file mode 100644 index 00000000000..1a57998cece --- /dev/null +++ b/tests/proxy_behavior/management/test_key_bulk_update.py @@ -0,0 +1,123 @@ +import uuid + +import pytest + +from litellm.proxy.utils import hash_token + +from .actors import Actor +from .conftest import create_scratch_key + +pytestmark = pytest.mark.asyncio(loop_scope="session") + +_MARKER_BUDGET = 42.0 + + +# POST /key/bulk_update is PROXY_ADMIN-only. The handler's own gate is +# user_role != PROXY_ADMIN -> 403, but ORG_ADMIN-role callers never reach it: +# the management-route gate 401s them first (the body carries no org context, +# and /key/bulk_update is an internal_user route, not an org-admin one). +# INTERNAL_USER-role callers clear the route gate and hit the handler's 403. +_MATRIX = [ + ("proxy_admin", Actor.PROXY_ADMIN, 200), + ("org_admin", Actor.ORG_ADMIN, 401), + ("team_admin", Actor.TEAM_ADMIN, 403), + ("internal_user", Actor.INTERNAL_USER, 403), + ("owner", Actor.OWNER, 403), + ("unrelated_same_org", Actor.UNRELATED_SAME_ORG, 403), + ("cross_org_user", Actor.CROSS_ORG_USER, 403), + ("service_account", Actor.SERVICE_ACCOUNT, 403), +] + + +@pytest.mark.parametrize( + "actor,expected_status", + [(a, s) for (_id, a, s) in _MATRIX], + ids=[s[0] for s in _MATRIX], +) +async def test_key_bulk_update_authz_matrix( + actor: Actor, + expected_status: int, + proxy_client, + prisma, + scratch, + world, +): + caller = world.keys[actor] + seeder = world.keys[Actor.PROXY_ADMIN].cleartext + target = await create_scratch_key( + proxy_client, seeder, scratch.prefix, user_id=caller.user_id + ) + hashed = hash_token(target) + + resp = await proxy_client.post( + "/key/bulk_update", + headers={"Authorization": f"Bearer {caller.cleartext}"}, + json={"keys": [{"key": target, "max_budget": _MARKER_BUDGET}]}, + ) + assert ( + resp.status_code == expected_status + ), f"{actor.value}: {resp.status_code} {resp.text}" + + row = await prisma.db.litellm_verificationtoken.find_unique(where={"token": hashed}) + assert row is not None + if expected_status == 200: + body = resp.json() + assert len(body["successful_updates"]) == 1 + assert body["failed_updates"] == [] + assert row.max_budget == _MARKER_BUDGET + else: + assert row.max_budget != _MARKER_BUDGET, "denied but key mutated" + + +async def test_key_bulk_update_empty_keys_is_400(proxy_client, world): + """An empty batch is rejected 400 before any per-key processing.""" + resp = await proxy_client.post( + "/key/bulk_update", + headers={"Authorization": f"Bearer {world.keys[Actor.PROXY_ADMIN].cleartext}"}, + json={"keys": []}, + ) + assert resp.status_code == 400, resp.text + + +async def test_key_bulk_update_over_max_batch_is_400(proxy_client, world): + """A batch larger than the 500-key cap is rejected 400.""" + items = [{"key": "sk-" + uuid.uuid4().hex} for _ in range(501)] + resp = await proxy_client.post( + "/key/bulk_update", + headers={"Authorization": f"Bearer {world.keys[Actor.PROXY_ADMIN].cleartext}"}, + json={"keys": items}, + ) + assert resp.status_code == 400, resp.text + + +async def test_key_bulk_update_per_key_failure_is_isolated( + proxy_client, prisma, scratch, world +): + """One bad key in the batch does not abort the others — it lands in + failed_updates while the valid key is still updated.""" + admin = world.keys[Actor.PROXY_ADMIN] + valid = await create_scratch_key( + proxy_client, admin.cleartext, scratch.prefix, user_id=admin.user_id + ) + missing = "sk-" + uuid.uuid4().hex + + resp = await proxy_client.post( + "/key/bulk_update", + headers={"Authorization": f"Bearer {admin.cleartext}"}, + json={ + "keys": [ + {"key": valid, "max_budget": _MARKER_BUDGET}, + {"key": missing, "max_budget": _MARKER_BUDGET}, + ] + }, + ) + assert resp.status_code == 200, resp.text + body = resp.json() + assert body["total_requested"] == 2 + assert len(body["successful_updates"]) == 1 + assert len(body["failed_updates"]) == 1 + + row = await prisma.db.litellm_verificationtoken.find_unique( + where={"token": hash_token(valid)} + ) + assert row is not None and row.max_budget == _MARKER_BUDGET diff --git a/tests/proxy_behavior/management/test_key_delete.py b/tests/proxy_behavior/management/test_key_delete.py index 05844ac0031..0b483edc056 100644 --- a/tests/proxy_behavior/management/test_key_delete.py +++ b/tests/proxy_behavior/management/test_key_delete.py @@ -1,3 +1,5 @@ +import uuid + import pytest from litellm.proxy.utils import hash_token @@ -99,3 +101,13 @@ async def test_key_delete_authz_matrix( else: assert row is not None, f"{actor.value}: denied but row vanished" assert auth_check.status_code == 200 + + +async def test_key_delete_missing_key_is_404(proxy_client, world): + """Deleting a key absent from the DB is a 404 — not 401/403.""" + resp = await proxy_client.post( + "/key/delete", + headers={"Authorization": f"Bearer {world.keys[Actor.PROXY_ADMIN].cleartext}"}, + json={"keys": ["sk-" + uuid.uuid4().hex]}, + ) + assert resp.status_code == 404, resp.text diff --git a/tests/proxy_behavior/management/test_key_health.py b/tests/proxy_behavior/management/test_key_health.py new file mode 100644 index 00000000000..62147e7fa13 --- /dev/null +++ b/tests/proxy_behavior/management/test_key_health.py @@ -0,0 +1,24 @@ +import pytest + +from .actors import Actor + +pytestmark = pytest.mark.asyncio(loop_scope="session") + + +# POST /key/health has no role gate — it reflects the caller's OWN key logging +# metadata. The world keys carry no "logging" metadata, so every authenticated +# actor gets 200 with key="healthy". This pins auth-required + route coverage. +@pytest.mark.parametrize("actor", list(Actor), ids=[a.value for a in Actor]) +async def test_key_health_each_actor_is_healthy(actor: Actor, proxy_client, world): + caller = world.keys[actor] + resp = await proxy_client.post( + "/key/health", + headers={"Authorization": f"Bearer {caller.cleartext}"}, + ) + assert resp.status_code == 200, f"{actor.value}: {resp.status_code} {resp.text}" + assert resp.json()["key"] == "healthy" + + +async def test_key_health_requires_auth(proxy_client): + resp = await proxy_client.post("/key/health") + assert resp.status_code == 401, resp.text diff --git a/tests/proxy_behavior/management/test_key_info_v2.py b/tests/proxy_behavior/management/test_key_info_v2.py new file mode 100644 index 00000000000..b0fb27a19fa --- /dev/null +++ b/tests/proxy_behavior/management/test_key_info_v2.py @@ -0,0 +1,82 @@ +import uuid + +import pytest + +from .actors import Actor + +pytestmark = pytest.mark.asyncio(loop_scope="session") + + +# POST /v2/key/info resolves the posted keys, then drops any key the caller +# cannot see via _can_user_query_key_info — silently, no 403. A non-admin sees +# a key it owns (user_id match) or a key whose team it belongs to. The world's +# TEAM_ALPHA members all see each other's keys; CROSS_ORG_USER and the org +# admins see only their own. The request is posted with every world key, and +# the returned info set is asserted to equal the visible subset. +_ALPHA_KEYS = frozenset( + { + Actor.TEAM_ADMIN, + Actor.INTERNAL_USER, + Actor.OWNER, + Actor.UNRELATED_SAME_ORG, + Actor.SERVICE_ACCOUNT, + } +) +_VISIBILITY = { + Actor.PROXY_ADMIN: frozenset(Actor), + Actor.ORG_ADMIN: frozenset({Actor.ORG_ADMIN}), + Actor.TEAM_ADMIN: _ALPHA_KEYS, + Actor.INTERNAL_USER: _ALPHA_KEYS, + Actor.OWNER: _ALPHA_KEYS, + Actor.UNRELATED_SAME_ORG: _ALPHA_KEYS, + Actor.SERVICE_ACCOUNT: _ALPHA_KEYS, + Actor.CROSS_ORG_USER: frozenset({Actor.CROSS_ORG_USER}), + Actor.ORG_B_ADMIN: frozenset({Actor.ORG_B_ADMIN}), +} + + +@pytest.mark.parametrize( + "actor,expected_visible", + list(_VISIBILITY.items()), + ids=[a.value for a in _VISIBILITY], +) +async def test_key_info_v2_visibility(actor, expected_visible, proxy_client, world): + caller = world.keys[actor] + user_id_to_actor = {world.keys[a].user_id: a for a in Actor} + + resp = await proxy_client.post( + "/v2/key/info", + headers={"Authorization": f"Bearer {caller.cleartext}"}, + json={"keys": [world.keys[a].cleartext for a in Actor]}, + ) + assert resp.status_code == 200, f"{actor.value}: {resp.status_code} {resp.text}" + + visible = { + user_id_to_actor[entry["user_id"]] + for entry in resp.json()["info"] + if entry.get("user_id") in user_id_to_actor + } + assert visible == set(expected_visible), ( + f"{actor.value}: expected {sorted(a.value for a in expected_visible)}, " + f"got {sorted(a.value for a in visible)}" + ) + + +async def test_key_info_v2_no_body_is_422(proxy_client, world): + """A request with no body is a 422 — the handler has no keys to resolve.""" + resp = await proxy_client.post( + "/v2/key/info", + headers={"Authorization": f"Bearer {world.keys[Actor.PROXY_ADMIN].cleartext}"}, + ) + assert resp.status_code == 422, resp.text + + +async def test_key_info_v2_unknown_key_returns_empty_info(proxy_client, world): + """Keys that resolve to no rows yield an empty info list, not an error.""" + resp = await proxy_client.post( + "/v2/key/info", + headers={"Authorization": f"Bearer {world.keys[Actor.PROXY_ADMIN].cleartext}"}, + json={"keys": ["sk-" + uuid.uuid4().hex]}, + ) + assert resp.status_code == 200, resp.text + assert resp.json()["info"] == [] diff --git a/tests/proxy_behavior/management/test_key_list.py b/tests/proxy_behavior/management/test_key_list.py index bda8788c9a7..0ed101d5868 100644 --- a/tests/proxy_behavior/management/test_key_list.py +++ b/tests/proxy_behavior/management/test_key_list.py @@ -2,7 +2,10 @@ from typing import FrozenSet import pytest -from .actors import Actor +from litellm.proxy.utils import hash_token + +from .actors import TEAM_ALPHA, Actor +from .conftest import create_scratch_key pytestmark = pytest.mark.asyncio(loop_scope="session") @@ -61,3 +64,108 @@ async def test_key_list_visibility( f"{actor.value}: expected {sorted(a.value for a in expected_visible)}, " f"got {sorted(a.value for a in visible_seeded)}" ) + + +async def _list_hashes(proxy_client, caller_cleartext: str, query: str) -> set: + resp = await proxy_client.get( + f"/key/list?{query}&size=100", + headers={"Authorization": f"Bearer {caller_cleartext}"}, + ) + assert resp.status_code == 200, resp.text + hashes: set = set() + for entry in resp.json().get("keys", []): + tok = entry.get("token") if isinstance(entry, dict) else entry + if tok: + hashes.add(tok) + return hashes + + +async def test_key_list_admin_key_alias_substring_match(proxy_client, scratch, world): + """A PROXY_ADMIN's key_alias filter is a case-insensitive substring match; + a narrower fragment selects the subset whose alias contains it.""" + admin = world.keys[Actor.PROXY_ADMIN] + a = await create_scratch_key( + proxy_client, + admin.cleartext, + scratch.prefix, + user_id=admin.user_id, + key_alias=f"{scratch.prefix}-sub-a", + ) + b = await create_scratch_key( + proxy_client, + admin.cleartext, + scratch.prefix, + user_id=admin.user_id, + key_alias=f"{scratch.prefix}-sub-b", + ) + seeded = {hash_token(a), hash_token(b)} + + broad = await _list_hashes( + proxy_client, admin.cleartext, f"key_alias={scratch.prefix}-sub" + ) + assert broad & seeded == seeded + + narrow = await _list_hashes( + proxy_client, admin.cleartext, f"key_alias={scratch.prefix}-sub-a" + ) + assert narrow & seeded == {hash_token(a)} + + +async def test_key_list_non_admin_key_alias_is_exact_match( + proxy_client, scratch, world +): + """A non-admin's key_alias filter is exact-match only — substring filtering + is restricted to admins. The full alias matches; a fragment does not.""" + caller = world.keys[Actor.INTERNAL_USER] + alias = f"{scratch.prefix}-exact" + key = await create_scratch_key( + proxy_client, + world.keys[Actor.PROXY_ADMIN].cleartext, + scratch.prefix, + user_id=caller.user_id, + key_alias=alias, + ) + key_hash = hash_token(key) + + exact = await _list_hashes(proxy_client, caller.cleartext, f"key_alias={alias}") + assert key_hash in exact + + fragment = await _list_hashes( + proxy_client, caller.cleartext, f"key_alias={scratch.prefix}-exac" + ) + assert key_hash not in fragment + + +async def test_key_list_team_id_filter(proxy_client, scratch, world): + """A team_id filter narrows the listing to keys of that team.""" + admin = world.keys[Actor.PROXY_ADMIN] + team_key = await create_scratch_key( + proxy_client, + admin.cleartext, + scratch.prefix, + user_id=world.keys[Actor.OWNER].user_id, + team_id=TEAM_ALPHA, + key_alias=f"{scratch.prefix}-team", + ) + no_team_key = await create_scratch_key( + proxy_client, + admin.cleartext, + scratch.prefix, + user_id=admin.user_id, + key_alias=f"{scratch.prefix}-noteam", + ) + + hashes = await _list_hashes(proxy_client, admin.cleartext, f"team_id={TEAM_ALPHA}") + assert hash_token(team_key) in hashes + assert hash_token(no_team_key) not in hashes + + +async def test_key_list_non_admin_cannot_filter_other_team(proxy_client, world): + """A non-admin filtering by a team it does not belong to is rejected 403.""" + resp = await proxy_client.get( + f"/key/list?team_id={world.team_beta_id}", + headers={ + "Authorization": f"Bearer {world.keys[Actor.INTERNAL_USER].cleartext}" + }, + ) + assert resp.status_code == 403, resp.text diff --git a/tests/proxy_behavior/management/test_key_regenerate.py b/tests/proxy_behavior/management/test_key_regenerate.py index a3289144eef..724b8b6d65b 100644 --- a/tests/proxy_behavior/management/test_key_regenerate.py +++ b/tests/proxy_behavior/management/test_key_regenerate.py @@ -1,5 +1,10 @@ +import litellm import pytest +from litellm.types.proxy.management_endpoints.ui_sso import ( + LiteLLM_UpperboundKeyGenerateParams, +) + from .actors import TEAM_ALPHA, TEAM_BETA, Actor from .conftest import create_scratch_key @@ -115,3 +120,46 @@ async def test_key_path_regenerate_smoke(proxy_client, scratch, world): assert new_cleartext.startswith("sk-") and new_cleartext != target_cleartext assert (await _info(proxy_client, target_cleartext)).status_code == 401 assert (await _info(proxy_client, new_cleartext)).status_code == 200 + + +async def test_key_regenerate_enforces_upperbound_key_params( + proxy_client, scratch, world, monkeypatch +): + """Regenerate runs _enforce_upperbound_key_params: a max_budget above + litellm.upperbound_key_generate_params is rejected 400, a value within the + bound is accepted. Pins #26340 (db8ef44323) — regenerate previously + bypassed the upperbound. upperbound_key_generate_params is module-level + litellm.* state, so monkeypatch save/restores it.""" + admin = world.keys[Actor.PROXY_ADMIN] + over_key = await create_scratch_key( + proxy_client, + admin.cleartext, + scratch.prefix, + user_id=admin.user_id, + key_alias=f"{scratch.prefix}-over", + ) + within_key = await create_scratch_key( + proxy_client, + admin.cleartext, + scratch.prefix, + user_id=admin.user_id, + key_alias=f"{scratch.prefix}-within", + ) + monkeypatch.setattr( + litellm, + "upperbound_key_generate_params", + LiteLLM_UpperboundKeyGenerateParams(max_budget=100.0), + ) + headers = {"Authorization": f"Bearer {admin.cleartext}"} + + over = await proxy_client.post( + "/key/regenerate", headers=headers, json={"key": over_key, "max_budget": 500.0} + ) + assert over.status_code == 400, over.text + + within = await proxy_client.post( + "/key/regenerate", + headers=headers, + json={"key": within_key, "max_budget": 50.0}, + ) + assert within.status_code == 200, within.text diff --git a/tests/proxy_behavior/management/test_key_reset_spend.py b/tests/proxy_behavior/management/test_key_reset_spend.py new file mode 100644 index 00000000000..fb1c266f655 --- /dev/null +++ b/tests/proxy_behavior/management/test_key_reset_spend.py @@ -0,0 +1,136 @@ +import uuid + +import pytest + +from litellm.proxy.utils import hash_token + +from .actors import TEAM_ALPHA, TEAM_BETA, Actor +from .conftest import create_scratch_key + +pytestmark = pytest.mark.asyncio(loop_scope="session") + +_SEED_SPEND = 5.0 +_RESET_TO = 2.0 + + +# POST /key/{key}/reset_spend. The target key is pre-seeded with spend=5.0 so +# reset_to=2.0 always clears _validate_reset_spend_value (which runs before +# authz). _check_proxy_or_team_admin_for_key then allows only PROXY_ADMIN or a +# team admin of the key's team — there is no org-admin branch, and a teamless +# "self" key has no team to admin. ORG_ADMIN-role callers are stopped 401 at +# the management-route gate before the handler runs. +_SCENARIOS = [ + ("self/proxy_admin", Actor.PROXY_ADMIN, "self", 200), + ("self/org_admin", Actor.ORG_ADMIN, "self", 401), + ("self/team_admin", Actor.TEAM_ADMIN, "self", 403), + ("self/internal_user", Actor.INTERNAL_USER, "self", 403), + ("self/cross_org_user", Actor.CROSS_ORG_USER, "self", 403), + ("team_alpha/proxy_admin", Actor.PROXY_ADMIN, "team_alpha", 200), + ("team_alpha/org_admin", Actor.ORG_ADMIN, "team_alpha", 401), + ("team_alpha/team_admin", Actor.TEAM_ADMIN, "team_alpha", 200), + ("team_alpha/internal_user", Actor.INTERNAL_USER, "team_alpha", 403), + ("team_alpha/owner", Actor.OWNER, "team_alpha", 403), + ("team_alpha/unrelated_same_org", Actor.UNRELATED_SAME_ORG, "team_alpha", 403), + ("team_alpha/cross_org_user", Actor.CROSS_ORG_USER, "team_alpha", 403), + ("team_alpha/service_account", Actor.SERVICE_ACCOUNT, "team_alpha", 403), + ("team_alpha/org_b_admin", Actor.ORG_B_ADMIN, "team_alpha", 401), + ("team_beta/proxy_admin", Actor.PROXY_ADMIN, "team_beta", 200), + ("team_beta/org_admin", Actor.ORG_ADMIN, "team_beta", 401), + ("team_beta/team_admin", Actor.TEAM_ADMIN, "team_beta", 403), + ("team_beta/cross_org_user", Actor.CROSS_ORG_USER, "team_beta", 403), + ("team_beta/org_b_admin", Actor.ORG_B_ADMIN, "team_beta", 401), +] + + +async def _seed_target(proxy_client, seeder, prefix, world, shape, caller) -> str: + if shape == "self": + return await create_scratch_key( + proxy_client, seeder, prefix, user_id=caller.user_id + ) + if shape == "team_alpha": + return await create_scratch_key( + proxy_client, + seeder, + prefix, + user_id=world.keys[Actor.OWNER].user_id, + team_id=TEAM_ALPHA, + ) + if shape == "team_beta": + return await create_scratch_key( + proxy_client, + seeder, + prefix, + user_id=world.keys[Actor.CROSS_ORG_USER].user_id, + team_id=TEAM_BETA, + ) + pytest.fail(f"unknown shape={shape}") # pragma: no cover + + +@pytest.mark.parametrize( + "actor,shape,expected_status", + [(a, sh, s) for (_id, a, sh, s) in _SCENARIOS], + ids=[s[0] for s in _SCENARIOS], +) +async def test_key_reset_spend_authz_matrix( + actor: Actor, + shape: str, + expected_status: int, + proxy_client, + prisma, + scratch, + world, +): + caller = world.keys[actor] + seeder = world.keys[Actor.PROXY_ADMIN].cleartext + target = await _seed_target( + proxy_client, seeder, scratch.prefix, world, shape, caller + ) + hashed = hash_token(target) + await prisma.db.litellm_verificationtoken.update( + where={"token": hashed}, data={"spend": _SEED_SPEND} + ) + + resp = await proxy_client.post( + f"/key/{target}/reset_spend", + headers={"Authorization": f"Bearer {caller.cleartext}"}, + json={"reset_to": _RESET_TO}, + ) + assert ( + resp.status_code == expected_status + ), f"{actor.value} {shape}: {resp.status_code} {resp.text}" + + row = await prisma.db.litellm_verificationtoken.find_unique(where={"token": hashed}) + assert row is not None + if expected_status == 200: + assert row.spend == _RESET_TO + else: + assert row.spend == _SEED_SPEND, "denied but spend reset" + + +@pytest.mark.parametrize( + "actor", [Actor.PROXY_ADMIN, Actor.TEAM_ADMIN], ids=["proxy_admin", "team_admin"] +) +async def test_key_reset_spend_missing_key_is_404(actor: Actor, proxy_client, world): + """A well-formed but unseeded key is 404 before any spend validation.""" + resp = await proxy_client.post( + f"/key/sk-{uuid.uuid4().hex}/reset_spend", + headers={"Authorization": f"Bearer {world.keys[actor].cleartext}"}, + json={"reset_to": 0.0}, + ) + assert resp.status_code == 404, resp.text + + +async def test_key_reset_spend_above_current_spend_is_400( + proxy_client, prisma, scratch, world +): + """reset_to above the key's current spend is rejected 400.""" + admin = world.keys[Actor.PROXY_ADMIN] + target = await create_scratch_key( + proxy_client, admin.cleartext, scratch.prefix, user_id=admin.user_id + ) + resp = await proxy_client.post( + f"/key/{target}/reset_spend", + headers={"Authorization": f"Bearer {admin.cleartext}"}, + json={"reset_to": 1.0}, + ) + assert resp.status_code == 400, resp.text diff --git a/tests/proxy_behavior/management/test_key_service_account_generate.py b/tests/proxy_behavior/management/test_key_service_account_generate.py new file mode 100644 index 00000000000..3b5bbe39754 --- /dev/null +++ b/tests/proxy_behavior/management/test_key_service_account_generate.py @@ -0,0 +1,98 @@ +import pytest + +from .actors import Actor + +pytestmark = pytest.mark.asyncio(loop_scope="session") + + +# POST /key/service-account/generate. PROXY_ADMIN always passes. ORG_ADMIN-role +# callers are stopped 401 by the management-route gate (the body carries a +# team_id but no organization_id, so the org-admin route branch never matches). +# INTERNAL_USER-role callers reach the handler: a team admin of the target team +# passes (200); a "user"-role member is 401 (no service-account-generate +# permission); a non-member is 400 ("not assigned to team"). A request with no +# team_id is 400 ("team_id is required") for every actor that reaches the handler. +_SCENARIOS = [ + ("own/proxy_admin", Actor.PROXY_ADMIN, "own", 200), + ("own/org_admin", Actor.ORG_ADMIN, "own", 401), + ("own/team_admin", Actor.TEAM_ADMIN, "own", 200), + ("own/internal_user", Actor.INTERNAL_USER, "own", 401), + ("own/owner", Actor.OWNER, "own", 401), + ("own/unrelated_same_org", Actor.UNRELATED_SAME_ORG, "own", 401), + ("own/cross_org_user", Actor.CROSS_ORG_USER, "own", 400), + ("own/service_account", Actor.SERVICE_ACCOUNT, "own", 401), + ("own/org_b_admin", Actor.ORG_B_ADMIN, "own", 401), + ("cross_org/proxy_admin", Actor.PROXY_ADMIN, "cross_org", 200), + ("cross_org/org_admin", Actor.ORG_ADMIN, "cross_org", 401), + ("cross_org/team_admin", Actor.TEAM_ADMIN, "cross_org", 400), + ("cross_org/internal_user", Actor.INTERNAL_USER, "cross_org", 400), + ("cross_org/cross_org_user", Actor.CROSS_ORG_USER, "cross_org", 401), + ("cross_org/org_b_admin", Actor.ORG_B_ADMIN, "cross_org", 401), + ("none/proxy_admin", Actor.PROXY_ADMIN, "none", 400), + ("none/org_admin", Actor.ORG_ADMIN, "none", 401), + ("none/team_admin", Actor.TEAM_ADMIN, "none", 400), + ("none/internal_user", Actor.INTERNAL_USER, "none", 400), + ("none/cross_org_user", Actor.CROSS_ORG_USER, "none", 400), +] + + +@pytest.mark.parametrize( + "actor,team_target,expected_status", + [(a, t, s) for (_id, a, t, s) in _SCENARIOS], + ids=[s[0] for s in _SCENARIOS], +) +async def test_key_service_account_generate_authz_matrix( + actor: Actor, + team_target: str, + expected_status: int, + proxy_client, + prisma, + scratch, + world, +): + caller = world.keys[actor] + team_id = { + "own": world.team_alpha_id, + "cross_org": world.team_beta_id, + "none": None, + }[team_target] + + body = {"key_alias": scratch.prefix} + if team_id is not None: + body["team_id"] = team_id + + resp = await proxy_client.post( + "/key/service-account/generate", + headers={"Authorization": f"Bearer {caller.cleartext}"}, + json=body, + ) + assert ( + resp.status_code == expected_status + ), f"{actor.value} {team_target}: {resp.status_code} {resp.text}" + + rows = await prisma.db.litellm_verificationtoken.find_many( + where={"key_alias": scratch.prefix} + ) + if expected_status == 200: + assert len(rows) == 1 + # A service-account key belongs to the team, not a user. + assert rows[0].user_id is None + assert rows[0].team_id == team_id + else: + assert rows == [], f"{actor.value}: denied but key row leaked" + + +async def test_key_service_account_generate_unknown_team_is_400( + proxy_client, prisma, scratch, world +): + """A team_id absent from the database is rejected 400.""" + resp = await proxy_client.post( + "/key/service-account/generate", + headers={"Authorization": f"Bearer {world.keys[Actor.PROXY_ADMIN].cleartext}"}, + json={"key_alias": scratch.prefix, "team_id": scratch.tag("no-such-team")}, + ) + assert resp.status_code == 400, resp.text + rows = await prisma.db.litellm_verificationtoken.find_many( + where={"key_alias": scratch.prefix} + ) + assert rows == [] diff --git a/tests/proxy_behavior/management/test_key_update.py b/tests/proxy_behavior/management/test_key_update.py index 36ddefa5750..7b7f6f5558b 100644 --- a/tests/proxy_behavior/management/test_key_update.py +++ b/tests/proxy_behavior/management/test_key_update.py @@ -1,3 +1,5 @@ +import uuid + import pytest from litellm.proxy.utils import hash_token @@ -98,3 +100,85 @@ async def test_key_update_authz_matrix( assert row.models == [MARKER_MODEL] else: assert row.models != [MARKER_MODEL], "denied but row mutated" + + +async def _seed_shape(proxy_client, seeder, prefix, world, shape, caller) -> str: + if shape == "self": + return await create_scratch_key( + proxy_client, seeder, prefix, user_id=caller.user_id + ) + if shape == "owner": + return await create_scratch_key( + proxy_client, + seeder, + prefix, + user_id=world.keys[Actor.OWNER].user_id, + team_id=TEAM_ALPHA, + ) + if shape == "cross_org": + return await create_scratch_key( + proxy_client, + seeder, + prefix, + user_id=world.keys[Actor.CROSS_ORG_USER].user_id, + team_id=TEAM_BETA, + ) + pytest.fail(f"unknown shape={shape}") # pragma: no cover + + +async def test_key_update_missing_key_is_404(proxy_client, world): + """An update targeting a key absent from the DB is a 404 — not 401/403.""" + resp = await proxy_client.post( + "/key/update", + headers={"Authorization": f"Bearer {world.keys[Actor.PROXY_ADMIN].cleartext}"}, + json={"key": "sk-" + uuid.uuid4().hex, "models": [MARKER_MODEL]}, + ) + assert resp.status_code == 404, resp.text + + +# A denied /key/update must not partially apply: the budget/limit columns are +# left untouched. Each scenario is a denial cell from the matrix above. +_DENIED_BUDGET = [ + ("team_admin/self", Actor.TEAM_ADMIN, "self", 403), + ("internal_user/owner", Actor.INTERNAL_USER, "owner", 403), + ("cross_org_user/cross_org", Actor.CROSS_ORG_USER, "cross_org", 401), +] + + +@pytest.mark.parametrize( + "actor,target_shape,expected_status", + [(a, t, s) for (_id, a, t, s) in _DENIED_BUDGET], + ids=[s[0] for s in _DENIED_BUDGET], +) +async def test_key_update_denied_does_not_touch_budget_counters( + actor: Actor, + target_shape: str, + expected_status: int, + proxy_client, + prisma, + scratch, + world, +): + caller = world.keys[actor] + seeder = world.keys[Actor.PROXY_ADMIN].cleartext + target = await _seed_shape( + proxy_client, seeder, scratch.prefix, world, target_shape, caller + ) + target_hashed = hash_token(target) + + resp = await proxy_client.post( + "/key/update", + headers={"Authorization": f"Bearer {caller.cleartext}"}, + json={"key": target, "max_budget": 999.0, "tpm_limit": 888, "rpm_limit": 777}, + ) + assert ( + resp.status_code == expected_status + ), f"{actor.value} {target_shape}: {resp.status_code} {resp.text}" + + row = await prisma.db.litellm_verificationtoken.find_unique( + where={"token": target_hashed} + ) + assert row is not None + assert row.max_budget is None, "denied but max_budget applied" + assert row.tpm_limit is None, "denied but tpm_limit applied" + assert row.rpm_limit is None, "denied but rpm_limit applied" diff --git a/tests/proxy_behavior/management/test_route_coverage.py b/tests/proxy_behavior/management/test_route_coverage.py new file mode 100644 index 00000000000..1139e251a59 --- /dev/null +++ b/tests/proxy_behavior/management/test_route_coverage.py @@ -0,0 +1,91 @@ +"""PR3.M1 — codified route coverage. + +Every route declared in the two management-endpoint source files must be +exercised by at least one behavior-suite scenario. This is a permanent +regression guard: a future route added without a behavior test fails CI here, +the same way test_no_management_imports.py codifies the G3 import grep. +""" + +import ast +import pathlib +import re + +REPO_ROOT = pathlib.Path(__file__).resolve().parents[3] +SOURCE_FILES = [ + REPO_ROOT / "litellm/proxy/management_endpoints/key_management_endpoints.py", + REPO_ROOT / "litellm/proxy/management_endpoints/team_endpoints.py", +] +TEST_DIR = pathlib.Path(__file__).resolve().parent +SELF = pathlib.Path(__file__).resolve() + +# Captures the route literal from `@router.(""` — `\s*` spans +# newlines so multi-line decorators are matched too. +_ROUTE_DECORATOR = re.compile( + r"@router\.(?:get|post|put|delete|patch)\(\s*[\"']([^\"']+)[\"']" +) + + +def _source_routes() -> set: + routes: set = set() + for path in SOURCE_FILES: + routes.update(_ROUTE_DECORATOR.findall(path.read_text())) + return routes + + +def _route_to_regex(route: str) -> re.Pattern: + # A plain path param ({team_id}) matches a single path segment; a Starlette + # ':path' param ({key:path}) matches across '/'. Keeping plain params + # slash-bounded stops a loose regex from falsely reporting a future + # multi-segment route as already covered. + pattern = ["^"] + pos = 0 + for match in re.finditer(r"\{([^}]+)\}", route): + pattern.append(re.escape(route[pos : match.start()])) + pattern.append("[^?]+" if match.group(1).endswith(":path") else "[^/?]+") + pos = match.end() + pattern.append(re.escape(route[pos:]) + "$") + return re.compile("".join(pattern)) + + +def _test_urls() -> set: + """Every request-URL string literal across the behavior test suite. + + f-strings are reconstructed with each interpolation collapsed to a single + placeholder char, so f"/key/{target}/regenerate" becomes /key/X/regenerate. + Query strings are dropped — coverage is a path-level property. + """ + urls: set = set() + for path in sorted(TEST_DIR.glob("test_*.py")): + if path.resolve() == SELF: + continue + tree = ast.parse(path.read_text()) + for node in ast.walk(tree): + literal = None + if isinstance(node, ast.Constant) and isinstance(node.value, str): + literal = node.value + elif isinstance(node, ast.JoinedStr): + chunks = [] + for value in node.values: + if isinstance(value, ast.Constant) and isinstance(value.value, str): + chunks.append(value.value) + else: + chunks.append("X") # interpolated path / query segment + literal = "".join(chunks) + if literal and literal.startswith("/"): + urls.add(literal.split("?", 1)[0]) + return urls + + +def test_every_management_route_has_a_behavior_scenario(): + routes = _source_routes() + assert routes, "no @router routes parsed — the decorator regex is stale" + + urls = _test_urls() + uncovered = sorted( + route + for route in routes + if not any(_route_to_regex(route).match(url) for url in urls) + ) + assert ( + not uncovered + ), "management routes with no behavior-suite scenario:\n " + "\n ".join(uncovered) diff --git a/tests/proxy_behavior/management/test_scratch_teardown.py b/tests/proxy_behavior/management/test_scratch_teardown.py index 689c60fc78a..bcb53935558 100644 --- a/tests/proxy_behavior/management/test_scratch_teardown.py +++ b/tests/proxy_behavior/management/test_scratch_teardown.py @@ -1,13 +1,16 @@ import pytest -from .conftest import MASTER_KEY, SCRATCH_PREFIX +from litellm.proxy._types import LitellmUserRoles + +from .actors import ORG_A, ORG_B +from .conftest import MASTER_KEY, SCRATCH_PREFIX, create_scratch_actor pytestmark = pytest.mark.asyncio(loop_scope="session") -# The two tests run in file order: _a writes a scratch-tagged key and asserts -# it lands; _b runs after _a's fixture teardown and asserts no scratch row -# survived. A leak in either direction fails _b on the next collection. +# The minting tests run in file order, then _b runs after their fixture +# teardown and asserts no scratch row survived in any reclaimed table. A leak +# in either direction fails _b on the next collection. async def test_a_scratch_key_lands_in_db(proxy_client, prisma, scratch): @@ -24,8 +27,35 @@ async def test_a_scratch_key_lands_in_db(proxy_client, prisma, scratch): assert len(rows) == 1 +async def test_a2_scratch_actor_lands_in_db(proxy_client, prisma, scratch): + actor = await create_scratch_actor( + prisma, + scratch.prefix, + user_role=LitellmUserRoles.ORG_ADMIN.value, + org_admin_of=(ORG_A, ORG_B), + ) + user_row = await prisma.db.litellm_usertable.find_unique( + where={"user_id": actor.user_id} + ) + assert user_row is not None + info = await proxy_client.get( + "/key/info", headers={"Authorization": f"Bearer {actor.cleartext}"} + ) + assert info.status_code == 200, info.text + memberships = await prisma.db.litellm_organizationmembership.find_many( + where={"user_id": actor.user_id} + ) + assert {m.organization_id for m in memberships} == {ORG_A, ORG_B} + + async def test_b_scratch_namespace_is_clean(prisma): - rows = await prisma.db.litellm_verificationtoken.find_many( + tokens = await prisma.db.litellm_verificationtoken.find_many( where={"key_alias": {"startswith": SCRATCH_PREFIX}} ) - assert rows == [] + users = await prisma.db.litellm_usertable.find_many( + where={"user_id": {"startswith": SCRATCH_PREFIX}} + ) + memberships = await prisma.db.litellm_organizationmembership.find_many( + where={"user_id": {"startswith": SCRATCH_PREFIX}} + ) + assert tokens == [] and users == [] and memberships == [] diff --git a/tests/proxy_behavior/management/test_team_available.py b/tests/proxy_behavior/management/test_team_available.py new file mode 100644 index 00000000000..874c8dd4df7 --- /dev/null +++ b/tests/proxy_behavior/management/test_team_available.py @@ -0,0 +1,21 @@ +import pytest + +from .actors import Actor + +pytestmark = pytest.mark.asyncio(loop_scope="session") + + +# GET /team/available lists teams from +# litellm.default_internal_user_params["available_teams"]. The behavior world +# configures no available_teams, so the handler returns [] for every actor +# before it even reads the caller — this is the route-coverage + default-path +# pin. /team/available is an info route, so every authenticated actor reaches +# the handler. +@pytest.mark.parametrize("actor", list(Actor), ids=[a.value for a in Actor]) +async def test_team_available_default_is_empty(actor: Actor, proxy_client, world): + resp = await proxy_client.get( + "/team/available", + headers={"Authorization": f"Bearer {world.keys[actor].cleartext}"}, + ) + assert resp.status_code == 200, f"{actor.value}: {resp.status_code} {resp.text}" + assert resp.json() == [] diff --git a/tests/proxy_behavior/management/test_team_block_unblock.py b/tests/proxy_behavior/management/test_team_block_unblock.py new file mode 100644 index 00000000000..9412e51b909 --- /dev/null +++ b/tests/proxy_behavior/management/test_team_block_unblock.py @@ -0,0 +1,114 @@ +import pytest + +from .actors import Actor +from .conftest import create_scratch_team + +pytestmark = pytest.mark.asyncio(loop_scope="session") + + +# POST /team/block + /team/unblock. The handler gate is _verify_team_access +# (proxy admin / team admin / org admin), but the management-route gate fronts +# it: the request carries the team's organization_id so an org admin of that +# org clears the gate's org-scoped branch. A team admin is an INTERNAL_USER +# and these are not internal_user routes, so a team admin can never reach the +# handler — only PROXY_ADMIN and an org admin of the team's own org pass. +_MATRIX = [ + ("alpha/proxy_admin", Actor.PROXY_ADMIN, "alpha", 200), + ("alpha/org_admin", Actor.ORG_ADMIN, "alpha", 200), + ("alpha/team_admin", Actor.TEAM_ADMIN, "alpha", 401), + ("alpha/internal_user", Actor.INTERNAL_USER, "alpha", 401), + ("alpha/owner", Actor.OWNER, "alpha", 401), + ("alpha/unrelated_same_org", Actor.UNRELATED_SAME_ORG, "alpha", 401), + ("alpha/cross_org_user", Actor.CROSS_ORG_USER, "alpha", 401), + ("alpha/service_account", Actor.SERVICE_ACCOUNT, "alpha", 401), + ("alpha/org_b_admin", Actor.ORG_B_ADMIN, "alpha", 401), + ("beta/proxy_admin", Actor.PROXY_ADMIN, "beta", 200), + ("beta/org_admin", Actor.ORG_ADMIN, "beta", 401), + ("beta/org_b_admin", Actor.ORG_B_ADMIN, "beta", 200), +] + + +async def _seed_target(prisma, world, shape: str, team_id: str) -> str: + """Raw-seed the scratch target team; returns its organization_id.""" + org_id = world.org_a_id if shape == "alpha" else world.org_b_id + await create_scratch_team(prisma, team_id, organization_id=org_id) + return org_id + + +@pytest.mark.parametrize("route", ["block", "unblock"]) +@pytest.mark.parametrize( + "actor,shape,expected_status", + [(a, sh, s) for (_id, a, sh, s) in _MATRIX], + ids=[s[0] for s in _MATRIX], +) +async def test_team_block_unblock_authz_matrix( + route: str, + actor: Actor, + shape: str, + expected_status: int, + proxy_client, + prisma, + scratch, + world, +): + org_id = await _seed_target(prisma, world, shape, scratch.prefix) + caller = world.keys[actor] + + # /unblock starts from a blocked row so a 200 is observable as True->False. + if route == "unblock": + await prisma.db.litellm_teamtable.update( + where={"team_id": scratch.prefix}, data={"blocked": True} + ) + + resp = await proxy_client.post( + f"/team/{route}", + headers={"Authorization": f"Bearer {caller.cleartext}"}, + json={"team_id": scratch.prefix, "organization_id": org_id}, + ) + assert ( + resp.status_code == expected_status + ), f"{route} {actor.value} {shape}: {resp.status_code} {resp.text}" + + row = await prisma.db.litellm_teamtable.find_unique( + where={"team_id": scratch.prefix} + ) + assert row is not None + if expected_status == 200: + assert bool(row.blocked) is (route == "block") + else: + assert bool(row.blocked) is (route == "unblock"), "denied but blocked mutated" + + +async def test_team_block_unblock_round_trip(proxy_client, prisma, scratch, world): + """PROXY_ADMIN block then unblock flips the blocked column True then False.""" + await create_scratch_team(prisma, scratch.prefix, organization_id=world.org_a_id) + headers = {"Authorization": f"Bearer {world.keys[Actor.PROXY_ADMIN].cleartext}"} + + blocked = await proxy_client.post( + "/team/block", headers=headers, json={"team_id": scratch.prefix} + ) + assert blocked.status_code == 200, blocked.text + row = await prisma.db.litellm_teamtable.find_unique( + where={"team_id": scratch.prefix} + ) + assert row is not None and row.blocked is True + + unblocked = await proxy_client.post( + "/team/unblock", headers=headers, json={"team_id": scratch.prefix} + ) + assert unblocked.status_code == 200, unblocked.text + row = await prisma.db.litellm_teamtable.find_unique( + where={"team_id": scratch.prefix} + ) + assert row is not None and row.blocked is False + + +@pytest.mark.parametrize("route", ["block", "unblock"]) +async def test_team_block_unblock_missing_team_is_404(route: str, proxy_client, world): + """A team_id absent from the DB is 404 — the existence check precedes authz.""" + resp = await proxy_client.post( + f"/team/{route}", + headers={"Authorization": f"Bearer {world.keys[Actor.PROXY_ADMIN].cleartext}"}, + json={"team_id": "behavior-pin-no-such-team"}, + ) + assert resp.status_code == 404, resp.text diff --git a/tests/proxy_behavior/management/test_team_bulk_member_add.py b/tests/proxy_behavior/management/test_team_bulk_member_add.py new file mode 100644 index 00000000000..fc83cd414e5 --- /dev/null +++ b/tests/proxy_behavior/management/test_team_bulk_member_add.py @@ -0,0 +1,105 @@ +import pytest + +from .actors import Actor +from .conftest import create_scratch_team + +pytestmark = pytest.mark.asyncio(loop_scope="session") + + +def _member_ids(row) -> list: + return [m["user_id"] for m in (row.members_with_roles or [])] + + +async def test_team_bulk_member_add_proxy_admin_adds_explicit_members( + proxy_client, prisma, scratch, world +): + """PROXY_ADMIN bulk-adds an explicit member list to a scratch team.""" + await create_scratch_team(prisma, scratch.prefix, organization_id=world.org_a_id) + new_member = scratch.tag("m1") + resp = await proxy_client.post( + "/team/bulk_member_add", + headers={"Authorization": f"Bearer {world.keys[Actor.PROXY_ADMIN].cleartext}"}, + json={ + "team_id": scratch.prefix, + "members": [{"user_id": new_member, "role": "user"}], + }, + ) + assert resp.status_code == 200, resp.text + row = await prisma.db.litellm_teamtable.find_unique( + where={"team_id": scratch.prefix} + ) + assert row is not None and new_member in _member_ids(row) + + +async def test_team_bulk_member_add_empty_members_is_400( + proxy_client, prisma, scratch, world +): + """An empty member list (with all_users unset) is rejected 400.""" + await create_scratch_team(prisma, scratch.prefix, organization_id=world.org_a_id) + resp = await proxy_client.post( + "/team/bulk_member_add", + headers={"Authorization": f"Bearer {world.keys[Actor.PROXY_ADMIN].cleartext}"}, + json={"team_id": scratch.prefix, "members": []}, + ) + assert resp.status_code == 400, resp.text + + +async def test_team_bulk_member_add_over_max_batch_is_400( + proxy_client, prisma, scratch, world +): + """A member list larger than the 500-member cap is rejected 400.""" + await create_scratch_team(prisma, scratch.prefix, organization_id=world.org_a_id) + members = [ + {"user_id": f"{scratch.prefix}-u{i}", "role": "user"} for i in range(501) + ] + resp = await proxy_client.post( + "/team/bulk_member_add", + headers={"Authorization": f"Bearer {world.keys[Actor.PROXY_ADMIN].cleartext}"}, + json={"team_id": scratch.prefix, "members": members}, + ) + assert resp.status_code == 400, resp.text + + +@pytest.mark.parametrize( + "actor", + [Actor.TEAM_ADMIN, Actor.INTERNAL_USER], + ids=["team_admin", "internal_user"], +) +async def test_team_bulk_member_add_non_admin_is_401( + actor: Actor, proxy_client, prisma, scratch, world +): + """/team/bulk_member_add is neither an internal_user nor a self-managed + route — a non-proxy-admin with no org context is 401 at the route gate.""" + await create_scratch_team(prisma, scratch.prefix, organization_id=world.org_a_id) + resp = await proxy_client.post( + "/team/bulk_member_add", + headers={"Authorization": f"Bearer {world.keys[actor].cleartext}"}, + json={ + "team_id": scratch.prefix, + "members": [{"user_id": scratch.tag("m"), "role": "user"}], + }, + ) + assert resp.status_code == 401, f"{actor.value}: {resp.status_code} {resp.text}" + + +async def test_team_bulk_member_add_all_users_proxy_admin( + proxy_client, prisma, scratch, world +): + """all_users=True pulls every user in the DB into the team. The route is + reachable only by PROXY_ADMIN (the route gate 401s every other actor — even + an org admin with organization_id in the body), so the handler's own + all_users PROXY_ADMIN gate is never the deciding check at the boundary.""" + await create_scratch_team(prisma, scratch.prefix, organization_id=world.org_a_id) + resp = await proxy_client.post( + "/team/bulk_member_add", + headers={"Authorization": f"Bearer {world.keys[Actor.PROXY_ADMIN].cleartext}"}, + json={"team_id": scratch.prefix, "all_users": True}, + ) + assert resp.status_code == 200, resp.text + row = await prisma.db.litellm_teamtable.find_unique( + where={"team_id": scratch.prefix} + ) + assert row is not None + member_ids = _member_ids(row) + # every world actor is a user in the DB, so all are now team members + assert world.keys[Actor.INTERNAL_USER].user_id in member_ids diff --git a/tests/proxy_behavior/management/test_team_daily_activity.py b/tests/proxy_behavior/management/test_team_daily_activity.py new file mode 100644 index 00000000000..7a1e70b91fc --- /dev/null +++ b/tests/proxy_behavior/management/test_team_daily_activity.py @@ -0,0 +1,63 @@ +import pytest + +from .actors import Actor + +pytestmark = pytest.mark.asyncio(loop_scope="session") + + +# GET /team/daily/activity. A proxy admin (admin view) sees activity for any +# team. A non-admin is scoped to user_info.teams: a bare query defaults to its +# own teams (200), and an explicit team_ids filter naming a team it does not +# belong to is 404 (the VERIA-43 fix). Org admins have no team memberships, so +# they behave like a non-member for any specific team. +_MEMBERS = { + "alpha": { + Actor.TEAM_ADMIN, + Actor.INTERNAL_USER, + Actor.OWNER, + Actor.UNRELATED_SAME_ORG, + Actor.SERVICE_ACCOUNT, + }, + "beta": {Actor.CROSS_ORG_USER}, +} + + +def _expected(actor: Actor, team: str) -> int: + if team == "none" or actor == Actor.PROXY_ADMIN: + return 200 + return 200 if actor in _MEMBERS.get(team, set()) else 404 + + +_CASES = [ + (f"{team}/{actor.value}", actor, team, _expected(actor, team)) + for team in ("none", "alpha", "beta") + for actor in Actor +] + + +# start_date / end_date are required by the handler — pin only the team-scope +# authz, not the date validation. +_DATES = "start_date=2024-01-01&end_date=2024-12-31" + + +@pytest.mark.parametrize( + "actor,team,expected_status", + [(a, t, s) for (_id, a, t, s) in _CASES], + ids=[c[0] for c in _CASES], +) +async def test_team_daily_activity_matrix( + actor: Actor, team: str, expected_status: int, proxy_client, world +): + query = _DATES + if team == "alpha": + query += f"&team_ids={world.team_alpha_id}" + elif team == "beta": + query += f"&team_ids={world.team_beta_id}" + + resp = await proxy_client.get( + f"/team/daily/activity?{query}", + headers={"Authorization": f"Bearer {world.keys[actor].cleartext}"}, + ) + assert ( + resp.status_code == expected_status + ), f"{actor.value} -> {team}: {resp.status_code} {resp.text}" diff --git a/tests/proxy_behavior/management/test_team_delete.py b/tests/proxy_behavior/management/test_team_delete.py new file mode 100644 index 00000000000..bbf0a6563f3 --- /dev/null +++ b/tests/proxy_behavior/management/test_team_delete.py @@ -0,0 +1,78 @@ +import pytest + +from .actors import Actor +from .conftest import create_scratch_team + +pytestmark = pytest.mark.asyncio(loop_scope="session") + + +# POST /team/delete runs per-team _verify_team_access. The request carries the +# team's organization_id so an org admin of that org clears the management- +# route gate; a team admin is an INTERNAL_USER on a non-internal_user route, +# so a team admin never reaches the handler. Only PROXY_ADMIN and an org admin +# of the team's own org can delete it. +_MATRIX = [ + ("alpha/proxy_admin", Actor.PROXY_ADMIN, "alpha", 200), + ("alpha/org_admin", Actor.ORG_ADMIN, "alpha", 200), + ("alpha/team_admin", Actor.TEAM_ADMIN, "alpha", 401), + ("alpha/internal_user", Actor.INTERNAL_USER, "alpha", 401), + ("alpha/cross_org_user", Actor.CROSS_ORG_USER, "alpha", 401), + ("alpha/org_b_admin", Actor.ORG_B_ADMIN, "alpha", 401), + ("beta/proxy_admin", Actor.PROXY_ADMIN, "beta", 200), + ("beta/org_admin", Actor.ORG_ADMIN, "beta", 401), + ("beta/org_b_admin", Actor.ORG_B_ADMIN, "beta", 200), +] + + +@pytest.mark.parametrize( + "actor,shape,expected_status", + [(a, sh, s) for (_id, a, sh, s) in _MATRIX], + ids=[s[0] for s in _MATRIX], +) +async def test_team_delete_authz_matrix( + actor: Actor, + shape: str, + expected_status: int, + proxy_client, + prisma, + scratch, + world, +): + org_id = world.org_a_id if shape == "alpha" else world.org_b_id + await create_scratch_team(prisma, scratch.prefix, organization_id=org_id) + caller = world.keys[actor] + + resp = await proxy_client.post( + "/team/delete", + headers={"Authorization": f"Bearer {caller.cleartext}"}, + json={"team_ids": [scratch.prefix], "organization_id": org_id}, + ) + assert ( + resp.status_code == expected_status + ), f"{actor.value} {shape}: {resp.status_code} {resp.text}" + + row = await prisma.db.litellm_teamtable.find_unique( + where={"team_id": scratch.prefix} + ) + if expected_status == 200: + assert row is None, "deleted but team row survives" + else: + assert row is not None, "denied but team row vanished" + + +async def test_team_delete_batch_with_missing_id_deletes_nothing( + proxy_client, prisma, scratch, world +): + """A batch is validated whole before any deletion: one missing team_id + fails the request 404 and the accessible team in the batch survives.""" + await create_scratch_team(prisma, scratch.prefix, organization_id=world.org_a_id) + resp = await proxy_client.post( + "/team/delete", + headers={"Authorization": f"Bearer {world.keys[Actor.PROXY_ADMIN].cleartext}"}, + json={"team_ids": [scratch.prefix, "behavior-pin-no-such-team"]}, + ) + assert resp.status_code == 404, resp.text + row = await prisma.db.litellm_teamtable.find_unique( + where={"team_id": scratch.prefix} + ) + assert row is not None, "batch aborted but the accessible team was deleted" diff --git a/tests/proxy_behavior/management/test_team_filter_ui.py b/tests/proxy_behavior/management/test_team_filter_ui.py new file mode 100644 index 00000000000..69cbabf72a1 --- /dev/null +++ b/tests/proxy_behavior/management/test_team_filter_ui.py @@ -0,0 +1,39 @@ +import pytest + +from .actors import Actor + +pytestmark = pytest.mark.asyncio(loop_scope="session") + + +# GET /team/filter/ui (ui_view_teams) — include_in_schema=False. The handler +# body has no role/org check and never reads user_api_key_dict, but the +# endpoint is still effectively PROXY-ADMIN-only as its docstring claims: the +# management-route gate fronts it (not an internal_user / info / org-admin +# route) and 401s every non-proxy-admin before the handler runs. PROXY_ADMIN +# reaches the unscoped find_many and sees teams across every org. +@pytest.mark.parametrize("actor", list(Actor), ids=[a.value for a in Actor]) +async def test_team_filter_ui_is_proxy_admin_only(actor: Actor, proxy_client, world): + resp = await proxy_client.get( + "/team/filter/ui", + headers={"Authorization": f"Bearer {world.keys[actor].cleartext}"}, + ) + expected = 200 if actor == Actor.PROXY_ADMIN else 401 + assert ( + resp.status_code == expected + ), f"{actor.value}: {resp.status_code} {resp.text}" + + +async def test_team_filter_ui_proxy_admin_sees_cross_org_teams(proxy_client, world): + """The handler runs an unscoped query — PROXY_ADMIN sees teams from every + org, including the three seeded world teams.""" + resp = await proxy_client.get( + "/team/filter/ui", + headers={"Authorization": f"Bearer {world.keys[Actor.PROXY_ADMIN].cleartext}"}, + ) + assert resp.status_code == 200, resp.text + team_ids = {t.get("team_id") for t in resp.json() if isinstance(t, dict)} + assert { + world.team_alpha_id, + world.team_beta_id, + world.team_gamma_id, + } <= team_ids diff --git a/tests/proxy_behavior/management/test_team_key_bulk_update.py b/tests/proxy_behavior/management/test_team_key_bulk_update.py new file mode 100644 index 00000000000..5acf0c8185c --- /dev/null +++ b/tests/proxy_behavior/management/test_team_key_bulk_update.py @@ -0,0 +1,217 @@ +import uuid + +import pytest + +from litellm.proxy._types import KeyManagementRoutes +from litellm.proxy.utils import hash_token + +from .actors import Actor +from .conftest import create_scratch_key, create_scratch_team + +pytestmark = pytest.mark.asyncio(loop_scope="session") + +_MARKER_BUDGET = 42.0 +_KEY_UPDATE = KeyManagementRoutes.KEY_UPDATE.value + + +# POST /team/key/bulk_update — PROXY_ADMIN bypasses; otherwise +# can_team_member_execute_key_management_endpoint runs with route=KEY_UPDATE. +# A team admin always passes; a "user"-role member passes only when the team's +# team_member_permissions grants /key/update; a non-member is 401. ORG_ADMIN is +# stopped 401 at the management-route gate before the handler (the body has a +# team_id but no organization_id, so the org-admin route branch never matches). +_MATRIX = [ + ("admin/proxy_admin", Actor.PROXY_ADMIN, "admin", 200), + ("admin/internal_user", Actor.INTERNAL_USER, "admin", 200), + ("member_allowed/internal_user", Actor.INTERNAL_USER, "member_allowed", 200), + ("member_denied/internal_user", Actor.INTERNAL_USER, "member_denied", 401), + ("nonmember/internal_user", Actor.INTERNAL_USER, "nonmember", 401), + ("nonmember/org_admin", Actor.ORG_ADMIN, "nonmember", 401), + ("nonmember/proxy_admin", Actor.PROXY_ADMIN, "nonmember", 200), +] + + +async def _seed_team_key(prisma, proxy_client, prefix: str, world, shape: str) -> str: + """Raw-seed the scratch team for `shape`, return a team key's cleartext.""" + internal = world.keys[Actor.INTERNAL_USER].user_id + owner = world.keys[Actor.OWNER].user_id + if shape == "admin": + await create_scratch_team( + prisma, prefix, organization_id=world.org_a_id, admin_user_ids=[internal] + ) + key_owner = internal + elif shape == "member_allowed": + await create_scratch_team( + prisma, + prefix, + organization_id=world.org_a_id, + admin_user_ids=[owner], + member_user_ids=[internal], + team_member_permissions=[_KEY_UPDATE], + ) + key_owner = owner + elif shape == "member_denied": + await create_scratch_team( + prisma, + prefix, + organization_id=world.org_a_id, + admin_user_ids=[owner], + member_user_ids=[internal], + team_member_permissions=[], + ) + key_owner = owner + elif shape == "nonmember": + await create_scratch_team( + prisma, prefix, organization_id=world.org_a_id, admin_user_ids=[owner] + ) + key_owner = owner + else: + pytest.fail(f"unknown shape={shape}") # pragma: no cover + return await create_scratch_key( + proxy_client, + world.keys[Actor.PROXY_ADMIN].cleartext, + prefix, + user_id=key_owner, + team_id=prefix, + ) + + +@pytest.mark.parametrize( + "actor,shape,expected_status", + [(a, sh, s) for (_id, a, sh, s) in _MATRIX], + ids=[s[0] for s in _MATRIX], +) +async def test_team_key_bulk_update_authz_matrix( + actor: Actor, + shape: str, + expected_status: int, + proxy_client, + prisma, + scratch, + world, +): + key = await _seed_team_key(prisma, proxy_client, scratch.prefix, world, shape) + hashed = hash_token(key) + caller = world.keys[actor] + + resp = await proxy_client.post( + "/team/key/bulk_update", + headers={"Authorization": f"Bearer {caller.cleartext}"}, + json={ + "team_id": scratch.prefix, + "key_ids": [key], + "update_fields": {"max_budget": _MARKER_BUDGET}, + }, + ) + assert ( + resp.status_code == expected_status + ), f"{actor.value} {shape}: {resp.status_code} {resp.text}" + + row = await prisma.db.litellm_verificationtoken.find_unique(where={"token": hashed}) + assert row is not None + if expected_status == 200: + assert len(resp.json()["successful_updates"]) == 1 + assert row.max_budget == _MARKER_BUDGET + else: + assert row.max_budget != _MARKER_BUDGET, "denied but key mutated" + + +async def test_team_key_bulk_update_requires_team_id( + proxy_client, prisma, scratch, world +): + """An empty team_id is rejected 400.""" + resp = await proxy_client.post( + "/team/key/bulk_update", + headers={"Authorization": f"Bearer {world.keys[Actor.PROXY_ADMIN].cleartext}"}, + json={ + "team_id": "", + "key_ids": ["sk-" + uuid.uuid4().hex], + "update_fields": {"max_budget": _MARKER_BUDGET}, + }, + ) + assert resp.status_code == 400, resp.text + + +async def test_team_key_bulk_update_all_keys_in_team( + proxy_client, prisma, scratch, world +): + """all_keys_in_team=True broadcasts the update to every key in the team.""" + admin = world.keys[Actor.PROXY_ADMIN].cleartext + await create_scratch_team(prisma, scratch.prefix, organization_id=world.org_a_id) + keys = [ + await create_scratch_key( + proxy_client, + admin, + scratch.prefix, + user_id=world.keys[Actor.OWNER].user_id, + team_id=scratch.prefix, + key_alias=f"{scratch.prefix}-k{i}", + ) + for i in range(2) + ] + + resp = await proxy_client.post( + "/team/key/bulk_update", + headers={"Authorization": f"Bearer {admin}"}, + json={ + "team_id": scratch.prefix, + "all_keys_in_team": True, + "update_fields": {"max_budget": _MARKER_BUDGET}, + }, + ) + assert resp.status_code == 200, resp.text + assert len(resp.json()["successful_updates"]) == 2 + for key in keys: + row = await prisma.db.litellm_verificationtoken.find_unique( + where={"token": hash_token(key)} + ) + assert row is not None and row.max_budget == _MARKER_BUDGET + + +async def test_team_key_bulk_update_no_keys_found_is_404( + proxy_client, prisma, scratch, world +): + """all_keys_in_team=True on a team with no keys is a top-level 404.""" + await create_scratch_team(prisma, scratch.prefix, organization_id=world.org_a_id) + resp = await proxy_client.post( + "/team/key/bulk_update", + headers={"Authorization": f"Bearer {world.keys[Actor.PROXY_ADMIN].cleartext}"}, + json={ + "team_id": scratch.prefix, + "all_keys_in_team": True, + "update_fields": {"max_budget": _MARKER_BUDGET}, + }, + ) + assert resp.status_code == 404, resp.text + + +async def test_team_key_bulk_update_missing_key_is_isolated( + proxy_client, prisma, scratch, world +): + """A key_id absent from the team lands in failed_updates; the batch still + returns 200 and the real key is updated.""" + admin = world.keys[Actor.PROXY_ADMIN].cleartext + real = await _seed_team_key( + prisma, proxy_client, scratch.prefix, world, "nonmember" + ) + missing = "sk-" + uuid.uuid4().hex + + resp = await proxy_client.post( + "/team/key/bulk_update", + headers={"Authorization": f"Bearer {admin}"}, + json={ + "team_id": scratch.prefix, + "key_ids": [real, missing], + "update_fields": {"max_budget": _MARKER_BUDGET}, + }, + ) + assert resp.status_code == 200, resp.text + body = resp.json() + assert body["total_requested"] == 2 + assert len(body["successful_updates"]) == 1 + assert len(body["failed_updates"]) == 1 + + row = await prisma.db.litellm_verificationtoken.find_unique( + where={"token": hash_token(real)} + ) + assert row is not None and row.max_budget == _MARKER_BUDGET diff --git a/tests/proxy_behavior/management/test_team_list_v2.py b/tests/proxy_behavior/management/test_team_list_v2.py new file mode 100644 index 00000000000..81178ad73c0 --- /dev/null +++ b/tests/proxy_behavior/management/test_team_list_v2.py @@ -0,0 +1,141 @@ +from typing import FrozenSet, Optional + +import pytest + +from .actors import Actor + +pytestmark = pytest.mark.asyncio(loop_scope="session") + + +def _seeded(team_ids: set, world) -> set: + known = { + world.team_alpha_id: "alpha", + world.team_beta_id: "beta", + world.team_gamma_id: "gamma", + } + return {known[t] for t in team_ids if t in known} + + +async def _v2_team_ids(proxy_client, caller_cleartext: str, extra: str = "") -> set: + """Walk every /v2/team/list page and collect the returned team_ids.""" + ids: set = set() + page = 1 + while True: + resp = await proxy_client.get( + f"/v2/team/list?page={page}&page_size=100{extra}", + headers={"Authorization": f"Bearer {caller_cleartext}"}, + ) + assert resp.status_code == 200, resp.text + body = resp.json() + teams = body.get("teams", []) or [] + for t in teams: + tid = t.get("team_id") if isinstance(t, dict) else None + if tid: + ids.add(tid) + if page * 100 >= (body.get("total") or 0) or not teams: + return ids + page += 1 + + +# GET /v2/team/list is an info route reachable by every actor, but +# _enforce_list_team_v2_access still gates a BARE query: a proxy admin sees +# all teams, an org admin sees its orgs' teams, and a regular user — who has +# passed no user_id filter — is rejected 401 ("only admins can query all +# teams"). A regular user must scope the query to its own user_id. +_BARE = [ + ("proxy_admin", Actor.PROXY_ADMIN, 200, frozenset({"alpha", "beta", "gamma"})), + ("org_admin", Actor.ORG_ADMIN, 200, frozenset({"alpha", "gamma"})), + ("org_b_admin", Actor.ORG_B_ADMIN, 200, frozenset({"beta"})), + ("team_admin", Actor.TEAM_ADMIN, 401, None), + ("internal_user", Actor.INTERNAL_USER, 401, None), + ("owner", Actor.OWNER, 401, None), + ("unrelated_same_org", Actor.UNRELATED_SAME_ORG, 401, None), + ("cross_org_user", Actor.CROSS_ORG_USER, 401, None), + ("service_account", Actor.SERVICE_ACCOUNT, 401, None), +] + + +@pytest.mark.parametrize( + "actor,expected_status,expected_visible", + [(a, s, v) for (_id, a, s, v) in _BARE], + ids=[s[0] for s in _BARE], +) +async def test_team_list_v2_bare( + actor: Actor, + expected_status: int, + expected_visible: Optional[FrozenSet[str]], + proxy_client, + world, +): + caller = world.keys[actor] + if expected_status != 200: + resp = await proxy_client.get( + "/v2/team/list", + headers={"Authorization": f"Bearer {caller.cleartext}"}, + ) + assert resp.status_code == expected_status, resp.text + return + + visible = _seeded(await _v2_team_ids(proxy_client, caller.cleartext), world) + assert visible == set( + expected_visible + ), f"{actor.value}: expected {sorted(expected_visible)}, got {sorted(visible)}" + + +# A regular user scoping the query to its own user_id is allowed, and sees +# exactly the teams it belongs to. +_OWN = { + Actor.TEAM_ADMIN: frozenset({"alpha"}), + Actor.INTERNAL_USER: frozenset({"alpha"}), + Actor.OWNER: frozenset({"alpha"}), + Actor.UNRELATED_SAME_ORG: frozenset({"alpha"}), + Actor.CROSS_ORG_USER: frozenset({"beta"}), + Actor.SERVICE_ACCOUNT: frozenset({"alpha"}), +} + + +@pytest.mark.parametrize( + "actor,expected_visible", list(_OWN.items()), ids=[a.value for a in _OWN] +) +async def test_team_list_v2_own_user_id_query( + actor: Actor, expected_visible: FrozenSet[str], proxy_client, world +): + caller = world.keys[actor] + visible = _seeded( + await _v2_team_ids( + proxy_client, caller.cleartext, f"&user_id={caller.user_id}" + ), + world, + ) + assert visible == set( + expected_visible + ), f"{actor.value}: expected {sorted(expected_visible)}, got {sorted(visible)}" + + +async def test_team_list_v2_user_id_filter_other_user_is_401(proxy_client, world): + """A regular user filtering by another user's user_id is rejected 401.""" + resp = await proxy_client.get( + f"/v2/team/list?user_id={world.keys[Actor.OWNER].user_id}", + headers={ + "Authorization": f"Bearer {world.keys[Actor.INTERNAL_USER].cleartext}" + }, + ) + assert resp.status_code == 401, resp.text + + +async def test_team_list_v2_org_filter_foreign_org_is_403(proxy_client, world): + """An org admin filtering by an organization it does not administer is 403.""" + resp = await proxy_client.get( + f"/v2/team/list?organization_id={world.org_b_id}", + headers={"Authorization": f"Bearer {world.keys[Actor.ORG_ADMIN].cleartext}"}, + ) + assert resp.status_code == 403, resp.text + + +async def test_team_list_v2_invalid_status_is_400(proxy_client, world): + """status accepts only 'deleted' — any other value is 400.""" + resp = await proxy_client.get( + "/v2/team/list?status=bogus", + headers={"Authorization": f"Bearer {world.keys[Actor.PROXY_ADMIN].cleartext}"}, + ) + assert resp.status_code == 400, resp.text diff --git a/tests/proxy_behavior/management/test_team_member_me.py b/tests/proxy_behavior/management/test_team_member_me.py new file mode 100644 index 00000000000..bfbbe0504ae --- /dev/null +++ b/tests/proxy_behavior/management/test_team_member_me.py @@ -0,0 +1,83 @@ +import uuid + +import pytest + +from litellm.proxy.utils import hash_token + +from .actors import Actor + +pytestmark = pytest.mark.asyncio(loop_scope="session") + + +# GET /team/{team_id}/members/me resolves the CALLER's own membership row. +# A caller that is not a member of the team is 404 — even PROXY_ADMIN, which +# is not in any seeded team. The route is self-managed, so every actor reaches +# the handler. TEAM_GAMMA has no members, so every actor is 404 there. +_MEMBERS = { + "alpha": { + Actor.TEAM_ADMIN, + Actor.INTERNAL_USER, + Actor.OWNER, + Actor.UNRELATED_SAME_ORG, + Actor.SERVICE_ACCOUNT, + }, + "beta": {Actor.CROSS_ORG_USER}, + "gamma": set(), +} + +_CASES = [ + (f"{team}/{actor.value}", actor, team, 200 if actor in members else 404) + for team, members in _MEMBERS.items() + for actor in Actor +] + + +@pytest.mark.parametrize( + "actor,team,expected_status", + [(a, t, s) for (_id, a, t, s) in _CASES], + ids=[c[0] for c in _CASES], +) +async def test_team_member_me_matrix( + actor: Actor, team: str, expected_status: int, proxy_client, world +): + team_id = { + "alpha": world.team_alpha_id, + "beta": world.team_beta_id, + "gamma": world.team_gamma_id, + }[team] + caller = world.keys[actor] + + resp = await proxy_client.get( + f"/team/{team_id}/members/me", + headers={"Authorization": f"Bearer {caller.cleartext}"}, + ) + assert ( + resp.status_code == expected_status + ), f"{actor.value} -> {team}: {resp.status_code} {resp.text}" + + if expected_status == 200: + body = resp.json() + assert body["user_id"] == caller.user_id + assert body["team_id"] == team_id + + +async def test_team_member_me_team_key_without_user_id_is_400( + proxy_client, prisma, scratch, world +): + """A key with no associated user_id (a team / service-account key) cannot + resolve 'me' — the caller has no identity to look up — so it is 400.""" + cleartext = "sk-" + uuid.uuid4().hex + await prisma.db.litellm_verificationtoken.create( + data={ + "token": hash_token(cleartext), + "key_name": f"{scratch.prefix}-teamkey", + "key_alias": f"{scratch.prefix}-teamkey", + "team_id": world.team_alpha_id, + "models": [], + } + ) + resp = await proxy_client.get( + f"/team/{world.team_alpha_id}/members/me", + headers={"Authorization": f"Bearer {cleartext}"}, + ) + assert resp.status_code == 400, resp.text diff --git a/tests/proxy_behavior/management/test_team_model.py b/tests/proxy_behavior/management/test_team_model.py new file mode 100644 index 00000000000..3564e8df83a --- /dev/null +++ b/tests/proxy_behavior/management/test_team_model.py @@ -0,0 +1,78 @@ +import pytest + +from .actors import Actor +from .conftest import create_scratch_team + +pytestmark = pytest.mark.asyncio(loop_scope="session") + +_MARKER_MODEL = "behavior-pin-team-model-marker" +_ROUTE_URL = {"add": "/team/model/add", "delete": "/team/model/delete"} + + +# POST /team/model/add + /team/model/delete. The handler gate is PROXY_ADMIN +# or team admin or org admin, but the management-route gate fronts it — these +# are neither internal_user nor org-admin nor info routes, so every +# non-proxy-admin is 401 before the handler runs. Only PROXY_ADMIN reaches the +# handler, making the team-admin / org-admin handler branches unreachable here. +_MATRIX = [ + ("proxy_admin", Actor.PROXY_ADMIN, 200), + ("org_admin", Actor.ORG_ADMIN, 401), + ("team_admin", Actor.TEAM_ADMIN, 401), + ("internal_user", Actor.INTERNAL_USER, 401), + ("owner", Actor.OWNER, 401), + ("unrelated_same_org", Actor.UNRELATED_SAME_ORG, 401), + ("cross_org_user", Actor.CROSS_ORG_USER, 401), + ("service_account", Actor.SERVICE_ACCOUNT, 401), + ("org_b_admin", Actor.ORG_B_ADMIN, 401), +] + + +@pytest.mark.parametrize("route", ["add", "delete"]) +@pytest.mark.parametrize( + "actor,expected_status", + [(a, s) for (_id, a, s) in _MATRIX], + ids=[s[0] for s in _MATRIX], +) +async def test_team_model_authz_matrix( + route: str, + actor: Actor, + expected_status: int, + proxy_client, + prisma, + scratch, + world, +): + initial = [] if route == "add" else [_MARKER_MODEL] + await create_scratch_team( + prisma, scratch.prefix, organization_id=world.org_a_id, models=initial + ) + caller = world.keys[actor] + + resp = await proxy_client.post( + _ROUTE_URL[route], + headers={"Authorization": f"Bearer {caller.cleartext}"}, + json={"team_id": scratch.prefix, "models": [_MARKER_MODEL]}, + ) + assert ( + resp.status_code == expected_status + ), f"{route} {actor.value}: {resp.status_code} {resp.text}" + + row = await prisma.db.litellm_teamtable.find_unique( + where={"team_id": scratch.prefix} + ) + assert row is not None + if expected_status == 200: + assert (_MARKER_MODEL in row.models) is (route == "add") + else: + assert list(row.models) == initial, "denied but models mutated" + + +@pytest.mark.parametrize("route", ["add", "delete"]) +async def test_team_model_missing_team_is_404(route: str, proxy_client, world): + """A team_id absent from the DB is 404 — the existence check precedes authz.""" + resp = await proxy_client.post( + _ROUTE_URL[route], + headers={"Authorization": f"Bearer {world.keys[Actor.PROXY_ADMIN].cleartext}"}, + json={"team_id": "behavior-pin-no-such-team", "models": [_MARKER_MODEL]}, + ) + assert resp.status_code == 404, resp.text diff --git a/tests/proxy_behavior/management/test_team_permissions.py b/tests/proxy_behavior/management/test_team_permissions.py new file mode 100644 index 00000000000..5d16702fe6c --- /dev/null +++ b/tests/proxy_behavior/management/test_team_permissions.py @@ -0,0 +1,170 @@ +import litellm +import pytest + +from litellm.proxy._types import KeyManagementRoutes + +from .actors import Actor +from .conftest import create_scratch_team + +pytestmark = pytest.mark.asyncio(loop_scope="session") + +_PERM = KeyManagementRoutes.KEY_INFO.value + + +# GET /team/permissions_list and POST /team/permissions_update are self-managed +# routes, so every actor reaches the handler. Both grant access to PROXY_ADMIN, +# the team admin, or an org admin of the team's org. The scratch team is in +# ORG_A with TEAM_ADMIN as its team admin. +_MATRIX = [ + ("proxy_admin", Actor.PROXY_ADMIN, 200), + ("org_admin", Actor.ORG_ADMIN, 200), + ("team_admin", Actor.TEAM_ADMIN, 200), + ("internal_user", Actor.INTERNAL_USER, 403), + ("owner", Actor.OWNER, 403), + ("unrelated_same_org", Actor.UNRELATED_SAME_ORG, 403), + ("cross_org_user", Actor.CROSS_ORG_USER, 403), + ("service_account", Actor.SERVICE_ACCOUNT, 403), + ("org_b_admin", Actor.ORG_B_ADMIN, 403), +] + + +async def _seed_team(prisma, scratch_prefix, world) -> None: + await create_scratch_team( + prisma, + scratch_prefix, + organization_id=world.org_a_id, + admin_user_ids=[world.keys[Actor.TEAM_ADMIN].user_id], + member_user_ids=[ + world.keys[Actor.INTERNAL_USER].user_id, + world.keys[Actor.OWNER].user_id, + world.keys[Actor.UNRELATED_SAME_ORG].user_id, + world.keys[Actor.SERVICE_ACCOUNT].user_id, + ], + ) + + +@pytest.mark.parametrize( + "actor,expected_status", + [(a, s) for (_id, a, s) in _MATRIX], + ids=[s[0] for s in _MATRIX], +) +async def test_team_permissions_list_authz_matrix( + actor: Actor, expected_status: int, proxy_client, prisma, scratch, world +): + await _seed_team(prisma, scratch.prefix, world) + resp = await proxy_client.get( + f"/team/permissions_list?team_id={scratch.prefix}", + headers={"Authorization": f"Bearer {world.keys[actor].cleartext}"}, + ) + assert ( + resp.status_code == expected_status + ), f"{actor.value}: {resp.status_code} {resp.text}" + if expected_status == 200: + assert resp.json()["team_id"] == scratch.prefix + + +@pytest.mark.parametrize( + "actor,expected_status", + [(a, s) for (_id, a, s) in _MATRIX], + ids=[s[0] for s in _MATRIX], +) +async def test_team_permissions_update_authz_matrix( + actor: Actor, expected_status: int, proxy_client, prisma, scratch, world +): + await _seed_team(prisma, scratch.prefix, world) + resp = await proxy_client.post( + "/team/permissions_update", + headers={"Authorization": f"Bearer {world.keys[actor].cleartext}"}, + json={"team_id": scratch.prefix, "team_member_permissions": [_PERM]}, + ) + assert ( + resp.status_code == expected_status + ), f"{actor.value}: {resp.status_code} {resp.text}" + + row = await prisma.db.litellm_teamtable.find_unique( + where={"team_id": scratch.prefix} + ) + assert row is not None + if expected_status == 200: + assert _PERM in (row.team_member_permissions or []) + else: + assert _PERM not in (row.team_member_permissions or []), "denied but mutated" + + +async def test_team_permissions_available_team_self_join_divergence( + proxy_client, prisma, scratch, world, monkeypatch +): + """permissions_list honours the available-team self-join — a non-admin can + READ an available team's permissions — but permissions_update deliberately + does not: the same caller is 403 on update. default_internal_user_params is + module-level litellm.* state, so monkeypatch save/restores it.""" + await create_scratch_team(prisma, scratch.prefix, organization_id=world.org_a_id) + monkeypatch.setattr( + litellm, "default_internal_user_params", {"available_teams": [scratch.prefix]} + ) + caller = world.keys[Actor.CROSS_ORG_USER] # non-admin, unrelated to the team + + listed = await proxy_client.get( + f"/team/permissions_list?team_id={scratch.prefix}", + headers={"Authorization": f"Bearer {caller.cleartext}"}, + ) + assert listed.status_code == 200, listed.text + + updated = await proxy_client.post( + "/team/permissions_update", + headers={"Authorization": f"Bearer {caller.cleartext}"}, + json={"team_id": scratch.prefix, "team_member_permissions": [_PERM]}, + ) + assert updated.status_code == 403, updated.text + + +# POST /team/permissions_bulk_update is PROXY_ADMIN-only. ORG_ADMIN-role +# callers are stopped 401 by the management-route gate; INTERNAL_USER-role +# callers, on a route that is neither internal_user nor self-managed, are 401 +# there too — only PROXY_ADMIN reaches the handler's own admin gate. +_BULK_MATRIX = [ + ("proxy_admin", Actor.PROXY_ADMIN, 200), + ("org_admin", Actor.ORG_ADMIN, 401), + ("team_admin", Actor.TEAM_ADMIN, 401), + ("internal_user", Actor.INTERNAL_USER, 401), + ("cross_org_user", Actor.CROSS_ORG_USER, 401), + ("org_b_admin", Actor.ORG_B_ADMIN, 401), +] + + +@pytest.mark.parametrize( + "actor,expected_status", + [(a, s) for (_id, a, s) in _BULK_MATRIX], + ids=[s[0] for s in _BULK_MATRIX], +) +async def test_team_permissions_bulk_update_authz_matrix( + actor: Actor, expected_status: int, proxy_client, prisma, scratch, world +): + await create_scratch_team(prisma, scratch.prefix, organization_id=world.org_a_id) + resp = await proxy_client.post( + "/team/permissions_bulk_update", + headers={"Authorization": f"Bearer {world.keys[actor].cleartext}"}, + json={"team_ids": [scratch.prefix], "permissions": [_PERM]}, + ) + assert ( + resp.status_code == expected_status + ), f"{actor.value}: {resp.status_code} {resp.text}" + + row = await prisma.db.litellm_teamtable.find_unique( + where={"team_id": scratch.prefix} + ) + assert row is not None + if expected_status == 200: + assert _PERM in (row.team_member_permissions or []) + else: + assert _PERM not in (row.team_member_permissions or []), "denied but mutated" + + +async def test_team_permissions_bulk_update_no_selector_is_400(proxy_client, world): + """Neither team_ids nor apply_to_all_teams is a 400.""" + resp = await proxy_client.post( + "/team/permissions_bulk_update", + headers={"Authorization": f"Bearer {world.keys[Actor.PROXY_ADMIN].cleartext}"}, + json={"permissions": [_PERM]}, + ) + assert resp.status_code == 400, resp.text diff --git a/tests/proxy_behavior/management/test_team_update.py b/tests/proxy_behavior/management/test_team_update.py index 3baf2b2148f..9b21911cef2 100644 --- a/tests/proxy_behavior/management/test_team_update.py +++ b/tests/proxy_behavior/management/test_team_update.py @@ -1,7 +1,9 @@ import pytest +from litellm.proxy._types import LitellmUserRoles + from .actors import Actor -from .conftest import create_scratch_team +from .conftest import create_scratch_actor, create_scratch_team pytestmark = pytest.mark.asyncio(loop_scope="session") @@ -130,8 +132,8 @@ async def test_team_update_requires_proxy_admin_without_org_context( # in ORG_A; each scenario relocates it to ORG_B. PROXY_ADMIN bypasses; # ORG_B_ADMIN clears the route gate (dest-org admin) but fails # _verify_team_access on the source team (403); the rest fail the route gate -# (401). The relocation-allowed branch needs a caller who is org admin of both -# orgs — no seeded actor is, so it is left to a later slice. +# (401). The relocation-*allowed* branch (caller is org admin of both orgs) is +# covered by test_team_update_org_relocation_allowed_for_dual_org_admin below. _RELOCATION = [ ("proxy_admin", Actor.PROXY_ADMIN, 200), ("org_b_admin", Actor.ORG_B_ADMIN, 403), @@ -174,3 +176,34 @@ async def test_team_update_org_relocation_gate( assert row.organization_id == world.org_b_id else: assert row.organization_id == world.org_a_id, "denied but team relocated" + + +async def test_team_update_org_relocation_allowed_for_dual_org_admin( + proxy_client, prisma, scratch, world +): + """Relocation-allowed branch: a caller who is org admin of BOTH the source + and destination org may relocate a team between them. Completes the + _RELOCATION matrix, whose allowed branch PR2 left open — no seeded actor is + a dual-org admin, so one is minted with create_scratch_actor.""" + actor = await create_scratch_actor( + prisma, + scratch.prefix, + user_role=LitellmUserRoles.ORG_ADMIN.value, + org_admin_of=(world.org_a_id, world.org_b_id), + ) + team_id = await create_scratch_team( + prisma, scratch.tag("team"), organization_id=world.org_a_id + ) + + resp = await proxy_client.post( + "/team/update", + headers={"Authorization": f"Bearer {actor.cleartext}"}, + json={"team_id": team_id, "organization_id": world.org_b_id}, + ) + assert resp.status_code == 200, resp.text + + row = await prisma.db.litellm_teamtable.find_unique(where={"team_id": team_id}) + assert row is not None + assert ( + row.organization_id == world.org_b_id + ), "dual-org admin relocation not applied" From 643989989f46a26b4ad74b6fc1ca7b4fc4236f16 Mon Sep 17 00:00:00 2001 From: ryan-crabbe-berri Date: Fri, 22 May 2026 11:29:17 -0700 Subject: [PATCH 07/13] chore(test): remove dead old Playwright e2e suite (#28632) The Playwright suite under tests/proxy_admin_ui_tests/e2e_ui_tests/ is no longer wired into CI (only test_*.py is globbed) and every active spec is duplicated by ui/litellm-dashboard/e2e_tests/tests/ (login, auth redirect, search users, internal user list). team_admin.spec.ts was entirely commented out. Removing the directory plus its only-used-here playwright config, package.json/lock, and utils/login.ts keeps the canonical suite under ui/litellm-dashboard/e2e_tests/ as the single source of truth. --- .../e2e_ui_tests/login_to_ui.spec.ts | 51 ---- .../e2e_ui_tests/redirect-fail-screenshot.png | Bin 49131 -> 0 bytes .../require_auth_for_dashboard.spec.ts | 37 --- .../e2e_ui_tests/search_users.spec.ts | 222 ---------------- .../e2e_ui_tests/team_admin.spec.ts | 250 ------------------ .../e2e_ui_tests/view_internal_user.spec.ts | 72 ----- .../e2e_ui_tests/view_user_info.spec.ts | 124 --------- tests/proxy_admin_ui_tests/package-lock.json | 97 ------- tests/proxy_admin_ui_tests/package.json | 14 - .../proxy_admin_ui_tests/playwright.config.ts | 84 ------ tests/proxy_admin_ui_tests/utils/login.ts | 27 -- 11 files changed, 978 deletions(-) delete mode 100644 tests/proxy_admin_ui_tests/e2e_ui_tests/login_to_ui.spec.ts delete mode 100644 tests/proxy_admin_ui_tests/e2e_ui_tests/redirect-fail-screenshot.png delete mode 100644 tests/proxy_admin_ui_tests/e2e_ui_tests/require_auth_for_dashboard.spec.ts delete mode 100644 tests/proxy_admin_ui_tests/e2e_ui_tests/search_users.spec.ts delete mode 100644 tests/proxy_admin_ui_tests/e2e_ui_tests/team_admin.spec.ts delete mode 100644 tests/proxy_admin_ui_tests/e2e_ui_tests/view_internal_user.spec.ts delete mode 100644 tests/proxy_admin_ui_tests/e2e_ui_tests/view_user_info.spec.ts delete mode 100644 tests/proxy_admin_ui_tests/package-lock.json delete mode 100644 tests/proxy_admin_ui_tests/package.json delete mode 100644 tests/proxy_admin_ui_tests/playwright.config.ts delete mode 100644 tests/proxy_admin_ui_tests/utils/login.ts diff --git a/tests/proxy_admin_ui_tests/e2e_ui_tests/login_to_ui.spec.ts b/tests/proxy_admin_ui_tests/e2e_ui_tests/login_to_ui.spec.ts deleted file mode 100644 index e5a397a6a66..00000000000 --- a/tests/proxy_admin_ui_tests/e2e_ui_tests/login_to_ui.spec.ts +++ /dev/null @@ -1,51 +0,0 @@ -/* - -Login to Admin UI -Basic UI Test - -Click on all the tabs ensure nothing is broken -*/ - -import { test, expect } from "@playwright/test"; - -test("admin login test", async ({ page }) => { - // Go to the specified URL - await page.goto("http://localhost:4000/ui"); - await page.waitForLoadState("networkidle"); - - await page.screenshot({ path: "test-results/login_before.png" }); - - // Enter "admin" in the username input field - await page.fill('input[placeholder="Enter your username"]', "admin"); - - // Enter "gm" in the password input field - await page.fill('input[placeholder="Enter your password"]', "gm"); - - page.screenshot({ path: "test-results/login_after_inputs.png" }); - - // Optionally, you can add an assertion to verify the login button is enabled - const loginButton = page.getByRole("button", { name: "Login" }); - await expect(loginButton).toBeEnabled(); - - // Optionally, you can click the login button to submit the form - await loginButton.click(); - const tabs = [ - "Virtual Keys", - "Playground", - "Models", - "Usage", - "Teams", - "Internal User", - "Settings", - "Experimental", - "API Reference", - "AI Hub", - ]; - - for (const tab of tabs) { - const tabElement = page.locator("span.ant-menu-title-content", { - hasText: tab, - }); - await tabElement.click(); - } -}); diff --git a/tests/proxy_admin_ui_tests/e2e_ui_tests/redirect-fail-screenshot.png b/tests/proxy_admin_ui_tests/e2e_ui_tests/redirect-fail-screenshot.png deleted file mode 100644 index b2e332512608703b6acb1c7839e7f40fc7460ab8..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 49131 zcmc$`byQW`+dqo!5l}$|B^4w@I;9mhA>CafAuZiu&;rsD(%s#qbV+w9oty5u&&Kn; z?>p}JjdAa9{N6F{`s17>d+)W@ob#E_d_FOkx0Hks<}JKiXlQ7duZ0mZXlU2q^m6yj z%kaOK#3GmAf0ry}g!s^Mei1C8p*==>jd&$z7ri=RD|dfr?8b(^O?WbO^=s*uSol=h zjhCsk@8>Ii^qLsPX7GtjV<@3y5d2E1@I@MHwE^cXv-Vu-!@Kx44VOs>aZ0+5o1Y$I zU;QX7y?@Nk&R%JY9JjX#wLGy{AAh41Mg}*A!!3X?HwXRiIa(U#&APv*aaecVP>%-B zNctG<%Eg(R;7hdY7iVaGZfF>P&+sl?dieK@`SNA@zi02RUe)@0)_wD1MKzZw0zoB{ zPc7M4K;6D&*KoI<9;4Ru`M(zp5LLgf^*|ImltfZuL`EPix;30KG)qLWj%iS@!=3A1 zeBH~sneB%uw?(gBT+=N;VB)G)SejCL8>PZiopz+|gYcdYx|mi6->C)sFp9poI}!1F z{rG$T4cdocx<-dVRAHULh2HE1nFfc{?RgjsW+8QquC^?L14j)C{V$((PctAIw0Qn) zNJdi06?)MSn!xiwg_Ea>M^oRG!`?jP$=BGs-s$&wl#A#;?UWNYS-F+Q!em;n zec!l$3G(t$s);^u<-%26zO}LxMyr%8^DI<^WP1CMYqECB>D8+r?csEfJPf+yKIG)k zvc-0sp7V;zr^``=`gqxKX{5hV4x7tqp~r~TbSN}TmT48w*giZ7!p%6K{(JW{gJN7a z0Ghb5%GRJ1+F({!TZ0Ull(eMjk(%UF>?-|XycM~4_s>{PO7_<7deK&Im6 z<|alxe3IgFdPH{TS!i1@g(P-hb+z-#V7BOYyTh%ysw%E9U3GPJ(U@;BF~2a=a&xB= zg#tue4E{c=l)51uTBcWZq;!&8!JJra0avmN3GLupxsZs6@Ixx0oV+>J5zcRmlwQ*G zR36SDbmDX}%-y(QTx-h%ncMT7Gn;mtHmj5m+$=3EXJ%#&om}3$d4hF%vfWjCx)&w< zw8Cns*Hz;3RE70(dit+kgW0NzRA}uw1#~qBADd?mkB@5~xeev0J8sX5li;zLj{kH= zFCTdurn|p7QfNLS=<$YtkPz$d?1r}P+k z(a|r1xg_kHs->tx!l$QksSjNYee(-FBU}-otQ7D*lp}k{W7{pV4D)WfLOiJshq(@P&%_b}{w$PNUml@P)cigx8o`y0T(y7Je9nY(@DMOomd{{ z3^^(j6B9y0LQY#`R&w(0d8pzpE{(FXa;5FMM%nu(SOP*qwGO+>+=t&WFB(#{aNsm~ zQrA^BMBPrSvN@wDI&?x%u(>DkB< zd=;nfG(?Lz@AVEg8UE_-fUfDc%HptZ@*T~LN`)*8-C3e+GS}o{gC;1B#ZtV9L`?E`h1iLB9>@vw> z=^3*E!NI|`$H>axzklP>X%Mhm%uZH2BA$kd@bmEp5cB?;$gR}tjD%@-wl`*`-xYm` ztXx;fQL~?J@V1Rj8o<8!(8Jr?+kR`db$g-f4iQ%i{g*sM&ff%{hT-vstHP?IqwN_b z<vBQ_2HN}le@aMc?_r=F6l-0wrYZZ!&^*QjqZt_|3o|5+O?8qc$~b^0kDDzZD! z%i`cr5i3b(WbkIM({|-)@43rElTdEEkOKQ=jiJF`edK<>+I}flo8H5ka6Qo3tq`Br zOPM+K#C?FhzZxWlW0(o$Oy5fKZE0x8O9E_;iaCO^mqs98PJgo%rb%gf74Lz8{w#%;A4r^9Aq zS1W64=jJXWJyY7dH)FURaUMNN&&-U9i7|cqwti+)qsng6pY&qEa!biCEhI~b9zW6c zX8zN(mRl1&IFL=azZe1i=EUJtmt3}T?V3Apwz9UKA){MM!?2X7=yOgA#dJBphvT`T zsU_bz%XsWUWD+eco~3trztgT0b`I2N#}U_Me1t{#<*>80=Z$nX2FdbxkM($w{pLSx zs+J$`(DsKr4PLnI0$0&4` zg-1GV>fJHQjPBqPlunYO^nI`m5BPls46%F9a|ZJr%u$H#xP zyS=GgX|vXoD0Jw=!_REB*gZU~^zZU-r|RymO~S~+B2f`4hWI{O`F?GJqmN_ z{6;{`i{Sw_rc=l1_tW13$I*?Rx33SHu8!xM?B@_xTAOMn3fLGg@F8Go?JV~s=Z1T} z<#!~haG421kYH+*Do?D9_%hR?v6)*@no+6~uB8W`u$uUu>>fk}J^ChED@$=GPjfnG zh&DKP{v1ue)*&>;VP)Pgb!0vZ=e0aliPwPqvyq{ZMkj#}$vn@8!tx+X&3IeQQS)Pn zA{z3j-aH^mXs17~IES_017^`bq7C$7kfSClZOaFGFanzC#S9G%L$Bl=Ve@<5_Vf35 zIXhwJ*U>fMb~@PD&d*0I|s&;rFlgi%#D^4P|{N2`1%$XyPL&4 zEYWNIKuXcopiyJ7OV>MfZQQLPj;?T8E|j%5_(qeplr1ZSso(n7y9D;&10 zRySTv&IsO_DIr#%3{_kBML0I9GIV_0acf`Lf$2)E<56}sT1P>a_g*bejJ&1iiSdG( z3oEHSrO$o)T+91yYN0Y@ij+In_nl-YAKY^1`>}Cw-NWIWi{jhPK~kdpo^yNXxR!D? zrHZCztdYTJQM#`$`o5cHzLK$)*6-=*pWN)}9dATyvVZ#w4i1tEeK|0n;0SxS*d1?U zWAotr(%-tS>KO$6a$Wu)O%#^&jNl$(hk4iile064NX@yI*YcPWJB!VMrc(l6{N0;Z z5mviLn8(*k!f^B=BG}IouOA2CIsMrx{{6M{4pH)pP~LTwM}BytjTGYqGn}6tqoTsk zIhZQ#g9Q-~*&!kW;Y{Rt)c7Tb;q}cDe!KXi1p>0%>J#pksg|cqg$13RV&zd0=d!IB zVuO%q@yHtLc%UaRFfc%NUA93-N9X6Sm*G|p3l&KoD=`UfpF2I; zWnyOT=;{h@pW}Ae=^hxUw%eSB3g^q0FQcVquE=qAWR`5I1h?~%9Jkkp4?kK0UFj%< zVM#mKZh z;eLA=Yh<95g@xhBht7*(XU-1uxL&2mFVoxhgrg?4hho^ z*(%#t`P?M8OFxO^Ifsf<#^2R;!GPX{}y?_d9TrE2q19gQwT>Zz7QnifZ;#L{|V0%8)8CeU-vE&^G-3Aj$hM7?@OTAu z1I*#9=xrY5Z@GtU${J{$9ssvqqS z>6n^MrpuBU$+Fc=r z0{h3O8`SNiBO~GwjE(j6pSl*JA|u1tvmzoM1vIbpr{4=ubRa7cN$&0KH5@5Oj)@@( zxARVeRJnYcMzy4WXsEQLBxfu9=0#i+SHJmmbQ{0Fv?_6Mm%1e9JV#0`iR^Aj)%sBO zr%WhHR-@Ip)z!bd7EI&HHcLdk;GpB3>gZ3Z7)-dmfn=$xRMc?!P*Nf#bbQXA{^5g5 z%W|VYW1~!ZN5io21W%K#*Lm6x<&kpoCP62j056Okf#yAv%##P1WJea?Mx(#pZ=Yk3 z+=}%nfBn1kU{Be@E-ORRc_IwQBUz&T*Dpxyhf%4E%gZWCN?eY6U%c>Gj46AE^p*Gi z)O$X9^yu=3x~3)++QF!&yzk?AhgeTvzmVEUM+dM`Qg*gD;_&d0Opzzkz`y`t_(YBK zac3#N13diWQ8hsHy$yPl@T|4=(?*L~9Fh+}-dO|F&VsDK6|tuPd; zm+tSkjj>?vgv8((`68)X5bRkrFtwxI&nl^k)qdN(*Z}A64_`jB#&Kh( z&c8+WTs~b#L?K;UIn0i$cRH9g0Bhs$vWMsIf9Z*V1ph1VUsGe{e?E5l_gTrSgpR40 z%=V47{$9(iR4Oyz6^dGcBklSBIGgcb7raW>$Fx%MFCF5B*PKVKI>hq#zG(d~s6YH~ z6D9w3>Hjh2g%&L?1@&Bish*cCS`SlHnqHxK1Ym|c>N1wgk%za=-bxlpmKm^?zIgPF zK|8J(x2}7)ii_M;_~HjEDc)Q!)Y<6yea0x7#F$^E)L6j5F*?de^TDla;Y~U(>PK~+ zLg_Nk?u(?0(FO|JV~}DmEx$vbBD+qCRxk8XS2;{~d#lmH{pU+fNhnZro?ncOo1rXq zAkKXpgMY3HdH`RN5%vMg?a$k~f-ITjwpN<%UhIeX%R|=e7Y`*xYO#$zXW{Ohe<`*( zlkd79n#Ua?w*9Ek;M{B)=Ki6~6^o1S#a*An#d)2~$aJrss^bdW-g7NBw96eMz_VQQ z;tS$mqcmGG+|#cbhzpEZk*CLaW!jZ`N&Gnn`hO z6u`cc<#aKW-A}I{IZTuM6MmD*73VU&)36RBdwap~_9N79UnVXtzbmAE9^5*$Onp`B zK+3#!HBeZWL%0_8DO#kEkdP`|ci<^K>X)+??j=&0Ihhxqxs9gK^ar&+Ftfe|pVcXa z0TRA^sWu1wBEsE9$?tB2iabDGx{jHkt;}=B)$ii(%(r8kWvl1jItLQ`zT%OpbWwA? zd(rVPQ->Df@&A>d`#%Z0k8!{p+G)1T`c%+fiWAh)n;TV%7_xqVYMAvMrn-v;qWzCi ziN$zsGt<+C+kb}}4b89^3-t#lXJac^vnQi!tPTL!T?sYE&t$ z)&I>0FDEkm&{kJgL=n=`($6jq#O&|iQV}D+y}WMTxN#$2Nv*)V(dWV1Xz^QRR%T`^ z0FAma5202dCe|!A1PEPbyFMx7r{5K0{r){AHTCQyIRynLCnw;hLc{RCt#un!oVoPk z>C>n5^z>9zZJnKdM*m=7SXo+Tc5uC~bvZ5DNH>+Hybr7cA-4l_J^hV4MB|0}k`WZ9 zVpUF;zx zCl{Q+#>J(;@x6hGhd})J@k8w@_xs;<-x%Khtgjy{Hp0cj;{Zg0j!TOW6m+n&8;A7= zR&cm*V`Jm^O<1nx^^(fHr(b=#&d$lXzc#jC_u)$D;fta>f+5|>uZ%iBAG zN#8#_d?-t~FgiMVd}5;Za6SqiT=Z#Bf4^L)NTZ*Wl+?z~k}Qb>P>RF(Z-Fp!J@|72 z7dO1za$&N{zHnN*x3BMbX9;4Y;7$E?8XB4$^MMR`EG(><=73mM6BR3~9pLu7TLuQK z7k+)m_Wrz52n>^=qGFhC^Or}7$;rhgV1MusiOOt*JeNL|RPjOJQN* z%g2iomHYW`o3ZcRQ!O@>|MBe8jXOBLjX60veDPn6uzVZGO3m(|qvO+SpbyBEprH|?M6Ey z-dXC4iHXtw6>Tt>rQDYy?%gsm;ruI_okB8Zb+Wp$w6wIm{7`k^U;RK{;ge__KOgEC zA0MBdp6)*uFb_avIHAqYs{?x*%k5}0+k!<*9LuOLK7bz7o>()cP>pjmT5P1Dq0t;b zJTx?9Yh#lmMg}FRQi0CvLcPuuiSH^>QZErQ=^ zTFJfTKE2qJSnq*3BRC9G!UGc@0}~U-4b=Sj{Q0wp%fEHahQFcV^>3%lT((*%;8P20>+BQ7zeN$6yaFjx-Lip( zY5f8WIAn{Eko!pKIx=E!xpZuME=|p;Y}Ha91j(H{0d3K}05$;!LEt{X#bq<>_wsT% zT1rhyN`g5e1asJ#nueyYxAzR#b0}x+x98<+h=hcif#)ZN`~}rOe}8{ppC!z<{QUgZ z)>d>J>c!FG=lj341BID&TE4HJ6g+IBBjD-c3B6}giN$>&G z9MptqlHldMs3`pXxJ2lKAkptrRQq8K-1{v6 zJ!WO}J>Imf@EQ-(2MW)wtVBVS=;+)hAaL58Zv1fN#`POFjt>vbondX6n3$jt`_^4* zrpd%q2;-G;gW62Tx-vI6x3qNsX@4n8Atf%``P+7o;d5%nRnpIND0F_vbo z-dqPu2Xtp#uE~b6Bs127Aziib`=g>d_frqQbAxS(WEo}e?TZ}7gCQo)5 z=uvwCV}zKhYLsK99M$&LmYA5Bh`VaOQpe0D@S;`&d1S5KuD`eu0sbg4r~Ul#*G^K;qmC0};{l$=f`OG>igC4?15r`* z?U~eH$Z^Z}1Et#)Hl~`zJ-<%_BjinZFDSY7(D0>*u=yq@p4yLxJwb*~KbhMeD%jVQ zs?cL3O4K467Pi~R$yJ_W%eT)J6%~o3OnvvzD=O zmBn1!NTL3a^%E)lrsHsp&dC&O-&kA1w1cd3b3k~^W65`$MxwBG=C?l{Nt;-}`(Qj^n>tF<6{U0$H;!@jBD#59!i zq4T6MQ#Q6XL^#wx23g=4t?l&7(OrN>$~D&fYlYCSOZgKF>Qty>u@RB>TX z$tRu_R*SFTJczYBTD^vD$&}iJxhvp+Uy3~1KAFSfcPiu0$2~7S-!M0q-W;(E_im{+ zYQEK^Tx>&%wY0R{mc&RS>f-tJKVEHS-Cty5jjtUt_J>Ua=O zX{kmF-mZtoN@m0ntxGzHJvx7ewKi@kC8=n;^m7D~I^kHQoz1$owsDBldQahkLA+5I zb6&AQPgZNsJ=bWz>eazZWAP}^@zReU?Yf=fOcTZe@)E5hoFn;0uK3|NY6(Vt6^#cBq}V~G=T$3pcIQwh%H4OPQ>zg~+_u@>6X}TV%6w`(z1TUnZ?)8)<5?M%?HB5u-xJ>=u6DznFP*HPB!Q=z z%Vv$+X?A$MD<-uw<}g=z{3G9H6HV>T#^QLoGd>LJ>$gILFPx@-B!=*4Oq8E=ScBUB znB=*ruj3t}d(u$g_*qv+S5+~hXPO!9)*5+>BnNdZJP&Ofvt4JiwkRqoc$(1SC53e7 zi@$&KAoL{ZqJ8TLDS#Ziua3*Q>R+QFT<1-Iu;jE^-(O_fT2Q}H_Ec@NoIdX&(52SJvyD%J<4r9 zU*zsSH{IeVDKOGn86RIt?c@~y;x}4xV)TjjigV9e<((SU!ap75qOVJlId`U{<~2o|k|0`8J-jS}-21h#U3)w7aH*{8(~$dl zp2eS`S4<^HuhDzXpS_y=h$a)1-{z>E*HoKMIn(QC5fkO^tk>3PZVqG+v(ldid(yJ0f{8-6M+@oZM|YFD$d4fR{h;R}=^eyj z+_9CCE#)J@zU#x9?tS6#mYYaGbYb#98FK-6}5uwt8|>>WkfaVW#~W)(9woJ zjyDLjb_*5^pT&{S&d=d5`j#yanY(bi9C_}l?w=9p8L8)X$j(~W$CrWhm?l?=pqewC zqZvt)qk80F&m5!m!?dRulu1?9gFRyO$=fkkQ1v)*=KeH$9U7kY__sd~ zc$Z3lJ;HyKp(*;bSD@Xs49_*uY#lk5tci>tG)tLcOQcJcOQDgBxDqz{@q(d?OL>BJ zMQ*||#zpiQ6|HG+ne@J?sdN79g{_7e`!c;-3Ox(MJWzGhd7s+JE;8{h#m= z|IL8(N*xz%<41dYI}Nrk2&?S{*6Qd^pC5W)-Ewyz2@DK0S06&L0vm52eWh`@^PO>! z@>AZ2wQ_lJksd>K=)n&mFI{DCL{L;z)ZUi6dvfy2IQgLmsD$s71_WIO(pu0XK0Y4G zGk_Y?;oeFEh<)v;Db z@l|!6w=O-5^HB1nZ7?d3hc0uPe@cLtX4Y42$6(Dzx=%UtGG|#5`5jD}w;M{P5W{YMjauB4iFsBU^Lr_~54j8wW56 zyU2-SkZJ~G^i55X1-kA2#Jn0bi5CziM5Yi$p)>%Q0qE|>k0(!`CJE+V+i>ym^V{9u zH=C?F1i2z8C2i0SYI*9pEYw}}~FMyJuLLJKhZWVLX4B&Bca{-~B zRSp?wYg>R7Nx4w3`dDhcFeD@dfalij0f!5^5ijZbmG{CIhn$=oSO~PVwA9qb$0V)^ z2?}zAkg>9sG-X^*(|hyg4WL}na9R@(NFfj^D~|z^78Vo$%CR{;vIcN9J*}Oy01|<- zv$OXi)XhM-0AgGLiw=KK>!+9sNV&$X^quV5MR7frHx5f3^z9yR~U_I3Q;_; zO%SRm>w=99a;o-xcV)=?9WWU9tj5BkqT3rAUOqk!`;t~5I7)oy093lPw4|)6>Uz9Z z8pUcdU0`MJ{+-9U!u9;@*DsWqQ(?DBG6aH7J;U3k7r^zbjxdpse8h)N-UraG=6P^% z0I(GZtpEac8TCWJK}}yC6*}&l-nnxJSRFFaFek6fwswpP2x4n#)GT(po<8#EcU|2} zHa0foweSy9hD3ICZkK>fK`{}))_7BLa6~$e0W^xt!iyUPedOT4PEt~mX#L!Eki%;6 z?!9|`aj&kxM2n2fkV!AwgHjtPJnTR@Dsgdfb+Br|Dj2 znIWG^LPF96P8b-0#l=MkI}1{PRlk8#LUza3PdEBDYMZ}*Prq+DJ2yv$?Ok&?kB?l~ zB!qtY4r2r?pPEk}P(QY|Y>bT4Klh^CL6Efqy9^SE1n$FRyez-A)>T5{c6-6wrY}9c zy`v>2V=!{zG^$Of+AGjg?(Xi0orBF8XdP8(8)#>WibL1XaR_*vzNvn5{`1qFRLKA8 zwQHEfJWkLfDD%nw9r-8%0)okox^Y$HUDOoozx%2aQ{AE_JPOkW>}vBLp0i`>Ld
P{RGm4EDz6pjPklobtw0^)JqS(yNPWPu!NIxq67AB@MD5?-)faw6iD!kRi zc=r`NfA^$;7|DEZ3iDunvU z2lo|#3kb1jYTNz<@SajP$fL-|vedg{CK?)((+trn; zQSC7IpvlB=cPTX%v_$L0?kA5QzofiB57Jq3a_4u>QUJ`TU>Ye*gS9pi=T*$J`hNe>W#hKKKpn;?ZSF!T(g4seD5 z^$W6m+DZkg|9LD}UR=qF*_s2&<9&Pll(TBql8V@;?StbO=8IJI^T;c z2X3kSbvsq`%Id1x7O;vYyUblAgcg4DS z^|@ls34}Db6o+7hy1P+FnsJs;-6}a)d`97mxHpkN_!YYrD^GIgcU@ zA?Bb*P-K#QD2%)favZuo%5epWbXggsNtY8r=;`UfH`E-#!&5fJ=%n<;&kt;!i5lC{ zK|vocgcCOvsZUIG&)UvUa2BFiH+yAyz_IX4^1f148+9?~zrqCpFbMtD4-L3t7 zu<&uPiBbm&1OJ!EqdQWdJHFbsInxXbT+P8WI!Rkw+tJza`ha&rKgIy3>m@WDFw*Gh z?fw37#msCyXw9Nr?eW2Js$})OPzM@j&o|!~r!1ay1Gj`SB7sCtO-!A4M7QIXLc-l{|9ic#K`EJx*?rehy^_-&oy?c2Az zT=b=tuhbP$Zsd*4%pYj0V9=YN$9ed$g=Su^6jX@!e)5S@l-L)C2+|1^z!2svrsD^* zt--)jwzRggzyJNRwGamAXEi+o!=FJe%kz&5DgFaYa=^nva4rsHhUes%hCaVa+1lA@ z0#`UYJ4;DPQA014fV_eZ;W4hBo2^nzk^uZ$8%E2i?%vjxg{|#^3klpy*T6t|Q<*Vh zwNnu2t=UUT_ArLi1F1g+{N`2@$}Rz97K`EiQcYZ}VwkDZKg(hZ+%67SpdB0? zHDNIgmgH$|Z->B+4G8Fhz~()i4TfA2ictsmu6Y0Fqe$}feD%G1_h3nCaehmTNsG2H zKc7*{kD6L((k#VSBSp(WZZSRo9rGD@woHje`TDs@xbx`Q+u4@DabDt7v4{e#A0NRR za~%^e4fmskhN!Uz$5Dw`t44u1i9avwRt3F!@p zQ26EhTHiy+Ay5=RAwh(NB^_}I77*ac7doSG@$rG+tAe}as8?D%Ddh6QV}T&#farxV zNarUN{4!i>HWkTYgq*C7`ug?m{rf2G2bSz`w)+9iMm-6QP`cb9c=0tT2<0TCt33&7 zKb(7Db8=t~DgSM=KLkG2lP4cC7l8dB(L}P?Hs?Cr$g+*TI@r@=Zeg)JR!Ud; zC8DxjT2qtv`STU9A;XNDDQSgbw$yYYWA>ZwFR8-KGp65|o5n^sp>f%Hcax{R)xvKeF`bb)99jUXf8iVu^&Z47 z2Tw>y_?}{mf9EKcn~M$_GjffaJSjwx3~rG>2i+hKW~r7+oQ_UTYMh4YjzBk#jE!lQ zTO@B?(WtcPoSZys^u-+?19Qpn*cd3TQ?;(HP;c$+?SY>3wN(dBnDjbO5+39t)uMNA z%*?t9jAdkks;f`ITwiW^<2C`L2wsa^*VA3a97!vyBKQQ1jeUK6P%6DOGjoL|E+6y6 z2?ClbR|A^b+8UEG6g1yj$lTQf2jKI#L(CJ>K9@ZD;Ff@bLKu+QV;^-Hl}$}e=Z;06 zVgvs={{#%QP;CLR*cLz@Dnbk;%aQe?Dc1;h@I%AcuyN|^>(?HB4+)8iQgx?WAhm%N zxb(DA#qU4tC9Xm4oQ3_|dASns{A9M(KEA#)M3=L8GHRZe1>ND7>C#Q7_yLPS$oKEn zR!cI#9I zHmPfA&oN|YZ119l{46kjfLy=$^6ks?FCYBl&gYGgL1C>x;ir7@d4EOv11S`+4U7>+ z4GHQ)$jJluyq7Cb_i?-P0QE~y`(O(Js7)2LSD6JQB>ct5nHU*!{)6&ZYS5YMggSoq zZUJsYkT9^a!rcDR86kB~DJ)d@DO53x5%6h2PCE(#9k`)x`2A&^uF2b|@bIX>z$d}2 zc@#MPS3TZ*=8KOJ*3mPS9x6sXx6?47(aF%jKn8{63!S^_{QaH(#Xg+>Azb}m8+-nL zyP%Z*r5CicunXp2eG^YuDhk)v*H?C7@I_?(0QYbG0&kq+H4OQe^m^{$X zGII9x+yIe*A2pw;RzE5)Oh7P^4b<}j19vpaW73k!j_fPEt67IU|7-N5MZ zkdP3>W)nKPTv*UJU%YSvND0#s*S8TsFRXx3^lyFw8Z9**u>isBHX(;t^oyQ}iSJ&w z30|Ne9VnAf5nj7`wIh-xv%}OEhlcUO5zA}81&-a5XX_v6@`%^hCT^iOmfIh)vMO+Sq#NeD1`8)eD33VtOpUh5Or*)AYot_@kl?R4a80_> zVMtSj8Z14UpHBfhlderm+b?MIv$C)dGUL z!u)W_@9dfx3{RFyKx|BG`UQJLq}k4_XHfrixu>UF^(&*J1-z7S-gq7=r{RgIYD1%Z zIHM|mLF-BN8QNU`CRzn(0O~%NT`2+`zwX(L3j@$t6 zExYv-m%X1LsnA63muLEJXLIg{WvP`?9JPFTc4@%Q8@drT1vxv|huH4~@X{UKK6fl{ zNbk!Zuv16Hm)pK9=5;OFm5fx^0i&35F16NIu!y7TSui-y&guZnAc5+XlC4n$0TrIb z`}b0|lSKxys50{*_*hJ!9z#6E@kPOM*r@`%z<=<@{afx83s858^DcwJ}G#4a6|n*2YSXv_}tQ(7i#X9WSIK2a$1MUZRiUMv;Z72x93`Zha~8 zY^oR$C~S71Wj6Kaip$Lji?d#N5P&$67Bnz_{W#DpJ-u4G zyYt|W=l0>oKt%-*ayCr_Iv#Ai;fd{6_z_&DhaVB@=%ywW3u|@s(QH*xjddBdgcikN1#g z6Q!?&B%}XPIha7MJvlmZu&hyeg|Mvowga|`gOZ6RK@a}xd)y9N7+B6MwiUujip$H~ z+#F{9URA*--R0%7tI3&&*`8ujTsJbAXKbUO3RRqMY>w`#T3Zzz?`#db9yI*ozHuse zyTDAV(nP<=A+7k0>RY-Kv%Q6T$=FMvx!yXc>LT~g@n64i0>(uS{Km+02*?C8icIB8 z#YtFHE3K9a?6>tUR!0EEU?rJdTx_{E3On2s<4b@+0;iB8Omj$fKzjehK8>VKSk01E-6$tcgd<_M?R0*ZOT3Z{j(Y?>`_U_4_`c-5)Qf0PI zgqY=Ystb5C*4xL_uWue)@&6EFv-IDbt=f+^S+}c}yc0o4_z@7`kEjk5ZY+-GOR3Ec z{@Crzv|avPVYL2dN+Q`R*FC6V`c~w6g z%CBzk_fGU~>3(^hX;!;{)ZsZiF8}8e=~%IW0D}9T86nRR=ign(Rk4PK_p;bZopXLZ zhB^cg%J%*~+kt0lYAQJK_vMmo-RrHtRhs z89?@7-nnxQZa?5y|8IarjqMGLJ>&^C5^WJKixf~kpqEiD$I`{ot}Zj6(amnFYOulf zv!VAiKou%0D}!bL|0M~02WwboBnxQ*P*){L5A2{+u$KcKB3Ig#5<~ z@P2niSYjo($~urmVNs>1DRHT-Wlv`Wiy+qbl!gjO7G=YOH!jr=KSCgaE+;*?i-TLZ8+Ut z?n#l@aXdIMSZzEPkLB|4;;=QWwCws(FC@u6JT#Seh?`R=ELrM2HaKRBEX>bH&81VR zwn>!d_I)tb;q1ANjq2FA6zEtG2z-5fGE^3``7!F#*er8ilFcltCM$rIE&BIBZ!h1G zb64jr6_v?&AKUs!uF2B0GQ0Y)+TbL`E>k96d{IQKKS4-v)c)Jyl7wk(*ddquDP?78 zneqHYCdIm4PxD6Sa0RKMRV2c@GHX-p%r|%I8%&FU4y_4q3(F8+2F_Oyob7nbp25P!Cn+3|%a&qZ1e!9n{=>9KA z2G{MFKE{FHwB2aL+{l0G<_HPd^bTBU{pL6A^is%mOBBDlc&BwFUe}%L%1EBt5_}$- zGf-6QZg+A)H&iX4pt<+aVYOMtPro0iVMLR%M2W|yu*O@f|Fxyd;n<{KsGey58*kP4 z-v{5Xq=v4cIc)$*?AxFnA=Xr23<@~$lmGAuIy!3qjZessi2qvqtL5%3_pq=q4h{~C z`hIJH4J>VC1(Lv708Q0SZ{8R@yZdecMD7h&cvXp-?9lOzjo??8KZ_54bQ2Kx`h-&J zhhL?&g1PT##;$7P;D{k%aV>|6Z?R#vZ-#Y3*?!Z9Pc%Mp!qON~pC9oxtq1f?rnub{ zc{#W3a=5%QKW@41Y^qIlXCZpwGFn~bt$*n2R@ysT8Ng=6%4)j&^0#x7cD~ic0h=Z* zEiJOLOd>f*po86P9}J>hsUr7AfP3-lDPJN`Eihcz-eu`mP~B{&_~g=cJ8|9QWI;it z;D7wuz}1*lRFs~G8l8VXyf8ynrYE}13&8$D@?aM0`>0RXsh-xM<-SgqiC=!e%!uRb zu6-f7=VJczFWSWNl}89eE(MCbH}jIP+eVJ77?jP zFc+xG$jFGIb{;4x4SQi-#KB#@o2BVj(CoV8<>a_Pr;3Ynt6^qgG3iUe3}|*#J_mdc zwqjU*hsxkh5Mi*6`1?yZ-vQbYY{6{sMKB^eRPx1pfu!`=3aVG6KETP+yu9wdzDzO+ zcfJoolqgp9C29RldN)08*mM>zWd*F@_;qCSGWN%%IrN&Ogwt#N#G_i@E7tB?aS1l1%zj?AL%suU@|N1R*KeH84Mpe|2>&Y#%~Mv9hub+yNE1tFsf{VFJ($$QOd!LZ>fpdy?M21-m(P7?q)j zdol|I3ET(JnhbA%8m%NWx~mIz(tCJ#NJvP~;eW*OM!-)y+Tj%>d^fApj;!HsV&dX! zYilFMtb~*j@Pp1y-FzkBewQga+StOx@1p2!w$!9-|q5-h|B@70>TN@L9+X|KyQODgw~_Bo}L_9 zf>sLxdZ2JfP!L}s>{?w1+84-ZU|d1{| zS!MS|TbtzWP0(O&K4I1(CMM?a$%GeufGH)?SRb@wgFcVzV6b*wpX7G<`Q;0Sk&zMf zA)i-%{w5et8nn%KHzJ=nIy!Ppih4T12*I&Z+00|b&H|pTln@=Dx;BexJRStg+Fq-ep8pF-2 zAHf>}+95otiinuyi)fHKxLnWRk5vT)GEcDqNcXOWYeaS}Fj;;xm{*XI`Kg`%&P8+# zxS7QegHREe!*IZ~_$C?>67u*`%6g2ar;wT&fw%)aG{YW>GwHs`9E#@Y;UOp>z(5DQ zBx-`Ig7X|snXPv5;B69DvKWsN@8LiV2PMIQGthT_9tAKs^wMKxs;rVI*i`F zg{>fAfq_81yrQ_$C#}pO3>}*s^SoC_Nl`I8XJ`&yC`1&{JOghIs@N()0{KPKqH7zkp2u`y8+(;JBeh-E`blG7l;;ktlTrPaIomSPVNQMLhY&?Xe%&s*09&% zD;k?6jO^>zuff&QF-3gk%9V;&s-Sz7snLGlP*YZJ4T(yqFWJd|ZPl44lAMv2_IuL~ z1k<^>xd#s(!1$h(2#^HQUn}Zw{`xFJEG#y5eRWk#al2q-RRvq?!%Hn49eB~uGaO&g zh~i@gdsiUK8X6e%tT4Xny;jdm1giOs%n{>~mycn61nMZs8GJ_lW_RCRrpEEj#BI1( z*2TNEu&6az2JlzxupuAtwE*`!fDc0s&F}5r929g`yB&uF&seC)9MDwgw{9iL7{FVe zfP#kYBcQ{?$3KB=4qQyT3vktqzI}*8u7gC?QLLq;N>R~uZQTMqe-#7+uGYRxqOF#o<6Mr zS`GNP_)on+MF0y9WGlA!ESQ!$o4+sMjF z&hU0|VPWt+i3khBu)@~#)WdA>=&8{LK+5y;n|uEP#R*zuL9&5>=HvSb77H-{czSw< zh2ey^1EZU0i38I7)yp4(gYT`G>FZMj3Ik(0Ao)|T5Vp&O$Huw>$c2qXimt;l4`4f1 z^9=03lCzh?##D-z>Rwr1Sz)>aLyCU&BWx3bm!0tQ^Xn-ML%#d<3kk|1=!+mvr^Cey z<&>O_%OmQ!uuu!6VB!xFr4BBA4#G%NptpPdDq{8Fof$xh*0`S6WM)2pl2lcN@(C3M z+9-kV3A`b&V!*DHoiVdol=XzeO8D{J&z_#SO8MI0QiA*jf+Z~S7&mXm>3V_@YJGjZ z7Fd7qq(PEIc0@1%pPEf}9YjZGy=ULoA_4>B&xRlF{wW5T1P(BPfg1;|9Ls5Yd(JiT z32c&rH5mYuv~)b|YvQ?~Bq^B-YQGf&eV?g;K`TZzI5S{(+|f|L?O7Sv_1@Pf8!rXX zwrdTdMi`iZ1}Lb%!C*v0z>BCj93#SX6`JQPBCs2%aWcUL(LoE{#)0)oBmBDBTm=_V z7YGIhioCTzUnVBLg(Wt;T{mcQwAK{53k?k~A^mFEQPBdG;|dQtScCpA_TD-w%I^Ie zMJ!MWMNvUOly2z;2Nk77q@=sMYw!_7K|n>i1O%kJQ@U$tgh9GHBIv!10mGu-#S_rCUZU7y%c8+Sn&eo%us5<|2AT>}rx^GLwAF*U`G74mpf zvfQZzHX|sn0nA^6B?37W)bB8gGI)#dSCBF;;1f{@g55ziw29fm$8xl5ASwgr5y)eM zH{MlLxQB*8>$j3~3qbD}(nBD#tPkf+!@oWYx~JA$@qYhWc>C*aoq;MlHJ0ek?7XGd zftT}DGs!NisIicCdZB-}<7Ur3BU|b_y9H~+JA9kf`pS-68%=B-4E+OuWcawbuYt4= zs2AJUB0!pj_z)aSh4ZYPd~Kj^>0D({g>&+qvsv`93l7_^i{i2 z7so+q`sx)5`Z?g!LoF(LIvRekp__H`=)ld92*i<~PpOKRx5^xe^pCiP_*QI>-f)KV%O?uI591ST6; ziV%e|);06=8e#c^WD(Zz8+G-zjt}qm&j&QtZRdUe0^9A)9;}Ys#WE@jSF_4D?W!ia z0OHEEd;A;FH-aFQYWOxn6Owr>U5MZI0d)hY>fn#SQ1bZ<^c?8J|HHPnI+!&Ex-q~= z@Tj5d0YIcSWMvRo@u;!XrbqhER#lKA@-dJgLlUkI3?9JmAWeK0OzC;BWoWbxtVmb} zQStE*_^@~jwr#ba{l~ItdJ~eA!zmja*$}DZ*Oc;SaH(Tz8g*t*kHRiXc$lZr93n&B z&x!$p0A0wV#=JYBkQ?5*1v~2i5uoL`Zzm0lT$kk%6rR)E0Nj7cHD57}g^8)oUXFR$W5+%8V%Cb96o*v<%1~$s{M|dya!s#0+#WCZ_!S;v^;B@^Eg5?*=^8JunP;?wTN zl<)pEr^MNy`}|)TeDMPBvz!P%6LpcM7W-L9^;qtoJF^pjg6Z7bc4E^>ky+l7}XlPvmH zlclpWqG?7_-l$7cJvp@~w6QvO*RkeIk5Kyt@e1F@#KZ*hjQ4Vc8>IfS&!c#p)i#FZ z>4L~9Z)3cPZwRl2s8ALe9|;Gq&HrNsX14B@=Vkeb?aqA|%fih70cpZ} zqA&37Viv=N_9?ZF$K~Ol|JwcWKmMm~=hJ+MaRkv`t;pIWCx}_4eWcA!c8&sQ^O;ZruIjK zak(Z3!2)Fm1e;#i<^xVYxxdai?oW>9VZ;ch6~X7)oTkRBdQ)pCvCWS}_|&&<njfw!lRZ)#rt+>XE)Av5gdG~pX-RKSXtEW#-oNyfL+YWf?#a^_uRWj4=8A~8F+CgW)#C!4WC z$NjQ^FFv19^V!Fhx^KFzV&{8XFBE0|YX0;R$d$0R#^Y4&LA1ULGr12ZkhUyU>E56!WSCDs@D}9pEC^ zPB%fmHdNwGEFKu~w1)d~9FOze&bRNH2Th6j4NmRK6Ec$Q=US+cd|skyzJUXI5_9Lx4FdSa=aV8e@}fhUAsiSn=MK5+5;b9kDb4#?<7Y@kGn)EidTQ5N1EGz zO@vjpNuIc8;g83}h8UatL~eB?=RKB)aP8eTl6SNmqRaKe0F0kAUOHZ3T>Fvh_P9pu z(IuycJ{JfuCaZ^8YH9+5%A~ul(PzmEnYcbpEz~&h>i9bO{ri28t@E{dffSuiCSr1W zx&SiWhK2^9+ix5kLna8Nk6a{YGPr*LA)p$P@E8Gm6J)3q&~EvG%mZmhQ)45mcJ=Sx z-aIUbU~(~UJTcS{+{noSp9XPcWI>?5?znYSg= zOn0l6t{&(XD@bMEy*2xE(K`HS$Eo~8O&j6DGe~w1IO~|W&=6vJ&^*@u7xo7FD zkWMYm$0v3GMI)Z;>Lxs=i2?_Wcp&>`r`a=Kf9P@4zFTQg zhW?WDri*&cioPx#jn9WC0z1hJ%#xng%6np~iT9xG^5yYW^7b+3>6=$B`qhJbNjF!) zv4>#)c9+&;ZTX&z3e3RrOVT-`Rn5_!AK`6XT|7NQ4}FBYZx$66QTr5Hj;AMQb^eo+ zLIS-r+JMG6o7}%=J7<))5%OB{i1qX5Ex5;8zY zGVt;7fdKs*8TSLoGva(8P=|!F`fw?O@m0#oXpuBXw^Vg@@X5qA>dq7GI~7iH#a`6h z-%{xh{An^5tg5J_PbDJqHlP4Cx9V;+{;|wydXEZ8sfeyHY!)4@*57}OtO>eD*j%FT zm%p{8&Ff2>Dy&vwE2-vM7F^QI{1AG`4}6|9(;Cgr4AO3$dY04EiIm$M9C}xa^p8pK zS;?1THo8wFZ-h`fZaLBWIPC5VxNnX&GV^gg9T7XtHyn;IomDi|WILPLJ-OUMxH86~ zs83k#*b+$s+DC>CsEZxa&?8eP#>(t}{O~bVK~KOV23QPO>7_tR2XZU;F+d}u+LflK z!?m>N>e#KdvbkiUBhO@C=~Gs_YOCEFq(^A1w~= zSl#lAdmHgJF0_zbb{hT)N12GE=gfBv=j&@XY!(2m1@Cq2H_K2hFmtE|zP7-wKfZBE#9(*r4J-gBZ^L7V~>TibJ?L zXy^7`6X56PfAApD9ONA34lksnupJveu%gQZR#ZILDT9Xxe22fkKQw)!*4|HwT#x=d z^Jg%-+CC2L<5hGse9BH>pgNvD#i-!R zSyx9K)i;S}v+5YPTzbc4vi2=hoGR8M)t>IqDw2y6J^+d|)W}Nri9dmu5Bfsf)9efk zr5*=%Qd0gE6%|s+QP5d{@=1+l5L=i3{o~iyr-*0)Ln3IN{&@EyV!gCP_d=;DENd!J zFWy{9Z`mXjrD~glrtRB_M4pu~qBc5YuPDW;T!q|XMjw8eWPN*G2L{eXd%*dE>RD>G zz^kNV#pYC$Sf#vU9M@_lqfp0Aa zS-7LmSyDcsVl9pPo94-bUI?a(8+KsJ^PFZB!YH84QG#qec>np<^*>~j<)G{Xy+a+8 zQ_ydR=B%fOhdNNR`X8T7W*`>-sgm->r-m*X?B}u+XXO)=Fpk5H3upNbFhjc}{u2|z z&IKHt|C`Bu1#X?we{%uOKL4MP(m5P!-34Cbl?77r6{LgI zn5)_U`JEFjK93~E$AtL0ica|~g{jlSS6SKE%G^$z&=Z>c_b5>d3py1}vwS|E$x9N0 z<0^CA8jqXs&*(#wtnkRbzXfr;RI<|CZGTjpC|q$2r|pljNJeTJI?=_MMQPuu#631k z*j-kSk>?8t@rr+5M$E1BT@ zDPSSKE3Krge7e)li)g8{9=di=>MTAVRP=;_k?GWHw<#?|vdAtMHZ&>OvAyPBjoZdC zqDWb~klFbFj9Zwgp{b~Y0@WzBSd9%jfdul-Ws~ir{l&R?)LY^k;tQfMqFp{<^vUW{N~EVlUM5M&+xuTC6}x>Rur3yxYps7Y2@wK+9FyC<`co^wX{qS z`N*;ok=D`9{=+XkT$)|8YM>$&lltpn<5Z&Tn|)#gXuTJjID9o8S+*^B6A4iWy3vVl zQi~KQslS}2Wnt}*4E`HI%(=b&kwRF*Jb0u>T-(IVW7+ANrz;mz(`cWDwzfP|Xj232 z$f$Z}grJ;>S*2-vd}AZ6U75#;wNTl;HB3s-GU=o{$JEYrVGSPjZ8kK&{aX4QS_gYS z-k088xHB0TBkIiKP)~SBFF+f}TU=PQxTQ-!&zt4uu-ksqkOVPBLKTHCdx>CWR>xY% zwu6KB2%QrbTvB08-_0u*G%ZQg+1AC%#@OjX$Hd4O%V|3UiEG8hzb>Uk0p%nrv*oYZ ze*MYKV(G-ARt_`5)r?hnzlt6aV*8T-?b%kQ5XzK8=e?BGre~KV#MV%4zpRcI5-fOy zW^E6(yR~lK3cujVsldR!xi|x`(@m1_CwP$u2R^>`DSQj=|5jpZ5x4{!&&)K=L%X%s zqGzNlG4md|`PI>uS-WdHyAfSrsDZJF=xyr*-Yc#q#W-?Hb&OP5<)L-W){Wh%*+nu+ zTfs++$@%$TiKqVT?X#*zME4R9`dR8(FtoKXGPBeBupHHBN9b>FT{U4`Jk&aalfRon zzGrd`k?>*^W$f@KS@G7baE^+I=9|5)y@*#N(6&`qW(#L7)gZLIeI<~bmr^u%er|Nn z%AqndKmVQI)$sNc`iMH@KLEJqMfNoGD8v(QO=CPejS9$k&q@#PXYaAk=B9r@e!l%8 zZ+ti3sj*J+6_xqHFMMkgrMFi9%z5nT{Wy$Rg)E70s5m9%*-^dx!Xo~sz-tbUg=HFR z)f?0AE?pAs?kIN@GFoW%)VljIF3y{HYbCO!t1BusdAIHK)N6vgWQEShr{nMbm*-`! z7l*8VZ*JLC*~}XG%FOk*X{mg#pS2^R6g_x}iEE@yt~o8X7q6kLrVx2xJrOF!jO*UN z{z49-BQLSxd?OuAK>jw_cH+tTlPngWK-Z@@-b>K^sQjW&&wfwnV86I3wzH#~K7HLV z@060q@vyd9dp-Mlb2rTTroOrk8I!slTdSjADOpISP^t5tRd?e9X?->hDlxJYC7$!Y zXqgcs7EZ9XJIKbDBBbw-Ex0?I)Y!JKK^gO&Xi)_OpPOUKL9F=H7QAC!gHfgyy84cP zwmKg+ciWf!&U^e$R^}7bpwWVC7W)Zf?L&R!n-12Jcpn3$nOK=6A-Zp{2|Av-O>1-W zQjahZW6W^GN%Yf{*gZ2DYR7=F@?T9b!}?=EHDyNY&)fa~bXg7#k#MjEgy&H7Tu^W~%AF-6b0$~dURTEQ7~{>hMs(Kdz)3x&=7~vrmx$kMCQh#% zg%>ZsB)mPhToca9$~y`kLR5Cetq${WYkP~NHd|;*Ja#V z_m!v0ab-Op>i_g_&MN{W%uUG~ofUV&3B9)6m2l!j`kt(H+9l3|Na7nL;SQ^Z6lEH{ zBP9}{nk=M`I5Tx0IEPO4J@-6^GjU#-6d_F@FAaYG<}+o3(2xV>L(jMJzs~u2`6kE9 zZA{Fot8YHxll|zN=L)pSwhWE9fs4}=Vw3+dmm`&BF+nwxF^sH7UQ|4EcO7`m@IVDGrY8^MbQhlakUmw{mIiwXqWU zq#;ic0IPvHK*eJQeMa3^jh_KrAtB)c7sr$w2hi!3j}_c)y8~?Z!~`6&&xnMd`qS<*bRnkZ=6sGfdn!%L%|Ysv z5`+sqzQh$_x})TXnLWdFho4@TaJ);}gmN#rfV2}bKx3__hzOGf1kels;~4tAfu+`i zKw)f2v2ik zbmHwi4sbAj2IG+5xf3JgoDEnPDDHl=4g+Ys@N+ToX#i-B3!t|TE|6gQ295kcffr6r zdq6L&s;mT==uJ}6XXGF!XbvPVF-6eW)Hp2D~Lk zMn-^yoRBKwymn7SgDo7hgOZ0gtoJ3igC-PSrtaX14@5*dxtN8iDNeB7Tv#wcv=E6M zW2^En&-;_5Zwp@X0 zR@Vubuj+=9cB^D9W_dl|C53VIGa{LUR(#6LEzOhV<>jONHq|19G-{unl@ZkM-&l#b zGE*oxBw*Y7oQ7AZX_%SKL8J*f46)-42~a}8*&tvINr`y=^l77UyeBCYRSkGhz?DOz zX$R6^pU*%%fT9b41TRocfYtz@bNKsr)K#dkZnLUmwGr0fq5$eetbRT-^B4rTP#uaL ztrYnCUx`9lp{qss`CVYtVD@6>qODB|fq8+2?SZHOFr0?Edb!yE6QV_u%CkR1h5fDg z#%c-r@bIwOWmB)t2Ooqphrqc9$G!QE$fUBgA%cMVKcK1vt;QlKU$`%)jI0C1BILL< zGe1A%@eE70nh;xJc7eSjhsA7maZp)gx)E0g$EDrW;|jOtzBC^Gl`CMm9}nIi(9o?4 zJ7+zLJ()*hO=!k#1DsP1MGO$W%7^;<9e~;o7X~H*d8#ZEP>})p z1&up!fqJh!#QS|Lb&y7QO_ssf@|<=B|x_@^u|@8hxf4GkmXBDh6B%?6ls zad9zdP@p*lyqa++>w#`9cDV2m@d5lAzA?UnrG_w80BM6MFok`HJv~o=0uT2HI&`?m zIWRzgr}4n&In7siIDnknl>LTz3JlqX#>OI8(bN-mfy=+kxK37^V0?fh3TZxw-azDX z^TrK?GYnI$P+J><-*rW&Jg9_CzYKzQhB&5t|1yq=I#~{GY+RgNNhE_Mt41NDS22Vd3HQw6xa|0AWdvL7^zmeCIj> zC?s0NKZi#~BCYP+y$hzYsJF7CW%>D>!1X20ht?dxRwM{JdwYCpSZ4r=l2A7c41`ij z^8u~}^riuMD-#n2Vmy#5e+44Om*-N_(vf%j*DpjZWUJBm3S11(s3>f42`r;UyIh#A6x1~Q{1JqN z;h~`)8}%qCV)^jDLy+|O48{VW=$%(3Nf#kMm?Z9Wondd z$^K)?1O@pic;KF4l(H1Pj#UyXo@)jlpke5go})dziaPMFXzE4m=+f!Yi1qp;vP zje4%mW7u_ET>b)-zKB2r1wSe(sw?8rL*OY23cB_GeChCZEgFm(fJM*@p(0zWD(YW9 z0c?~9>?ceDz-=JffM!igK|#mLitWZn=tw{}Mi|VB_^zekySlno`QGgxn}d)6jF=`oh4@dD?#{t6zOnqU%kAY^=TZ7+$>!&W2&t;aGBntTkjJ1gS9+0wV z85v(x@w&5aX2L9h-wxn2*d0TYEqN6zd9XX=TUPbLpL;pR5B0-*egZqv-2V(Ojk-=i zSIa$QxTUpQxMC}=IhF;w_sbuLe77$aZgV@v(#Fve*PO)yk zkd=X5H`Ww6aKm=!BE@vp*Yl|GJEzzbx`#nl_HD}f%8uaCf!>5H6+zpnCxNRS-9bPL zhyIt|C@>u%`m1PYbj)urlIf&FY=I^2aE^xSq63!8M<=Uyh~0DDTGVR|NeVkbHH zhnH7!ZDd29!6<78uHV4haEGVo?Ce~o;RUgF&OQfxJlug+49#WuJWQnKwovB9lGd>? zcSwm~Fbl^(%rvzCdfH|xV3wjGvq8y=-NL&74=k9RFDT4-xi9%@mT$IVTDEqLoa8$s z=42o8x8CG3C=4Jsz&e-1Rt3xP1p1+UyBC3@d<74$)bofpHXSUSt0C5ufBqW)X9^z7 z$AbmPA>{L)7< zinAftql8-t>tU2$VCc~~FA*8)1*VApl5=nl$$L!xd5{5lyv7_@SDQeQrQR@ld#1{O8& zGa)9}l?jWA#))nQ5GQ73ozxSF_0aD45Cg$?)CzJYW8-jOev-(g*pYXlO)Q;A0}PFg zn^_!k@@Rxe#8(~rGm=iUNrO{U>0>28Kc#DCCWV@VCW#T_Zd`TD_I<3(70GcTaPUbpg;&_(_2JN0WNbQjNIom0&cCJ^0|* zIyz##=fDdFLW8+566`_5@7p*y{5UY%-9|a>n%)FzDWidA_}wF(Fau-t2*ITtd|jXg zg7#a6Vt2_bIQ6gyj~d$bWndf)A$t$jEEGea&4;-4Csdb@4E8Cbd&3hGlxIb*_`{w_ z{m*3D;0U+LLFqCEntk89C57M`zPk%j=wpbTe%J19I zSWMeCE9Kv5M;3WseT^My?|&H&G)|yNfd|_o@RY))$0dgjvqK*s-)XL|*9W^R$ZUaN z1)R1F_l>}JbX9tJcSg<}ZKiuB@i=f`f8$7F;Q_`O+-hi882Fi-7)mxaGyuDrNc69_ zoaq`WwHvlokj=`<$Ura&kBH!RS{V+fXuo*}P6?VQG$ldFQPmZ~T!xz-8yUg+{I#>! z{HVb04p2fA0~ieAWH_N}1il9@7UDJ}UPU^;aA77xIt}|nWK!*X(vjR;j@`)GYolTe3=6Qa6qb)c(1b@fH#bK@C-WEX4*VfD zz_KkPbr!B1G!)t;R+?U3)#CU6VolSEEv2c@P@QU&3lINqlrf$zWVY=yZg_-q!-X^z zU>r8s8Oh}*CtX}npuhB!h@5u?07b}oAyh9d?=9aV3U42WC=DB~+uq8Hcs1D!k1Ag? zkNX@r;;8(5V)cuMQ2N6W#=*&oX;FfgyoVAL({5gQDi=3Any>5ilod-K(Pp1+MEf7@OJp{K;{ zj86XXV_1lPa2Z$)Sms#mk?z>qLx-g6Oe$$Zs3>8lT=0&>>Ff!}6t+*e?pL2St#63V z;Hky2j)tZOl$Qg`pCKIq>$F1}YDW#WlAsaxtl-!BkNMf)05>=+xomoUJtyv@?9P1P9QptI+-J`ObujSgso#J>&^W4+o*6$<8r6t`v5t*gz|$4J|J0 zA0U~>iin09^b+3RTYwdQ@d@EY_U-Ow?S(aTd+xo@(>fHPV300BMAPW-bkk&t*8DWeRO+5WfU z_Agac8A4jG5tzHE--2&bRpb6u@Joc}xK_UWkQ;J=}gh!8Y%2#^U4nrA#RTU$MD_RSN|tu_rIvb{`)uozd-jZm8F0| zeZWg?I{4erhghEPHsS-cSSM}cD>DnElG{AKysjD_9sSy=XUeY#_7nEcCgZ#n%<#2{QRBGVx{80+wOl-LJeXrvg_V}PRSqeO$a>3-hBwm09GM7tv8BD## zMzIglE(+2}Uxa{LU*^)P-@XND6qz|E_Pcw^0)~QP%K>kO6BEV1bJ>qxAU6-7HwEDR z={T;(lFQe_J*3K$@sB9f?NULZ>xiIxVyOyDw><=k(%$v)?^XiqMc!>}!dRp4q zg`JkBN?*S7tQ-dzbI|fwf2jK5)#{k2u+x>pEO*BjCa;<5^uCnuBTIgN4C3Rv-KS); zhoMXRke>F?jAQ3r#+O4)^E&V9ad)TX%(#*@0}k}IyN48cKYk`>?Kb4q zclglB=)bgmK~C>yn{-;lG6Z?AW>qKIkCtq`1 zgjo?S@Wmt{l%8Ds+m*+4V*6B{TRV|KgqS2Rk&}~3JnD8YRW$rM3y-g5k&7#LLkKBE zNF4dH^A#tGi>80r))y5Wn99eWxK64%$$`9m`Sx(K%03YYu*0|bMP_Z?EIj(sXSWHT z#VDpL=|A(((D&KbyV0##KGD>=dD!VLTs_#5a2~L1rLvLBt---D{PCf&#+HVn9yVyA zdvE%VmZs2~TSa3hxD_zMp^F+XsmJn;dfv@6d&DX4XL|wr@^}$LdIPa0CL`jsRd@Ae zP-KsRMbCF-LMNVTfgPH8Hmw@Kv~6ERzRC!jen#+!vmfog;bN**#=y)cM87^1A6B`Q zJ!{t(e-undBcjOkO_X9>>~3Ft{ek@Tm?f%BX0$&%7$6<0@g<(iJI3H5Gn*TnGh z%@hEsaYfAGA7VR>DW&1~KW{~jQ;~vBm_H=GdLLz!U*M@7O8k_edsAU&YfP0x;5aVZdhz+2M(Fp0!$EEZ9{N&0?!KA&9+#JKqQsyx}(23LnS zll)lz+L;+9dtee?`Fw7Za&?l63kw^}?54lugi)vxv)_VqmO;0_zp%8w`+L#yP068% z1u`@P{lQ62pG@8WH5-!hv&F{ToiE>Q~DQuF2L z%;cUsbi=)TL|R}h^T6kuH1m(yTT8kblB#NqNa{Ama4BOMim|#Tv!iUAd$Z$%~hpi}ot`)IcR`Z)e6k@R$N_fNUjZ;h7RQ^~_Z08{h`qJG6}l z)D@m^bs{1~fkAt(?7Vn=9X4ga64n;y>I`abcKW|-t}et?W+nC7bu~4;8_?)R-Ns82 zB`POP@1s%egepmzFKpGvqlRe?2cG;uW&BW#%=_{s@5?XvvFKGwsOkr7oMFCo=@g8# zllo%$2UEjk=l$e?a!5$Yr(t4dVQHvqw$;(r_lbCFs1;Qq;m#WXXP=nRal4+%-$n37 zi|FnU3%+?>C075tV`u&~+D1SFfK*@K+^M+CN{#fSq)vy@ySIR7FjiB*M{O|rtrzAN z%OH}RJ{YLd-H9t26rxadaa(*(dY1(Z_JNi$q)5AiL$0GWde5z21a`+@R9}1Yty3>8 zq@jV7ztr4&kEU6h4&M}@;;uK+2o~kj`|PZaH;$n~iMJ5nWbt06e{(7xWUfB(?`$5#Hsa zp<`>6G2zrRK_WgRRax3esUM#hbnkWfe71f3Zq*`dfIUHhfin4rY&_>^mNF;>sBHlGO?jf%{Ld*KB3|+~Tv*L9h9ICh`>0 zsBsJ?9CO;Sb3fI1f`?BjB*Ufx7~t!Kix8W#&=iB;E<0e4B;Td}1XHRynK7s0I)bf%P4XPxk3bA5M=joK{Ih-fs-#hQ2;7 zV3@KvNDTX18)rDTbhV=MgpOT}NTWLk`{39?*TUFtRJv!D(NkXwNEYz-sd~563;h$^ z{B9@SLyB3KXVLczkSr*ha`SS+tOUBcm<_Oo00JRKsIq`@n1UK3mK(ah14|s35PXSx zE2|ctt`SU;gy2&Lgyk4!e~nfV=iAx6;85RLlvNWyG?;rHlC=`2dcPem?j9k})6wLN z5+){t!Z)D`+N8WB)YMNTqq~g?@TlpU-a`OwliE$#d zBoXQzzMwgmwmdYp4!5|-r;@U1mB$FghqrN* zD8(bq`597T!~*&2q;SuMqMBPve=BQ{=Q_c~5R-XjZrg<^)SvjPjo3l$AC%sU>+#(g zncv!PT|qpTlk-x+$Q7pg9-wQYzf(O9j2FV;zK_l2pNL#m?)Cx|#{l2aYo<_imiqd5 z!b$mnv~NliN}@Vpm2i($5p^1OA$0TjBC6({-*OjvsJf+1)b?;HEVl~cw|n`}x<5dq zZ+(t}uUu(_lu?zXukF{`^f{4|_Y^rk;g`%iJ-Kx>*8SHrij# z?epQl?T{4+}{@USh)wp;K3Vew!7O_M;GjeoXW9>*t0Rq|rvjSA4hn%QE1+oM&b_-!X`9nu_Ym`ooSNKmVDi zUWI$e7+U}KEiAU`PNepIC(EHj!&rfPauvl?E2HGL+fE$=%g$X|ACBi~x)$$7!kRV6 zZ-a?v)Qu5cjvpBsa4oNhsc{imT`YiXVoztcxV>V+dTymj|uRx_LoZ% zYt;5gsDA{8oTv8LMb{r*a_Ven*T9UyL33$Io$*4(_j$6Eod_XsNvN!%kw zhJZ3=%>LxirNW*c#0AB?Vg^kOth$$hm!CY89NhM5etrHHg?qeQI1fqGHL_u86SI}^ z1kZ){9EcBrMHizxQ|8wxCkxDtC4W?iY{;{} z{7`QDpb}sI{iv!MaNxA|dSxOxrgRz>LyXy$7x52w*W9)`49h+EOoqMdo4a5_#Sr&p zxEsUi>`U|szxJ=Ls4FlKj^9a9Vk#>Ay{Z3RiE+}x&~&e@;`A8xh};WZHFMN1QbwS5Pc{tt0Q}S;>zE6gNI3tE4B_>AF61d zct^kVx9ogvw7zaX-UQ&eQV9w(Gas{9eH|B0&GMyODclCU(=JjJGat(~c`=r0ywuQu zU8MF{M@dOzi(&SKB)c=>`Ala8n&nLaqe1}g{_Hbx%q z9u+Z#1O=(&TMl6Lz{tu9w6bHQGRx5L+eV|s^ow+%$}LRH1UtLwJDG|V%LETunAp<1 zJ}%|kZ*)U#=q3Kg!Z%CX!33vS2uBoEbGGo`+ zZnYOmwF%CMH=jPm9E58!Wf2?B3J#at_xU0ELc3%$7<5R9)=HY9&NvGD516|(Od9^! zEd4#Uq^aG=c5}a>vpcF{9dRWJLRg(-!9sC&YSHKxYK5Mwu=OTChEPpAq{k-l=XFD&I7u{s;*TNDv*?A+h>{n**V zHI&BV?V1}7D{{AB8T24H=13@ts&i1%wwr=Y}E~2wVvsiX=xE9=5csn!CO&MFrDhP z6-wy7yL;gd@wq$VvH-f5dl`Itz>p**mF)bvf)T}KQt0}vC|%I`?pDCXMH!q5$i*PR zFRK&=|8c95vDID^2PdPHDo?A zxEu4GQ&QK^Uyy-VE-s1a{z@oPEGeY%Ht$ zlRLCrG`8A42u4Z30&Hq}_WE@OKMQg7G30KKM%v4+Z`UjXGQ?9SA zISIxh@W2xTM2JdEwDFME{nD}V@|rL!;sy8WqN1$dB6qHV$;wjs+Wrx6kqq!l@{`ij z2?z)@H8l@BPNBDt_yCf19S#@p5o&vEWNZwFB>517bG8MLFaS=^Q@sb4$?@?+j^oR_ zSazfGk5_4MNnm$n!+$R}HWrAL;6vn4rU7U5I5;Sg=8=<;l~+`x(O4^T+!3~$X~_@m z7TJ&Kp3it00E2Mt?G+B2Ta5mSDk`_HU6M%n%u=hKH3%lB)KAJZu;+fmnHOtoKp%pv z8@$}xov6-^ey{-hrG*7NUj#7!?D#i9I@!AS&+S*V}nse<~(f-Yonb(P=(E;!Ic$~+@f zrz9s2Xr2b{+xp_-$0rW}RQNrSQ91@6&1Tta^V?#CJJ6t2@_w6<0VfNAaUpQhOiX}e z4z5-3ijYYGvAGPL_FjHCEQ3g~)RYthacGkEfhQ-Zqj2AP6C+TC5OY@GLU!c(P6nn%!r^Mc6I@oJ?G+w5XqGYQ_qyoOvht0jkmt{1U{6v-UjGDzMt# zy?eKX_wh;H@^0JjwM`c=ekj&%98W1Ics1y5*vfb3+NF2D#;hu`23N65YZkAzb|Bew z6zENTiw@jm1b+1ZSN59hfWro`JrEzHcwUDg!;PdzUI_v)l=M>IODM4(z_Yfty=Fyi zJIqHkV0>Zm^Mzb0oMz=@W|pqNpkx%IrJ(`wj`QRSb=Y2`rI{HS6_tFl;+96f5Fc;- z{5Ay~xmq?K7m@&%qj&^-gyBmUbaaFOF!HMhvsYeGQ7-@>HQ4CgZ>!)Mo114e>h(HS z?u;uib^q!N?9&%Elm4$70}MWMm?CmP8H*;PVKG*9YzHm!*D=CCX2I! zgGaY(OpS z9%lnHmMgk5zurDUK~=Eq8%yEgiPy-!>d>wRTdHYP`)r#5XPHJ93`3ixFl7!+U;B=H z*rH%OV0qp9;MU5+gHH|o+H19J;b3&w+0_NSqbtQ>xcssCJNU+W*R4u9!x3gCa2=BsO<{6Q zZvrgK=s#t{4>)#=T{C=4>#5ewzlMQ}EblN0ZqYwzoIVIrW> z*1XYJqck8`tc*NtuL0Xn5L5#RELm|8=BHdL%7AlVXb6V;^1ENpX{L7FAodz8s=gk8 z#|;!!!xPijuUT(i0q!%LwpUg*^2Y|q&@yszbYScn7Um+%RuJ zPNQuXVHh(&QGpyQms(I#qN1FYN6R!mIb;eU6yBJi*%!%r!~+khG==6iO3ts-fb;bd zUwA4Dg+1QN{qpQ^spQBqRrBqp6mNCuY6XpEWw9HT91~+>6WdY=6N#jU4m%i+jHx2G?|QnUr$I6n9i5f`_2;Bnv~8Yxp;Qo=s| zJ{^npI|-gjUKMY7?g89y;9@B;8FEHKBoChe=ndBGMaV6*zB5Sa>nFur#5%PNd*11S z?G*MT$SxFUpXw3{Snz2M4GH|h67iKE;GBQm$IS}8m-1U2VHUg@rt1H(TMNW=!J+in z-C8d%qJA;+KX&V!q44hcq+XLOa@?d3&yo&2CYBAItU#&6^yn(LLRFKvj*}5xbmyUm zXmn@Oq>KhZc{UX55Ei@V$y()(ZzRj(UWFhdis)E z)}BB1^wOFF#G+hAh(!gJT2xh7Y|`7DO8S?$^}gKvl$FR(2imR`-kfIBl9A_i z@V@9LIG1>z4()Dqcre9omd45k?vfTne)}gYaoH_FRmM0U^+zDJe&XK7_9AvCyVe8` z7v<=w$E3)%JXFMcAZ*pCM;)(&O;goaM1zmY2UG0S;v@WfIw@f;K$?t=Pek1vY3ZX2 z`(NB`Uo-mmE$pYIlf^|oJP(sqDG=wHlEl!igJ86jG5JyzB)Ojw6DHsB~WA|)xt+VpsD2RuE%&vbkPx9&s<{kXzWR&Ps?~H7NKM z5aQ@KF(;m+Q4k^3JpFMnEF~+J-h!GeprfOxnA=rk?e-u`gVd~|{yXH=ZWvkbaVS^r zjQ){0@gv%e7j>^LALklbIUzdUCGz_F$-`xu;PQIaJm1N#Bl_sHHP~Z?OZ%u#LOG+D zcD0iG(e7I!PY+?27ox`_Lp6JR$kopD^wZAyQ)*7G{a&Ifq{wC08R=k(mG!Dq9dsBs zc7g~I!irM@#=*&EJaXr}uDX`bwU)%u@;2TiLWn!|L~r{pSjY4vU!WQFNH7u*XmB zy??SZwUN#XU-GWXi))Nx7kU6nM#?5$G@?C!u0 zI3L7Ei5xfAX7#y4S+$YYeNw;3HSej_P42blShMqf%$L|1vyZMhp!MsHA@4AN^OxR{NI?0A=N!e&w)Gld)&IuQL!zaK|Mb~c!~HS_N!P73U| zwGWqS#y6X$Pdtyp;uem3o`X9r=ZVz%@vaav92rLBTv&2?m~*-%cD$Z*x*3ld2tPdy zAqhbt_iCH1F_Bcr`LT7k9dz|UoY=v}Q1wpF`te}S$%OkFdT9J$kPm}+9ggM52vMH9 z!x4irzLf4_mb|DitXm=w8`KUiK%Be+x7;>P_lrw-ATK`{twAme1!E=`qzT1#S}#i_ z+X;q1>jUxO>TP;!{(g3!ziCv+*>UI7sr+{`GVcZVHA)5Pdn@Xki zp()$k+H(hqe1b{b__*l+fMyukFetC?Wp@}ZdL zta-4uuL7 z5g*va4%g#Pe~X=-7$dofl#MtYYD;`Nh7NCjDMZ#D4)%*=r zso82~az0OrSXp0&_=r4N^joi94xid=@Z87D=2RX8`sIk-_puRNZB<&z2=+Yjr*~U5 z4?3x__Sz>xV{XHen|2o*tv+gK4M~mj91>d7GIwt@xPjoO*i&$s3&6~4GcAvesdo_s zY2e8@IA)I|V3U-JJ20&n1b&y?aGTwVbv{0>*FN474F1m0CWQ71M=c(&ka|@@hb5~bSyD+!Nisv39177QrO0x~`7n|* zQBKKev?2?gkeLv|V$3j(F^m|kLSqmnh8QX)$4SmP?`QN~dtdvyzVEvB;oJM4z5P{l z{LTBmzvq44`?;U{Uig^4*0u>dOQfvLNi<4UXf7mQlP3)LYtD|Y&C+SVXuTZWbv&U# zSj#mzhu?`4EiSmUGQYA)(3pReXvdrgYGAK%2$;hWhR|wG^ifmMDIYC2@11If(X^&5 zg-O2g0BhC8n_rhX=}Wo-zOWrgxLrK=>yipQ+ROFFh80T6U5#G+q_gw5> zb57saIPnu08L#2@S-Yj@zq<9rn|Kq_D^@Q1hz>`sO)?S}(vb|pyJy(%u^79d>yL4l zRM5QFpatlNub;$zJ&ms*u&BY`)UVQ5qC z`&#<>x%aBZm61tF3lfF!rAL5whF>b{e3)! z_XIBu%Z)#@`PR`I2*KrJ>CP`R*M#BN@?VXXgnB|GC2O@_U&w z$|<-OM-R5p9zE5BvhElDeM3CY2!>Gj9RU*&9U~D6Gfg*V_@@Wgv&MoBCHbgvr+%>* z`vNZ+SBH-li#aqNkisT4%f0bni|(oG)#~-?7!TO@>2Leh79nr);F|g~A00kR`6o^F1lre~6dzFq-4Sw2Vt=DqPDQG7zt>9%s zwK*^M7>L-K|6JSU6Jwk-m6VhYbD|uJ*zSqTl$qKxvfF%9CGWL)FivvXhVUe!z^Um6 zjqyJLBd#Y#|1BW$|33kS{+|Nr{vR936D;7VhUa;ys+*7T&d1BT9=_U_Tl5fo;p{>) z>^0tsKIV6ak2k#kt0z1jg3S91*vE_J5qY|RnX6=6+HB8zunS`QqX)Za(}ACs&mN2G zjXY*y6wyGQkudaDa~}DsJW(N~kKkQ%xZLxxMHh)i^(7kxe4f8aftc;?$%wGe=O(*d zVpR?(^sksLwKUe={~<-z-0r`!x+5p;c7O_@KilW@WK(I~lIGJ7mQG%>wTN&fovb!T zbP|IT(AlpHfKEiFj9Fd>H-OdAVH>7rxs;kLKQ!IbQQ>h`<9JREBRy26^zLXaTyd=| z2&~J9lvtoZmP|S2I$_y#6T*Jm2sN$Q`C4%oKt9j_1pO~oDGwfsriT`|=FT8F)1SMC zVzRQpf-qI#z*G5FM7U{$mL#Qz-QMm%nRq(kw*k>*6_eS;or8XPO3(hSk)0p(S+f&i z$Gsn?y+d+*K3DZKCZ4o7@Giio&Go%Qu}yImL@Pccb8{Cch_ww{?c2T`N1^DjaQkPb z0P_py#MA!-43LK}RNMnuZUFLpstd-b1xFLqvQ}>=N+nF8$HQX*`bVIEEe^E<9!gQG z^v1cA(>cs}sh!ymaJvJ+W+R;E>`&XW_ReoZ)EdWRjz75{#rOwx7SE^}yXnaua+z0T z#!kQ#x;Q+s(x#>oi15xM$rh;cQSO;9@HaHXLxs`+M}PWM25JtK^psr>&L5SQ##PrR zBSm8Y0RtWT?nqlrTNZ%j>KnsThre?_OtP#W^?R}Z+9S^Y^dZ}Qq{4u`oG}{GcqeT7 zYZ~7hn=x1@0cik@L5EM`n0Ljed(aAP^@866Y7F-F&xyo~8r#?5U-T;y=|fKsGJR{* zYk&(m&FRqA=Za2R4xC1|WPPKLlrVt*iBaoimv7KM&k%OjNLNW|mS$o{9XeSo%8bYq zvjhi48~_-bH-B8O=G24hf>~(9^`e=-ymX<{7+j8E{McUeW-4221BSGlM@W9XUlNeNWfP*B+U;C!{6 zcX04h$;OiHT`UZ?`t_&BoHs6Mx1qh30CQK8#o)MmdEu1k@GAgvyQslyQYx(ky>c+@ zI21>vQbF2(UHrVI@z|hA7!B*-pv7MdSCBwgR8VCylWI#f76(x_fx|s;24<41*B6l(0fdGl-bOpVl9flQ$FSppy~urD?C^xf6Sy zJ$+gMesP@F|5mTsnHDldhvuB)c~aKo&cwtnkeh>MIn`vq25@^)M8A z+3oa%qiU+E5S9Q7;=2p-^3;N*l~u5zH8nLMzK4<$JPSBA0K@Gk7C4YW6VczffGmSZHM>+9=5Y#Izo!A^q7JY3k-sB$Du7y8jwj~u@p z-R+4>e+R|4vj-0zf7sz}8#I_b7|BzfxMy$_%Z7K2kD~w|i@tX+R{M1w1dP@jldy79 z?!RJ{rTEz{GuP;x>BYL(?1jjZk-9#HH#TME?bXmt$)#A z)wDq|;iCGA{wEV`%1$lC@0V$}9%a?^T|vb}(>+2X7+(~i8=}YwBvin9rgB+}Ml_U> zx@ISjuY!K4Lvh1nW7A73P+x+8<0CZ+&CSP=5u&jiXnn-k*w_S}#WSE949@U_ui-ZE zQ{_$uwvWuf%L>PWU~M!E>=hG>Id`Ze)(uo>#?u%a$8%sy0mWUIa2RP2K^cLy1r65F zMlX-&8K6V)3RW@n*``9x=Gn8C`f4uD&d_<(3T}blKE4Xk(1hNb&PM3)hR6byl$sh1 z9V3Hu4LNV=(@IK8U=j;@qAV8PfN=NKbI-M1g-wgiU@a3L&l=mM1h}{eKoRVA{5F(j%cXMz@h_jd&{6MSWuK6TMXtn<}F10 zj~}1+aB&!<`h2f1@rwHyiD+G^_KuE@-d=SjrCpC2SV1Pw3|yoW;#jg!9A~;M-0biV zb$0%O-d}_dJ;l1MT3HAM8w(4I{-qrst^Ae)$Y{Rruu5oia}nbr)osPEVD^7CdV>k; z@@J*awVWBnh9K7-ti9bP`86nZiVwsbRDcB4OJQUrcaSx&6UOrH5N#!(%9d*!_+j7qX0K{ppvoaSsRI zD{`SC<6L@Lh#%M%mI2$MUYq63g3w`1Ch&xEm?Kc0@u`&Eaz!M&tjs?kz|8DE)L}Q> z3Genyd-}9hkqa-CPB;edj;({p(|CeaCaw|E>3~K#dCaJBVY@?AZu*jqi_j zry=c<{AERBOPpmSf~rCPq7Rl*W#ws8Q}Fod`S3w@vdg{fz^NN&jYMeuYv3GzG=Y98 z>4T>-qm2YnOdb^+c)P>~MI;4*w^C(qU!VK{*oJ68r=CXottz$w0fdFVLg?6Om7BL@q-u4eV-ezbgD5*QzKd#D6ZU z4X!8s9c477oz3P>FMxs#)ZG=uW&_pXW}y}km?JXX$Hz3+e9Ny7fIJO8NCfff?soQyR>{gtzcXLU?kz3WeXGh#ukMb%_oELOwLFt+bpIw%-B3xm zhIg^&B|Z$-RB?-${cEu21rKys3y2tJPE4 zqu2uY1Yp-7MVVO$NxYNe@9&>sm}`97E)k+3auhb~Zsmni<1*lQ;2HE&RChJt4nPEk zZu`Cbn;`|sFD#(8JC}kgC{%O@r*H2i0W1OTE!oD2@+Hy<^r30`)^9-tO+jx*en*UzZVyKY4`_Kl1r=<~TmA&U-TyIHeQ7mTD^FXj5o~()HF3|(BBG@|G=GtEHgiN!O3&jZ%Uwf2qqF)6-+X-$B&LF z&Cilb4*OvW7l*Gc?An*Btv$w#S^hr1?GcvN+Ga9MFjQ- zV-?xv%0z(f~}rjiJps2{F+Ch zUo83kQE)Oa6ZDt~R1c^Mg}SD5DL@KQzk+B;_Le7%ytz`_vJGQccVZ$JCW+GP18!GI z!1n;zCs)@~(DF+{LN@AX?y_FQ<{UJ=ZD;vsB2FGN*_huzGCyyiF+RTp2Ak7^3BR2` zv@^*7pF%;>VGqy<2u!fKgFItp)m1V1Qg)}NXRtOQ@1=bE@gnWn zS)z~XaK>xkv23i?uh=o$yzPzrcBr2vKlEU}&leHh<{v#i0;5?T-#>xRX4lo}SdtUL zhbr4XQCG^bt*p3MAx`&72{@)$D7^uG4eae7xO0t;23{03u?O;Jp1=NNXxib3hK#gv zQD3^$EZSSU)7=FuXB5dWEnv~^?(XjBD1~Q0x}1oFgaWhzoC1)iUsxE4fVSV6GjbSn z*~H}3)I{b5@^qt3T>grgdnN2GP%&247Cms>S1oh)@m1rwD|ipctP88YQX8c#7>8C= zP3`hM)A}b0n^bvXFqab&KOw;a53~mlJTj+#ZWx%nBo~y`8D!GyMIn)ZD*y`?AW5*- zfyyW}qZUkerWwc=mX>e-DCW03+nIfgLLEk0qCVFj{&H*eR>QPQ%$6OwVYFjt6H%@E z4{?GA6xTlQyZ%nYJcPNwy#73MV)a*_j>9O_6W`M-M^r2y(rP^0-qfVo+O0YXqY$;C zCZ?vMcOXyA4m?6whE(B_bOMy#!7<3E-FX4C3}G1*f1&0Kf=HmrKvs9YcI_HUbX%mB z&%s=q(u#^x%E~y9h|6e&lz1@Lw)^NDl|CG<&5ziFFU77DB7kSev6TZ z?luen?dX;L31ICQ6T#$MV3#c3SVYG_;YeOL9=w!9agS$v%HW6!8Oq6%0N{gX)H)Hv z%ECf7+ZgUbXp62`{>o5PQDH1lkP=TD39h`c7tx8UUgYEsK*9|v6p!&jGv}gd`-xXZ zlc_X?@CWg~&VT22z6)t&vhp;(*;8X?kP(633ftp!7v5f-PW3LY9+YM*hb+97xVTMH zV{vjOJGwp3j#xd0Jtem9^4jXGjr!{6Tdq`~`|E{;eLp-BVh87|Spvo*_1T9q${P7O zw7+)cJJ~91x^q@aI4=4{CL9S2Wz<(@m?b9(`5w2c`*_=!+SuDqLX#)sN-tVdGyNS;9P3crAKUXhQi05r!*sK3v-P;0boQX( z-dQpJ%~K1@pu=`z&y&dt3Wb7>?I@Wy(s-$_*#k~zAXosdQj6t1Ue-a>MAKWhsWFWq z_Iaej!ugWhH z)YSCFi`Pjf_`39vgW06a;x-XM+G96~#gH!Z#Z+G8UOEk8{%Rc|&XA3>KCw+@Mzi>>#&5mMnS1}SFt&~-0pD+n2XO~WKW3Yqg?>xAq&{t^o-V=rsME2&M*sGBj=Z#hgGb(*?cGn1Ne+q|6graCk!@&ZJDYO93A=-mo%ukhP?|QEga(K%T z4Ya{^dLA4JTsmG+JdQkONF_k(2RRiSkvcoeTlSkbcUe2E5Ul+7!I*ZMilA z2MIU;>e6ak%IKuy%F#fN?Yac=kQB3IKn)pLfl)|Rm$_o;V4(iY6FYwOudrP}tmf%ZJT4tg8 zx3yDkn?BvJ;kx%JKik-&ZS=&nN(+JLjJABvL)O~ zxlamkK$NH+#9u`*-Z6{g<7kfv-vEV+GAqDDtVSrARXri!Z;VtjIU!+HVz;ihD&G8F zM63#K7;N0nG(^F$-Fjz!C7@9#&KIPLj6Fl!qB7J?1(En+j)~gJq9aB@N&R6T)XIlj=N4K{Y zrRm48RU&kaB6{Q&gA%V58dlAd+6^w!{sz5nL~laIloU3+uQrF%7p3L7*yg{por2=+IycO?x!|;n)xnhtrNfy_Ensz zpGqy+t@4}g$L(V>EtK?>Q}X<^d2jNhcq+Sepss{pJ(%p==6?qIa^cbYQVGpKqH)6s zaCH#%2H7$(nKF?7dokl@ydz!mXZcRA6c%Mv+gK`R6 zK3Fw1rrxX?2X1Z_2O}8)1^y1VJw=!OY+3%*HuAf-MmowKj`$pKdR*Cq z+{0t$*n`Sy<)DKVKb!l)8ikW>gatP)oVeWvC0be;qMV{aBsDvixr>%snqHpqgEs(Z zYg}oA!uc#fdn*!uNjb&2)E&S&>Wm(u{%ZzJxBg>D&Ew-DvDs*e8@w+VG8VaNb~(8& z-wl`DMv39Qtl*GlQ{o;J^b&{hW~c&3zhS-L7G!i}-V7fGWxq%>X*-QUc&66i1?p#S z>BX8GXG=B&AT@e;k9{qo_bz1(2=VBh2*ml?@rw0-X#WGC`+szm|G4@7C%)p-+NOE& Wc$`4lAeHy?F6tPZE6}q4{a*k&kMIfr diff --git a/tests/proxy_admin_ui_tests/e2e_ui_tests/require_auth_for_dashboard.spec.ts b/tests/proxy_admin_ui_tests/e2e_ui_tests/require_auth_for_dashboard.spec.ts deleted file mode 100644 index 4e4bd2fcd93..00000000000 --- a/tests/proxy_admin_ui_tests/e2e_ui_tests/require_auth_for_dashboard.spec.ts +++ /dev/null @@ -1,37 +0,0 @@ -// tests/auth.spec.ts -import { test, expect } from "@playwright/test"; - -test.describe("Authentication Checks", () => { - test("should redirect unauthenticated user from a protected page", async ({ - page, - }) => { - test.setTimeout(30000); - - page.on("console", (msg) => console.log("PAGE LOG:", msg.text())); - - const protectedPageUrl = "http://localhost:4000/ui?page=llm-playground"; - const expectedRedirectUrl = "http://localhost:4000/ui/login/"; - - console.log( - `Attempting to navigate to protected page: ${protectedPageUrl}` - ); - - await page.goto(protectedPageUrl); - - console.log(`Navigation initiated. Current URL: ${page.url()}`); - - try { - await page.waitForURL(expectedRedirectUrl, { timeout: 10000 }); - console.log(`Waited for URL. Current URL is now: ${page.url()}`); - } catch (error) { - console.error( - `Timeout waiting for URL: ${expectedRedirectUrl}. Current URL: ${page.url()}` - ); - await page.screenshot({ path: "redirect-fail-screenshot.png" }); - throw error; - } - - await expect(page).toHaveURL(expectedRedirectUrl); - console.log(`Assertion passed: Page URL is ${expectedRedirectUrl}`); - }); -}); diff --git a/tests/proxy_admin_ui_tests/e2e_ui_tests/search_users.spec.ts b/tests/proxy_admin_ui_tests/e2e_ui_tests/search_users.spec.ts deleted file mode 100644 index d72c44ab8cc..00000000000 --- a/tests/proxy_admin_ui_tests/e2e_ui_tests/search_users.spec.ts +++ /dev/null @@ -1,222 +0,0 @@ -/* -Search Users in Admin UI -E2E Test for user search functionality - -Tests: -1. Navigate to Internal Users tab -2. Verify search input exists -3. Test search functionality -4. Verify results update -5. Test filtering by email, user ID, and SSO user ID -*/ - -import { test, expect } from "@playwright/test"; - -test("user search test", async ({ page }) => { - // Set a longer timeout for the entire test - test.setTimeout(60000); - - // Enable console logging - page.on("console", (msg) => console.log("PAGE LOG:", msg.text())); - - // Login first - await page.goto("http://localhost:4000/ui"); - await page.waitForLoadState("networkidle"); - console.log("Navigated to login page"); - - page.screenshot({ path: "test-results/search_users_before_login.png" }); - - // Wait for login form to be visible - await page.waitForSelector('input[placeholder="Enter your username"]', { - timeout: 10000, - }); - console.log("Login form is visible"); - - await page.fill('input[placeholder="Enter your username"]', "admin"); - await page.fill('input[placeholder="Enter your password"]', "gm"); - console.log("Filled login credentials"); - - const loginButton = page.getByRole("button", { name: "Login" }); - await expect(loginButton).toBeEnabled(); - await loginButton.click(); - console.log("Clicked login button"); - - // Wait for navigation to complete and dashboard to load - await page.waitForLoadState("networkidle"); - console.log("Page loaded after login"); - - // Take a screenshot for debugging - await page.screenshot({ path: "after-login.png" }); - console.log("Took screenshot after login"); - - // Try to find the Internal User tab with more debugging - console.log("Looking for Internal User tab..."); - const internalUserTab = page.locator("span.ant-menu-title-content", { - hasText: "Internal User", - }); - - // Wait for the tab to be visible - await internalUserTab.waitFor({ state: "visible", timeout: 10000 }); - console.log("Internal User tab is visible"); - - // Take another screenshot before clicking - await page.screenshot({ path: "before-tab-click.png" }); - console.log("Took screenshot before tab click"); - - await internalUserTab.click(); - console.log("Clicked Internal User tab"); - - // Wait for the page to load and table to be visible - await page.waitForSelector("tbody tr", { timeout: 30000 }); - await page.waitForTimeout(2000); // Additional wait for table to stabilize - console.log("Table is visible"); - - // Take a final screenshot - await page.screenshot({ path: "after-tab-click.png" }); - console.log("Took screenshot after tab click"); - - // Verify search input exists - const searchInput = page.locator('input[placeholder="Search by email..."]'); - await expect(searchInput).toBeVisible(); - console.log("Search input is visible"); - - // Test search functionality - const initialUserCount = await page.locator("tbody tr").count(); - console.log(`Initial user count: ${initialUserCount}`); - - // Perform a search - const testEmail = "test@"; - await searchInput.fill(testEmail); - console.log("Filled search input"); - - // Wait for the debounced search to complete - await page.waitForTimeout(500); - console.log("Waited for debounce"); - - // Wait for the results count to update - await page.waitForFunction((initialCount) => { - const currentCount = document.querySelectorAll("tbody tr").length; - return currentCount !== initialCount; - }, initialUserCount); - console.log("Results updated"); - - const filteredUserCount = await page.locator("tbody tr").count(); - console.log(`Filtered user count: ${filteredUserCount}`); - - expect(filteredUserCount).toBeDefined(); - - // Clear the search - await searchInput.clear(); - console.log("Cleared search"); - - await page.waitForTimeout(500); - console.log("Waited for debounce after clear"); - - await page.waitForFunction((initialCount) => { - const currentCount = document.querySelectorAll("tbody tr").length; - return currentCount === initialCount; - }, initialUserCount); - console.log("Results reset"); - - const resetUserCount = await page.locator("tbody tr").count(); - console.log(`Reset user count: ${resetUserCount}`); - - expect(resetUserCount).toBe(initialUserCount); -}); - -test("user filter test", async ({ page }) => { - // Set a longer timeout for the entire test - test.setTimeout(60000); - - // Enable console logging - page.on("console", (msg) => console.log("PAGE LOG:", msg.text())); - - // Login first - await page.goto("http://localhost:4000/ui"); - await page.waitForLoadState("networkidle"); - console.log("Navigated to login page"); - - // Wait for login form to be visible - await page.waitForSelector('input[placeholder="Enter your username"]', { - timeout: 10000, - }); - console.log("Login form is visible"); - - await page.fill('input[placeholder="Enter your username"]', "admin"); - await page.fill('input[placeholder="Enter your password"]', "gm"); - console.log("Filled login credentials"); - - const loginButton = page.getByRole("button", { name: "Login" }); - await expect(loginButton).toBeEnabled(); - await loginButton.click(); - console.log("Clicked login button"); - - // Wait for navigation to complete and dashboard to load - await page.waitForLoadState("networkidle"); - console.log("Page loaded after login"); - - // Navigate to Internal Users tab - const internalUserTab = page.locator("span.ant-menu-title-content", { - hasText: "Internal User", - }); - await internalUserTab.waitFor({ state: "visible", timeout: 10000 }); - await internalUserTab.click(); - console.log("Clicked Internal User tab"); - - // Wait for the page to load and table to be visible - await page.waitForSelector("tbody tr", { timeout: 30000 }); - await page.waitForTimeout(2000); // Additional wait for table to stabilize - console.log("Table is visible"); - - // Get initial user count - const initialUserCount = await page.locator("tbody tr").count(); - console.log(`Initial user count: ${initialUserCount}`); - - // Click the filter button to show additional filters - const filterButton = page.getByRole("button", { - name: "Filters", - exact: true, - }); - await filterButton.click(); - console.log("Clicked filter button"); - await page.waitForTimeout(500); // Wait for filters to appear - - // Test user ID filter - const userIdInput = page.locator('input[placeholder="Filter by User ID"]'); - await expect(userIdInput).toBeVisible(); - console.log("User ID filter is visible"); - - await userIdInput.fill("user"); - console.log("Filled user ID filter"); - await page.waitForTimeout(1000); - const userIdFilteredCount = await page.locator("tbody tr").count(); - console.log(`User ID filtered count: ${userIdFilteredCount}`); - expect(userIdFilteredCount).toBeLessThan(initialUserCount); - - // Clear user ID filter - await userIdInput.clear(); - await page.waitForTimeout(1000); - console.log("Cleared user ID filter"); - - // Test SSO user ID filter - const ssoUserIdInput = page.locator('input[placeholder="Filter by SSO ID"]'); - await expect(ssoUserIdInput).toBeVisible(); - console.log("SSO user ID filter is visible"); - - await ssoUserIdInput.fill("sso"); - console.log("Filled SSO user ID filter"); - await page.waitForTimeout(1000); - const ssoUserIdFilteredCount = await page.locator("tbody tr").count(); - console.log(`SSO user ID filtered count: ${ssoUserIdFilteredCount}`); - expect(ssoUserIdFilteredCount).toBeLessThan(initialUserCount); - - // Clear SSO user ID filter - await ssoUserIdInput.clear(); - await page.waitForTimeout(5000); - console.log("Cleared SSO user ID filter"); - - // Verify count returns to initial after clearing all filters - const finalUserCount = await page.locator("tbody tr").count(); - console.log(`Final user count: ${finalUserCount}`); - expect(finalUserCount).toBe(initialUserCount); -}); diff --git a/tests/proxy_admin_ui_tests/e2e_ui_tests/team_admin.spec.ts b/tests/proxy_admin_ui_tests/e2e_ui_tests/team_admin.spec.ts deleted file mode 100644 index a753c724b37..00000000000 --- a/tests/proxy_admin_ui_tests/e2e_ui_tests/team_admin.spec.ts +++ /dev/null @@ -1,250 +0,0 @@ -import { test, expect } from "@playwright/test"; -import { loginToUI } from "../utils/login"; - -// test.describe("Invite User, Set Password, and Login", () => { -// let testEmail: string; -// const testPassword = "Password123!"; // Define a password -// const teamName1 = `team-invite-test-1-${Date.now()}`; -// const teamName2 = `team-invite-test-2-${Date.now()}`; -// const keyName1 = `key-${teamName1}`; -// const keyName2 = `key-${teamName2}`; - -// test.beforeEach(async ({ page }) => { -// await loginToUI(page); // Login as admin first -// await page.goto("http://localhost:4000/ui?page=teams"); - -// // --- Create Team 1 --- -// await page.getByRole("button", { name: "+ Create New Team" }).click(); -// await page -// .getByLabel("Team Name") -// .waitFor({ state: "visible", timeout: 5000 }); // Wait for label -// await page.getByLabel("Team Name").click(); -// await page.getByLabel("Team Name").fill(teamName1); -// await page.getByRole("button", { name: "Create Team" }).click(); -// // Wait for the modal to close or for a success message if applicable -// await expect( -// page.locator(".ant-modal-wrap").filter({ hasText: "Create New Team" }) -// ).not.toBeVisible({ timeout: 10000 }); -// console.log(`Created Team 1: ${teamName1}`); - -// // --- Create Team 2 --- -// await page.getByRole("button", { name: "+ Create New Team" }).click(); -// await page -// .getByLabel("Team Name") -// .waitFor({ state: "visible", timeout: 5000 }); // Wait for label -// await page.getByLabel("Team Name").click(); -// await page.getByLabel("Team Name").fill(teamName2); -// await page.getByRole("button", { name: "Create Team" }).click(); -// // Wait for the modal to close or for a success message if applicable -// await expect( -// page.locator(".ant-modal-wrap").filter({ hasText: "Create New Team" }) -// ).not.toBeVisible({ timeout: 10000 }); -// console.log(`Created Team 2: ${teamName2}`); - -// // // Verify both teams are listed -// // await page.goto("http://localhost:4000/ui?page=teams"); // Refresh or ensure on teams page -// // await page.waitForTimeout(3000); -// await expect(page.getByText(teamName1)).toBeVisible({ timeout: 10000 }); -// await expect(page.getByText(teamName2)).toBeVisible({ timeout: 10000 }); - -// // --- Navigate to Keys Page --- -// await page.goto("http://localhost:4000/ui?page=api-keys"); -// await page.waitForTimeout(3000); -// await expect( -// page.getByRole("button", { name: "+ Create New Key" }) -// ).toBeVisible(); // Wait for page load - -// // --- Create Key for Team 1 --- -// await page.getByRole("button", { name: "+ Create New Key" }).click(); -// const createKeyModal1 = page -// .locator(".ant-modal-wrap") -// .filter({ hasText: "Key Ownership" }); -// await expect(createKeyModal1).toBeVisible(); - -// // Select Team 1 -// await createKeyModal1 -// .locator(".ant-select-selector >> input") -// .first() -// .click(); // Click to open team dropdown -// await createKeyModal1 -// .locator(".ant-select-selector >> input") -// .first() -// .fill(teamName1); - -// await page -// .locator(".ant-select-item-option") -// .filter({ hasText: teamName1 }) -// .first() -// .click(); // Click specific team name - -// // Enter Key Name 1 -// await page.fill('input[id="key_alias"]', keyName1); - -// // Click on models dropdown -// await page.locator("input#models").click(); -// await page.waitForSelector( -// '.ant-select-item-option[title="All Team Models"]' -// ); -// await page -// .locator('.ant-select-item-option[title="All Team Models"]') -// .click(); - -// // Click Create Key -// await createKeyModal1.getByRole("button", { name: "Create Key" }).click(); - -// // Close the Key Generated modal (which appears after successful creation) -// const keyGeneratedModal1 = page -// .locator(".ant-modal-wrap") -// .filter({ hasText: "Save your Key" }); -// await expect(keyGeneratedModal1).toBeVisible({ timeout: 10000 }); -// await keyGeneratedModal1.locator('button[aria-label="Close"]').click(); -// await expect(keyGeneratedModal1).not.toBeVisible(); // Wait for close -// console.log(`Created Key 1: ${keyName1} for Team: ${teamName1}`); - -// // --- Create Key for Team 2 --- -// await page.getByRole("button", { name: "+ Create New Key" }).click(); -// const createKeyModal2 = page -// .locator(".ant-modal-wrap") -// .filter({ hasText: "Key Ownership" }); -// await expect(createKeyModal2).toBeVisible(); - -// // Select Team 2 -// await createKeyModal2 -// .locator(".ant-select-selector >> input") -// .first() -// .click(); // Click to open team dropdown -// await page -// .locator(".ant-select-item-option") -// .filter({ hasText: teamName2 }) -// .click(); // Click specific team name - -// // Enter Key Name 2 -// await page.fill('input[id="key_alias"]', keyName2); - -// // Click on models dropdown -// await page.locator("input#models").click(); -// await page.waitForSelector( -// '.ant-select-item-option[title="All Team Models"]' -// ); -// await page -// .locator('.ant-select-item-option[title="All Team Models"]') -// .click(); - -// // Click Create Key -// await createKeyModal2.getByRole("button", { name: "Create Key" }).click(); - -// // Close the Key Generated modal -// const keyGeneratedModal2 = page -// .locator(".ant-modal-wrap") -// .filter({ hasText: "Save your Key" }); -// await expect(keyGeneratedModal2).toBeVisible({ timeout: 10000 }); -// await keyGeneratedModal2.locator('button[aria-label="Close"]').click(); -// await expect(keyGeneratedModal2).not.toBeVisible(); // Wait for close -// console.log(`Created Key 2: ${keyName2} for Team: ${teamName2}`); -// }); - -// test("Invite user, set password via link, and login", async ({ page }) => { -// // Navigate to Users page -// await page.goto("http://localhost:4000/ui?page=users"); - -// // Go to Internal User tab -// const internalUserTab = page.locator("span.ant-menu-title-content", { -// hasText: "Internal User", -// }); -// await internalUserTab.waitFor({ state: "visible", timeout: 10000 }); -// await internalUserTab.click(); - -// // --- Invite User Flow --- -// await page.getByRole("button", { name: "+ Invite User" }).click(); - -// // Wait for the invite user modal to be visible -// const inviteModal = page -// .locator(".ant-modal-wrap") -// .filter({ hasText: "Invite User" }); -// await expect(inviteModal).toBeVisible(); - -// testEmail = `test-${Date.now()}@litellm.ai`; // Use a unique email -// // Assuming the email input is the first one with 'base-input' test id inside the modal -// await inviteModal.getByTestId("base-input").first().fill(testEmail); - -// // Select Global Admin Role (or another appropriate role) -// const globalRoleLabel = inviteModal.getByLabel("Global Proxy Role"); -// await globalRoleLabel.click(); -// // Wait for the dropdown option to be visible before clicking -// const adminRoleOption = page.getByTitle("Admin (All Permissions)", { -// exact: true, -// }); -// await adminRoleOption.waitFor({ state: "visible", timeout: 5000 }); -// await adminRoleOption.click(); - -// // Select Team - Add explicit wait before clicking -// const teamIdLabel = inviteModal.getByLabel("Team ID"); -// // Wait for the label associated with the Team ID select to be visible -// await teamIdLabel.waitFor({ state: "visible", timeout: 10000 }); // Increased timeout for safety -// await teamIdLabel.click(); - -// // Wait for the team name option to be visible in the dropdown -// const teamNameOption = page.getByText(teamName1, { exact: true }); -// await teamNameOption.waitFor({ state: "visible", timeout: 5000 }); -// await teamNameOption.click(); - -// // Create User -// await inviteModal.getByRole("button", { name: "Create User" }).click(); - -// // --- Capture Invitation Link --- -// const invitationModal = page -// .locator(".ant-modal-wrap") -// .filter({ hasText: "Invitation Link" }); -// await expect(invitationModal).toBeVisible({ timeout: 15000 }); // Wait longer for modal - -// // Locate the text element containing the URL more reliably -// const invitationUrl = await page -// .locator("div.flex.justify-between.pt-5.pb-2") // find the correct div -// .filter({ hasText: "Invitation Link" }) // find the div that has text "Invitation Link" -// .locator("p") // find all

inside that div -// .nth(1) // pick the second

(index 1) -// .innerText(); - -// // Close Invitation Link Modal -// await page -// .locator(".ant-modal-wrap") -// .filter({ hasText: "Invitation Link" }) -// .locator('button[aria-label="Close"]') -// .click(); - -// // Close Invite User Modal -// await page -// .locator(".ant-modal-wrap") -// .filter({ hasText: "Invite User" }) -// .locator('button[aria-label="Close"]') -// .click(); - -// // Open invite link as new page (simulate invited user) -// const context = await page.context()?.browser()?.newContext(); -// const invitedUserPage = await context?.newPage(); -// if (!invitedUserPage) { -// throw new Error("invitedUserPage is undefined"); -// } -// await invitedUserPage?.goto(invitationUrl || ""); - -// //Insert new password -// await invitedUserPage?.fill("input#password", testPassword); - -// //Click on submit -// await invitedUserPage?.getByRole("button", { name: "Sign Up" }).click(); - -// // // --- Verify Keys Created --- -// // await invitedUserPage?.waitForSelector("table"); - -// // // Verify keyName1 (associated with user's team) IS visible in the table -// // const keyTable = invitedUserPage.locator('table'); // Locate the table element -// // await expect(keyTable).toBeVisible({ timeout: 10000 }); // Ensure table exists -// // // Use getByText within the table scope to find the key name -// // await expect(keyTable.getByText(keyName1, { exact: true })).toBeVisible({ timeout: 10000 }); -// // console.log(`Verified key ${keyName1} is visible for user ${testEmail}`); - -// // // Verify keyName2 (associated with the *other* team) IS NOT visible -// // await expect(keyTable.getByText(keyName2, { exact: true })).not.toBeVisible(); -// // console.log(`Verified key ${keyName2} is NOT visible for user ${testEmail}`); -// }); -// }); diff --git a/tests/proxy_admin_ui_tests/e2e_ui_tests/view_internal_user.spec.ts b/tests/proxy_admin_ui_tests/e2e_ui_tests/view_internal_user.spec.ts deleted file mode 100644 index 832832d8ae8..00000000000 --- a/tests/proxy_admin_ui_tests/e2e_ui_tests/view_internal_user.spec.ts +++ /dev/null @@ -1,72 +0,0 @@ -/* -Test view internal user page -*/ - -import { test, expect } from "@playwright/test"; - -test("view internal user page", async ({ page }) => { - // Go to the specified URL - await page.goto("http://localhost:4000/ui"); - await page.waitForLoadState("networkidle"); - - page.screenshot({ path: "test-results/view_internal_user_before_login.png" }); - - // Enter "admin" in the username input field - await page.fill('input[placeholder="Enter your username"]', "admin"); - - // Enter "gm" in the password input field - await page.fill('input[placeholder="Enter your password"]', "gm"); - - // Click the login button - const loginButton = page.getByRole("button", { name: "Login" }); - await expect(loginButton).toBeEnabled(); - await loginButton.click(); - - // Wait for the Internal User tab and click it - const tabElement = page.locator("span.ant-menu-title-content", { - hasText: "Internal User", - }); - await tabElement.click(); - - // Wait for the table to load - await page.waitForSelector("tbody tr", { timeout: 10000 }); - await page.waitForTimeout(2000); // Additional wait for table to stabilize - await page.waitForLoadState("networkidle"); - - // Test all expected fields are present - // Verify that the API Keys column is rendered for all users - // The UI renders badges in each row - we just verify the column structure exists - const rowCount = await page.locator("tbody tr").count(); - expect(rowCount).toBeGreaterThan(0); - - const userIdHeader = await page.locator("th", { hasText: "User ID" }); - await expect(userIdHeader).toBeVisible({ timeout: 10000 }); - - // test pagination - // Wait for pagination controls to be visible - await page.waitForSelector(".flex.justify-between.items-center", { - timeout: 5000, - }); - - // Check if we're on the first page by looking at the results count - const resultsText = - (await page.locator(".text-sm.text-gray-700").textContent()) || ""; - const isFirstPage = resultsText.includes("1 -"); - - if (isFirstPage) { - // On first page, previous button should be disabled - const prevButton = page.locator("button", { hasText: "Previous" }); - await expect(prevButton).toBeDisabled(); - } - - // Next button should be enabled if there are more pages - const nextButton = page.locator("button", { hasText: "Next" }); - const totalResults = - (await page.locator(".text-sm.text-gray-700").textContent()) || ""; - const hasMorePages = - totalResults.includes("of") && !totalResults.includes("1 - 25 of 25"); - - if (hasMorePages) { - await expect(nextButton).toBeEnabled(); - } -}); diff --git a/tests/proxy_admin_ui_tests/e2e_ui_tests/view_user_info.spec.ts b/tests/proxy_admin_ui_tests/e2e_ui_tests/view_user_info.spec.ts deleted file mode 100644 index adda3088f12..00000000000 --- a/tests/proxy_admin_ui_tests/e2e_ui_tests/view_user_info.spec.ts +++ /dev/null @@ -1,124 +0,0 @@ -import { test, expect } from "@playwright/test"; -import { loginToUI } from "../utils/login"; - -test.describe("User Info View", () => { - test("should display user info when clicking on user ID", async ({ - page, - }) => { - await page.goto("http://localhost:4000/ui"); - await page.waitForLoadState("networkidle"); - - page.screenshot({ - path: "test-results/view_user_info_before_login.png", - }); - - // Enter "admin" in the username input field - await page.fill('input[placeholder="Enter your username"]', "admin"); - page.screenshot({ - path: "test-results/view_user_info_after_username_input.png", - }); - - // Enter "gm" in the password input field - await page.fill('input[placeholder="Enter your password"]', "gm"); - page.screenshot({ - path: "test-results/view_user_info_after_password_input.png", - }); - - // Click the login button - const loginButton = page.getByRole("button", { name: "Login" }); - await expect(loginButton).toBeEnabled(); - await loginButton.click(); - page.screenshot({ - path: "test-results/view_user_info_after_login_button_click.png", - }); - - // Wait for navigation to complete and dashboard to load - await page.waitForLoadState("networkidle"); - const tabElement = page.locator("span.ant-menu-title-content", { - hasText: "Internal User", - }); - await tabElement.click(); - page.screenshot({ - path: "test-results/view_user_info_after_internal_user_tab_click.png", - }); - // Wait for loading state to disappear - await page.waitForSelector('text="🚅 Loading users..."', { - state: "hidden", - timeout: 10000, - }); - page.screenshot({ path: "test-results/view_user_info_after_loading.png" }); - // Wait for users table to load - await page.waitForSelector("table"); - page.screenshot({ - path: "test-results/view_user_info_after_table_load.png", - }); - // Get the first user ID cell - const firstUserIdCell = page.locator( - "table tbody tr:first-child td:first-child" - ); - const userId = await firstUserIdCell.textContent(); - console.log("Found user ID:", userId); - - // Click on the user ID - await firstUserIdCell.click(); - await page.waitForLoadState("networkidle"); - - // Check for tabs - await expect(page.locator('button:has-text("Overview")')).toBeVisible({ - timeout: 10000, - }); - await expect(page.locator('button:has-text("Details")')).toBeVisible({ - timeout: 10000, - }); - - // Switch to details tab - await page.locator('button:has-text("Details")').click(); - - // Check details section - await expect(page.locator("text=User ID")).toBeVisible(); - await expect(page.locator("text=Email")).toBeVisible(); - - // Go back to users list - await page.locator('button:has-text("Back to Users")').click(); - - // Verify we're back on the users page - await expect(page.locator("table")).toBeVisible(); - await expect( - page.locator('input[placeholder="Search by email..."]') - ).toBeVisible(); - }); - - // test("should handle user deletion", async ({ page }) => { - // // Wait for users table to load - // await page.waitForSelector("table"); - - // // Get the first user ID cell - // const firstUserIdCell = page.locator( - // "table tbody tr:first-child td:first-child" - // ); - // const userId = await firstUserIdCell.textContent(); - - // // Click on the user ID - // await firstUserIdCell.click(); - - // // Wait for user info view to load - // await page.waitForSelector('h1:has-text("User")'); - - // // Click delete button - // await page.locator('button:has-text("Delete User")').click(); - - // // Confirm deletion in modal - // await page.locator('button:has-text("Delete")').click(); - - // // Verify success message - // await expect(page.locator("text=User deleted successfully")).toBeVisible(); - - // // Verify we're back on the users page - // await expect(page.locator('h1:has-text("Users")')).toBeVisible(); - - // // Verify user is no longer in the table - // if (userId) { - // await expect(page.locator(`text=${userId}`)).not.toBeVisible(); - // } - // }); -}); diff --git a/tests/proxy_admin_ui_tests/package-lock.json b/tests/proxy_admin_ui_tests/package-lock.json deleted file mode 100644 index 8c79edf9ad1..00000000000 --- a/tests/proxy_admin_ui_tests/package-lock.json +++ /dev/null @@ -1,97 +0,0 @@ -{ - "name": "proxy_admin_ui_tests", - "version": "1.0.0", - "lockfileVersion": 3, - "requires": true, - "packages": { - "": { - "name": "proxy_admin_ui_tests", - "version": "1.0.0", - "license": "ISC", - "devDependencies": { - "@playwright/test": "^1.47.2", - "@types/node": "^22.5.5" - } - }, - "node_modules/@playwright/test": { - "version": "1.56.1", - "resolved": "https://registry.npmjs.org/@playwright/test/-/test-1.56.1.tgz", - "integrity": "sha512-vSMYtL/zOcFpvJCW71Q/OEGQb7KYBPAdKh35WNSkaZA75JlAO8ED8UN6GUNTm3drWomcbcqRPFqQbLae8yBTdg==", - "dev": true, - "license": "Apache-2.0", - "dependencies": { - "playwright": "1.56.1" - }, - "bin": { - "playwright": "cli.js" - }, - "engines": { - "node": ">=18" - } - }, - "node_modules/@types/node": { - "version": "22.19.1", - "resolved": "https://registry.npmjs.org/@types/node/-/node-22.19.1.tgz", - "integrity": "sha512-LCCV0HdSZZZb34qifBsyWlUmok6W7ouER+oQIGBScS8EsZsQbrtFTUrDX4hOl+CS6p7cnNC4td+qrSVGSCTUfQ==", - "dev": true, - "license": "MIT", - "dependencies": { - "undici-types": "~6.21.0" - } - }, - "node_modules/fsevents": { - "version": "2.3.2", - "resolved": "https://registry.npmjs.org/fsevents/-/fsevents-2.3.2.tgz", - "integrity": "sha512-xiqMQR4xAeHTuB9uWm+fFRcIOgKBMiOBP+eXiyT7jsgVCq1bkVygt00oASowB7EdtpOHaaPgKt812P9ab+DDKA==", - "dev": true, - "hasInstallScript": true, - "license": "MIT", - "optional": true, - "os": [ - "darwin" - ], - "engines": { - "node": "^8.16.0 || ^10.6.0 || >=11.0.0" - } - }, - "node_modules/playwright": { - "version": "1.56.1", - "resolved": "https://registry.npmjs.org/playwright/-/playwright-1.56.1.tgz", - "integrity": "sha512-aFi5B0WovBHTEvpM3DzXTUaeN6eN0qWnTkKx4NQaH4Wvcmc153PdaY2UBdSYKaGYw+UyWXSVyxDUg5DoPEttjw==", - "dev": true, - "license": "Apache-2.0", - "dependencies": { - "playwright-core": "1.56.1" - }, - "bin": { - "playwright": "cli.js" - }, - "engines": { - "node": ">=18" - }, - "optionalDependencies": { - "fsevents": "2.3.2" - } - }, - "node_modules/playwright-core": { - "version": "1.56.1", - "resolved": "https://registry.npmjs.org/playwright-core/-/playwright-core-1.56.1.tgz", - "integrity": "sha512-hutraynyn31F+Bifme+Ps9Vq59hKuUCz7H1kDOcBs+2oGguKkWTU50bBWrtz34OUWmIwpBTWDxaRPXrIXkgvmQ==", - "dev": true, - "license": "Apache-2.0", - "bin": { - "playwright-core": "cli.js" - }, - "engines": { - "node": ">=18" - } - }, - "node_modules/undici-types": { - "version": "6.21.0", - "resolved": "https://registry.npmjs.org/undici-types/-/undici-types-6.21.0.tgz", - "integrity": "sha512-iwDZqg0QAGrg9Rav5H4n0M64c3mkR59cJ6wQp+7C4nI0gsmExaedaYLNO44eT4AtBBwjbTiGPMlt2Md0T9H9JQ==", - "dev": true, - "license": "MIT" - } - } -} diff --git a/tests/proxy_admin_ui_tests/package.json b/tests/proxy_admin_ui_tests/package.json deleted file mode 100644 index 5933490fb1d..00000000000 --- a/tests/proxy_admin_ui_tests/package.json +++ /dev/null @@ -1,14 +0,0 @@ -{ - "name": "proxy_admin_ui_tests", - "version": "1.0.0", - "description": "", - "main": "index.js", - "scripts": {}, - "keywords": [], - "author": "", - "license": "ISC", - "devDependencies": { - "@playwright/test": "1.56.1", - "@types/node": "22.19.1" - } -} diff --git a/tests/proxy_admin_ui_tests/playwright.config.ts b/tests/proxy_admin_ui_tests/playwright.config.ts deleted file mode 100644 index 8b66c47394a..00000000000 --- a/tests/proxy_admin_ui_tests/playwright.config.ts +++ /dev/null @@ -1,84 +0,0 @@ -import { defineConfig, devices } from '@playwright/test'; - -/** - * Read environment variables from file. - * https://github.com/motdotla/dotenv - */ -// import dotenv from 'dotenv'; -// import path from 'path'; -// dotenv.config({ path: path.resolve(__dirname, '.env') }); - -/** - * See https://playwright.dev/docs/test-configuration. - */ -export default defineConfig({ - testDir: './e2e_ui_tests', - testIgnore: ['**/tests/pass_through_tests/**', '../pass_through_tests/**/*'], - testMatch: '**/*.spec.ts', // Only run files ending in .spec.ts - /* Run tests in files in parallel */ - fullyParallel: true, - /* Fail the build on CI if you accidentally left test.only in the source code. */ - forbidOnly: !!process.env.CI, - /* Retry on CI only */ - retries: process.env.CI ? 2 : 0, - /* Opt out of parallel tests on CI. */ - workers: process.env.CI ? 1 : undefined, - /* Reporter to use. See https://playwright.dev/docs/test-reporters */ - reporter: 'html', - /* Shared settings for all the projects below. See https://playwright.dev/docs/api/class-testoptions. */ - use: { - /* Base URL to use in actions like `await page.goto('/')`. */ - // baseURL: 'http://127.0.0.1:3000', - - /* Collect trace when retrying the failed test. See https://playwright.dev/docs/trace-viewer */ - trace: 'on-first-retry', - }, - - /* Configure projects for major browsers */ - projects: [ - { - name: 'chromium', - use: { ...devices['Desktop Chrome'] }, - }, - - { - name: 'firefox', - use: { ...devices['Desktop Firefox'] }, - }, - - { - name: 'webkit', - use: { ...devices['Desktop Safari'] }, - }, - - /* Test against mobile viewports. */ - // { - // name: 'Mobile Chrome', - // use: { ...devices['Pixel 5'] }, - // }, - // { - // name: 'Mobile Safari', - // use: { ...devices['iPhone 12'] }, - // }, - - /* Test against branded browsers. */ - // { - // name: 'Microsoft Edge', - // use: { ...devices['Desktop Edge'], channel: 'msedge' }, - // }, - // { - // name: 'Google Chrome', - // use: { ...devices['Desktop Chrome'], channel: 'chrome' }, - // }, - ], - timeout: 4*60*1000, - expect: { - timeout: 10 * 1000 - } - /* Run your local dev server before starting the tests */ - // webServer: { - // command: 'npm run start', - // url: 'http://127.0.0.1:3000', - // reuseExistingServer: !process.env.CI, - // }, -}); diff --git a/tests/proxy_admin_ui_tests/utils/login.ts b/tests/proxy_admin_ui_tests/utils/login.ts deleted file mode 100644 index 25858d9f570..00000000000 --- a/tests/proxy_admin_ui_tests/utils/login.ts +++ /dev/null @@ -1,27 +0,0 @@ -import { Page, expect } from "@playwright/test"; - -export async function loginToUI(page: Page) { - // Login first - await page.goto("http://localhost:4000/ui"); - await page.waitForLoadState("networkidle"); - console.log("Navigated to login page"); - - page.screenshot({ path: "test-results/login_utils_before.png" }); - // Wait for login form to be visible - await page.waitForSelector('input[placeholder="Enter your username"]', { - timeout: 10000, - }); - console.log("Login form is visible"); - - await page.fill('input[placeholder="Enter your username"]', "admin"); - await page.fill('input[placeholder="Enter your password"]', "gm"); - console.log("Filled login credentials"); - - const loginButton = page.getByRole("button", { name: "Login" }); - await expect(loginButton).toBeEnabled(); - await loginButton.click(); - console.log("Clicked login button"); - - // Wait for navigation to complete - await page.waitForURL("**/*"); -} From 9600fda2cc94024182ce395093f21854d43a1aba Mon Sep 17 00:00:00 2001 From: milan-berri Date: Fri, 22 May 2026 22:00:42 +0300 Subject: [PATCH 08/13] fix(sagemaker): send native Cohere embed payload to Cohere SageMaker endpoints (#28613) * fix(sagemaker): use Cohere embed payload for Marketplace endpoints SageMaker embedding only special-cased Voyage; every other endpoint received HuggingFace TGI `{"inputs": [...]}`. AWS Marketplace Cohere containers expect the native Cohere embed payload (`texts`, `input_type`) and reject the HF shape with `422 EmbedReqV2.inputs is of type string but should be of type Object`. Add `SagemakerCohereEmbeddingConfig` that reuses Bedrock/Cohere request and response transforms, and route SageMaker endpoint names containing `cohere` or a Cohere embed model fragment (`embed-multilingual`, `embed-english`, `embed-v3`, `embed-v4`) to it. Supports `input_type`, `dimensions`, and `encoding_format`. Voyage and HuggingFace SageMaker endpoints are unchanged. Co-authored-by: Cursor * refactor(sagemaker): simplify cohere detection and align with file conventions - Detect Cohere SageMaker endpoints with a single `"cohere" in model.lower()` check, mirroring the existing Voyage branch instead of a separate helper function and marker constant. - Drop instance caches of sub-configs; instantiate `BedrockCohereEmbeddingConfig` / `CohereEmbeddingConfig` per call to match the existing pattern in `BedrockCohereEmbeddingConfig._transform_request`. - Match `SagemakerEmbeddingConfig`'s signatures, defaults, and `Any` typing for `logging_obj`; collapse the input-normalization helper inline. - Inline `transform_embedding_response` input lookup; no behavior change. Co-authored-by: Cursor * fix(sagemaker): restore provider-supported embedding params after map Cohere input_type is advertised in get_supported_openai_params but was filtered out of non_default_params by OPENAI_EMBEDDING_PARAMS before map_openai_params ran. Merge supported params from passed_params after map (same path Greptile flagged). Handle input_type explicitly in SagemakerCohereEmbeddingConfig.map_openai_params and add an integration test through get_optional_params_embeddings. Co-authored-by: Cursor * fix(embeddings): only restore non-OpenAI supported params after map The post-map restore loop must skip OPENAI_EMBEDDING_PARAMS so mapped fields (e.g. dimensions -> output_dimension) are not duplicated under their OpenAI names. Align SageMaker embedding import order with sibling files and add a regression test for dimensions mapping. Co-authored-by: Cursor * fix(sagemaker): avoid double post_call on Cohere embedding response Greptile review on #28613 caught that `CohereEmbeddingConfig._transform_response` calls `logging_obj.post_call` internally. The SageMaker embedding handler already calls `post_call` once before invoking the transform, so the Cohere SageMaker path fired callbacks, cost calculators, and log handlers twice per request. Extract the parsing body of `_transform_response` into `_populate_embedding_response` (pure extract-method, no behavior change for existing Cohere direct or Bedrock Cohere paths, which keep calling `_transform_response`). Have `SagemakerCohereEmbeddingConfig` call the new helper directly so it parses the response without re-logging. Add a regression test asserting `logging_obj.post_call` is not invoked by the SageMaker Cohere transform. Co-authored-by: Cursor --------- Co-authored-by: Cursor --- .../llms/cohere/embed/v1_transformation.py | 35 +++- litellm/llms/sagemaker/completion/handler.py | 2 +- .../embedding/cohere_transformation.py | 141 +++++++++++++++ .../sagemaker/embedding/transformation.py | 22 ++- litellm/utils.py | 15 ++ .../test_sagemaker_embedding_voyage.py | 169 ++++++++++++++++++ 6 files changed, 365 insertions(+), 19 deletions(-) create mode 100644 litellm/llms/sagemaker/embedding/cohere_transformation.py diff --git a/litellm/llms/cohere/embed/v1_transformation.py b/litellm/llms/cohere/embed/v1_transformation.py index feca9cb5b88..82c901e7eca 100644 --- a/litellm/llms/cohere/embed/v1_transformation.py +++ b/litellm/llms/cohere/embed/v1_transformation.py @@ -110,15 +110,35 @@ class CohereEmbeddingConfig: additional_args={"complete_input_dict": data}, original_response=response_json, ) + return self._populate_embedding_response( + response_json=response_json, + model_response=model_response, + model=model, + encoding=encoding, + input=input, + ) + + def _populate_embedding_response( + self, + response_json: dict, + model_response: EmbeddingResponse, + model: str, + encoding: Any, + input: list, + ) -> EmbeddingResponse: """ - response + Parse a Cohere embed response body into an OpenAI-style EmbeddingResponse. + + Split out from `_transform_response` so callers that already log + `post_call` themselves (e.g. SageMaker's embedding handler) can reuse + the parsing without triggering a second `post_call`. + + Response shape: { 'object': "list", - 'data': [ - - ] - 'model', - 'usage' + 'data': [...], + 'model', + 'usage', } """ embeddings = response_json["embeddings"] @@ -149,9 +169,6 @@ class CohereEmbeddingConfig: model_response.object = "list" model_response.data = output_data model_response.model = model - input_tokens = 0 - for text in input: - input_tokens += len(encoding.encode(text)) setattr( model_response, diff --git a/litellm/llms/sagemaker/completion/handler.py b/litellm/llms/sagemaker/completion/handler.py index efbb218f575..de7be18e8ba 100644 --- a/litellm/llms/sagemaker/completion/handler.py +++ b/litellm/llms/sagemaker/completion/handler.py @@ -578,7 +578,7 @@ class SagemakerLLM(BaseAWSLLM): logger_fn=None, ): """ - Supports both Huggingface Jumpstart embeddings and Voyage models + Supports Hugging Face (TGI), Voyage, and Cohere embedding endpoints """ ### BOTO3 INIT import boto3 diff --git a/litellm/llms/sagemaker/embedding/cohere_transformation.py b/litellm/llms/sagemaker/embedding/cohere_transformation.py new file mode 100644 index 00000000000..fdb67202ebb --- /dev/null +++ b/litellm/llms/sagemaker/embedding/cohere_transformation.py @@ -0,0 +1,141 @@ +""" +Translate from OpenAI's `/v1/embeddings` to Sagemaker's `/invoke` + +In the native Cohere embed format for self-hosted Cohere endpoints +(AWS Marketplace / JumpStart). Cohere containers expect +`{"texts": [...], "input_type": "..."}` and reject the HuggingFace TGI shape +`{"inputs": [...]}` with `422 EmbedReqV2.inputs is of type string but should +be of type Object`. + +Reference: https://docs.cohere.com/v2/reference/embed +""" + +from typing import TYPE_CHECKING, Any, List, Optional, Union, cast + +if TYPE_CHECKING: + from litellm.types.llms.openai import AllEmbeddingInputValues + +from httpx._models import Headers, Response + +import litellm +from litellm.llms.base_llm.chat.transformation import BaseLLMException +from litellm.llms.base_llm.embedding.transformation import BaseEmbeddingConfig +from litellm.llms.bedrock.embed.cohere_transformation import ( + BedrockCohereEmbeddingConfig, +) +from litellm.llms.cohere.embed.v1_transformation import CohereEmbeddingConfig +from litellm.types.utils import EmbeddingResponse + +from ..common_utils import SagemakerError + + +class SagemakerCohereEmbeddingConfig(BaseEmbeddingConfig): + """ + SageMaker invoke payload for self-hosted Cohere embed models. + """ + + def __init__(self) -> None: + pass + + def get_supported_openai_params(self, model: str) -> List[str]: + return ["encoding_format", "dimensions", "input_type"] + + def map_openai_params( + self, + non_default_params: dict, + optional_params: dict, + model: str, + drop_params: bool, + ) -> dict: + optional_params = BedrockCohereEmbeddingConfig().map_openai_params( + non_default_params=non_default_params, + optional_params=optional_params, + ) + if "input_type" in non_default_params: + optional_params["input_type"] = non_default_params["input_type"] + return optional_params + + def get_error_class( + self, error_message: str, status_code: int, headers: Union[dict, Headers] + ) -> BaseLLMException: + return SagemakerError( + message=error_message, status_code=status_code, headers=headers + ) + + def transform_embedding_request( + self, + model: str, + input: "AllEmbeddingInputValues", + optional_params: dict, + headers: dict, + ) -> dict: + """ + Transform embedding request for Cohere models on SageMaker + """ + if isinstance(input, str): + input_list: List[str] = [input] + elif isinstance(input, list): + if input and (isinstance(input[0], list) or isinstance(input[0], int)): + raise ValueError("Input must be a list of strings") + input_list = cast(List[str], input) + else: + input_list = [str(input)] + + return dict( + BedrockCohereEmbeddingConfig()._transform_request( + model=model, + input=input_list, + inference_params=optional_params, + ) + ) + + def transform_embedding_response( + self, + model: str, + raw_response: Response, + model_response: "EmbeddingResponse", + logging_obj: Any, + api_key: Optional[str] = None, + request_data: dict = {}, + optional_params: dict = {}, + litellm_params: dict = {}, + ) -> "EmbeddingResponse": + """ + Transform embedding response for Cohere models on SageMaker. + + Uses `CohereEmbeddingConfig._populate_embedding_response` (not + `_transform_response`) so we do not log `post_call` a second time + — the SageMaker embedding handler already logs `post_call` before + invoking this transform. + """ + input_value = ( + logging_obj.model_call_details.get("input") + or request_data.get("texts") + or request_data.get("images") + or [] + ) + if isinstance(input_value, str): + input_value = [input_value] + + return CohereEmbeddingConfig()._populate_embedding_response( + response_json=raw_response.json(), + model_response=model_response, + model=model, + encoding=litellm.encoding, + input=input_value, + ) + + def validate_environment( + self, + headers: dict, + model: str, + messages: List[Any], + optional_params: dict, + litellm_params: dict, + api_key: Optional[str] = None, + api_base: Optional[str] = None, + ) -> dict: + """ + Validate environment for SageMaker Cohere embeddings + """ + return {"Content-Type": "application/json"} diff --git a/litellm/llms/sagemaker/embedding/transformation.py b/litellm/llms/sagemaker/embedding/transformation.py index 09bdb9295e7..5e2aa99534f 100644 --- a/litellm/llms/sagemaker/embedding/transformation.py +++ b/litellm/llms/sagemaker/embedding/transformation.py @@ -11,12 +11,13 @@ if TYPE_CHECKING: from httpx._models import Headers, Response -from litellm.llms.base_llm.embedding.transformation import BaseEmbeddingConfig from litellm.llms.base_llm.chat.transformation import BaseLLMException -from litellm.types.utils import Usage, EmbeddingResponse +from litellm.llms.base_llm.embedding.transformation import BaseEmbeddingConfig from litellm.llms.voyage.embedding.transformation import VoyageEmbeddingConfig +from litellm.types.utils import EmbeddingResponse, Usage from ..common_utils import SagemakerError +from .cohere_transformation import SagemakerCohereEmbeddingConfig class SagemakerEmbeddingConfig(BaseEmbeddingConfig): @@ -38,17 +39,20 @@ class SagemakerEmbeddingConfig(BaseEmbeddingConfig): Returns: Appropriate embedding config instance """ - if "voyage" in model.lower(): + model_lower = model.lower() + if "voyage" in model_lower: return VoyageEmbeddingConfig() - else: - return cls() + if "cohere" in model_lower: + return SagemakerCohereEmbeddingConfig() + return cls() def get_supported_openai_params(self, model: str) -> List[str]: - # Check if this is an embedding model - if "voyage" in model.lower(): + model_lower = model.lower() + if "voyage" in model_lower: return VoyageEmbeddingConfig().get_supported_openai_params(model) - else: - return [] + if "cohere" in model_lower: + return SagemakerCohereEmbeddingConfig().get_supported_openai_params(model) + return [] def map_openai_params( self, diff --git a/litellm/utils.py b/litellm/utils.py index 18ee811f0f1..c28a88e0f1c 100644 --- a/litellm/utils.py +++ b/litellm/utils.py @@ -3350,6 +3350,21 @@ def get_optional_params_embeddings( # noqa: PLR0915 model=model, drop_params=drop_params if drop_params is not None else False, ) + # Provider-only params (e.g. Cohere input_type) are not in + # OPENAI_EMBEDDING_PARAMS, so embedding_pre_process drops them from + # non_default_params before map_openai_params. Restore only those extras + # from passed_params — skip OPENAI_EMBEDDING_PARAMS to avoid duplicating + # values already mapped (e.g. dimensions -> output_dimension). + if supported_params: + for param in supported_params: + if param in OPENAI_EMBEDDING_PARAMS: + continue + if ( + param in passed_params + and passed_params[param] is not None + and param not in optional_params + ): + optional_params[param] = passed_params[param] ## raise exception if non-default value passed for non-openai/azure embedding calls elif custom_llm_provider == "openai": # 'dimensions` is only supported in `text-embedding-3` and later models diff --git a/tests/test_litellm/llms/sagemaker/test_sagemaker_embedding_voyage.py b/tests/test_litellm/llms/sagemaker/test_sagemaker_embedding_voyage.py index a36aec32d13..943a3160bb7 100644 --- a/tests/test_litellm/llms/sagemaker/test_sagemaker_embedding_voyage.py +++ b/tests/test_litellm/llms/sagemaker/test_sagemaker_embedding_voyage.py @@ -17,6 +17,9 @@ import pytest sys.path.insert(0, os.path.abspath("../../../../..")) from litellm import embedding +from litellm.llms.sagemaker.embedding.cohere_transformation import ( + SagemakerCohereEmbeddingConfig, +) from litellm.llms.sagemaker.embedding.transformation import SagemakerEmbeddingConfig from litellm.llms.voyage.embedding.transformation import VoyageEmbeddingConfig from litellm.types.utils import EmbeddingResponse, Usage @@ -54,6 +57,172 @@ class TestSagemakerEmbeddingFactory: assert isinstance(config2, VoyageEmbeddingConfig) assert isinstance(config3, VoyageEmbeddingConfig) + def test_get_model_config_cohere_model(self): + """Cohere SageMaker endpoints route to SagemakerCohereEmbeddingConfig""" + for endpoint_name in ( + "cohere.embed-multilingual-v3", + "cohere-embed-english-v3-prod", + "my-cohere-marketplace-endpoint", + "COHERE-EMBED-V4", + ): + config = SagemakerEmbeddingConfig.get_model_config(endpoint_name) + assert isinstance(config, SagemakerCohereEmbeddingConfig), endpoint_name + + +class TestSagemakerCohereEmbeddingConfig: + """Cohere-specific SageMaker embedding request/response transforms""" + + def setup_method(self): + self.config = SagemakerCohereEmbeddingConfig() + + MODEL = "cohere.embed-multilingual-v3" + + def test_transform_request_uses_cohere_payload(self): + """Bug repro: request must use `texts` + `input_type`, not HF `inputs`""" + result = self.config.transform_embedding_request( + model=self.MODEL, + input=["hello"], + optional_params={"input_type": "search_query"}, + headers={}, + ) + assert "inputs" not in result + assert result["texts"] == ["hello"] + assert result["input_type"] == "search_query" + + def test_transform_request_default_input_type(self): + result = self.config.transform_embedding_request( + model=self.MODEL, + input=["hello"], + optional_params={}, + headers={}, + ) + assert result["texts"] == ["hello"] + assert result["input_type"] == "search_document" + + def test_transform_request_normalizes_string_input(self): + result = self.config.transform_embedding_request( + model=self.MODEL, + input="hello", + optional_params={}, + headers={}, + ) + assert result["texts"] == ["hello"] + + def test_map_openai_params_dimensions_to_output_dimension(self): + params = self.config.map_openai_params( + non_default_params={"dimensions": 512, "encoding_format": "float"}, + optional_params={}, + model=self.MODEL, + drop_params=False, + ) + assert params["output_dimension"] == 512 + assert params["embedding_types"] == ["float"] + + def test_map_openai_params_input_type_from_non_default_params(self): + params = self.config.map_openai_params( + non_default_params={"input_type": "search_query"}, + optional_params={}, + model=self.MODEL, + drop_params=False, + ) + assert params["input_type"] == "search_query" + + def test_get_optional_params_embeddings_preserves_input_type(self): + """Exercises get_optional_params_embeddings, not transform in isolation.""" + from litellm.utils import get_optional_params_embeddings + + optional_params = get_optional_params_embeddings( + model=self.MODEL, + custom_llm_provider="sagemaker", + input_type="search_query", + ) + assert optional_params.get("input_type") == "search_query" + + body = self.config.transform_embedding_request( + model=self.MODEL, + input=["hello"], + optional_params=optional_params, + headers={}, + ) + assert body["texts"] == ["hello"] + assert body["input_type"] == "search_query" + + def test_get_optional_params_embeddings_maps_dimensions_without_duplicate(self): + """dimensions must map to output_dimension only, not also stay as dimensions.""" + from litellm.utils import get_optional_params_embeddings + + optional_params = get_optional_params_embeddings( + model=self.MODEL, + custom_llm_provider="sagemaker", + dimensions=512, + input_type="search_query", + ) + assert optional_params.get("output_dimension") == 512 + assert "dimensions" not in optional_params + assert optional_params.get("input_type") == "search_query" + + def test_transform_response_parses_cohere_payload(self): + cohere_response = { + "embeddings": [[0.1, 0.2, 0.3]], + "meta": {"billed_units": {"input_tokens": 2}}, + } + mock_response = httpx.Response( + status_code=200, + content=json.dumps(cohere_response).encode("utf-8"), + headers={"content-type": "application/json"}, + ) + logging_obj = MagicMock() + logging_obj.model_call_details = {"input": ["hello"]} + + result = self.config.transform_embedding_response( + model=self.MODEL, + raw_response=mock_response, + model_response=EmbeddingResponse(), + logging_obj=logging_obj, + api_key=None, + request_data={"texts": ["hello"], "input_type": "search_query"}, + optional_params={}, + litellm_params={}, + ) + + assert result.object == "list" + assert len(result.data) == 1 + assert result.data[0]["embedding"] == [0.1, 0.2, 0.3] + assert result.usage.prompt_tokens == 2 + + def test_transform_response_does_not_double_call_post_call(self): + """ + Greptile review fix: SageMaker handler already calls + `logging_obj.post_call` once before invoking + `transform_embedding_response`. The transform must NOT call it again, + otherwise callbacks, cost calculators, and log handlers double-fire + for every Cohere SageMaker embedding call. + """ + cohere_response = { + "embeddings": [[0.1, 0.2, 0.3]], + "meta": {"billed_units": {"input_tokens": 2}}, + } + mock_response = httpx.Response( + status_code=200, + content=json.dumps(cohere_response).encode("utf-8"), + headers={"content-type": "application/json"}, + ) + logging_obj = MagicMock() + logging_obj.model_call_details = {"input": ["hello"]} + + self.config.transform_embedding_response( + model=self.MODEL, + raw_response=mock_response, + model_response=EmbeddingResponse(), + logging_obj=logging_obj, + api_key=None, + request_data={"texts": ["hello"], "input_type": "search_query"}, + optional_params={}, + litellm_params={}, + ) + + logging_obj.post_call.assert_not_called() + class TestVoyageEmbeddingConfig: """Test Voyage-specific embedding configuration""" From a3c953ed4e7eb583cb80a78f280325d0ac447702 Mon Sep 17 00:00:00 2001 From: Krrish Dholakia Date: Fri, 22 May 2026 12:10:37 -0700 Subject: [PATCH 09/13] style: apply black formatting to fix lint CI (LIT-3274) (#28639) (#28641) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * fix(bedrock): strip bedrock/ prefix and URL-encode ARNs in get_bedrock_model_id for invoke path The invoke path (used by /v1/messages → Anthropic SDK / Claude Code) called get_bedrock_model_id() which, when falling back to the raw model string, did not strip the 'bedrock/' routing prefix and did not URL-encode ARNs. For a model like: bedrock/arn:aws:bedrock:us-east-1::inference-profile/global.anthropic... the URL built was: /model/bedrock/arn:aws:bedrock:…/invoke-with-response-stream ❌ Bedrock returned a JSON error body. LiteLLM's AWSEventStreamDecoder passed those bytes into botocore's EventStreamBuffer which expects binary event-stream framing. Checksum validation failed on the JSON prelude (0x223a7b22 == ':{"') producing a misleading botocore.eventstream.ChecksumMismatch instead of the actual Bedrock error. Fix: strip 'bedrock/' (and 'invoke/') routing prefix from model string, then URL-encode if the result is an ARN — matching what the converse path already does in converse_handler.py. Fixes: LIT-3274 * fix(bedrock): use strip_bedrock_routing_prefix to handle compound prefixes Address greptile review: the original fix used a loop with break, so bedrock/invoke/arn:... only stripped bedrock/ leaving invoke/arn:... which is not an ARN → fell through to .replace('invoke/','',1) → bare unencoded ARN → same malformed-URL bug. strip_bedrock_routing_prefix() iterates without break, correctly stripping bedrock/ then invoke/ in sequence. Also adds test case for the compound-prefix scenario. * style: apply black formatting to fix lint CI (LIT-3274) --------- Co-authored-by: oss-agent-shin Co-authored-by: LiteLLM Bot --- litellm/llms/bedrock/base_aws_llm.py | 18 ++++ .../llms/bedrock/test_base_aws_llm.py | 99 +++++++++++++++++++ 2 files changed, 117 insertions(+) diff --git a/litellm/llms/bedrock/base_aws_llm.py b/litellm/llms/bedrock/base_aws_llm.py index 9dd2b055a12..8b316a587b4 100644 --- a/litellm/llms/bedrock/base_aws_llm.py +++ b/litellm/llms/bedrock/base_aws_llm.py @@ -450,6 +450,24 @@ class BaseAWSLLM: model_id = BaseAWSLLM.encode_model_id(model_id=model_id) else: model_id = model + # Strip LiteLLM routing prefixes (e.g. "bedrock/", "invoke/", + # "bedrock/invoke/", "bedrock/converse/") that are not part of the + # actual Bedrock model ID. The converse path already does this; the + # invoke path must do the same so that ARN models such as + # bedrock/arn:aws:bedrock:…:inference-profile/global.anthropic.… + # are not forwarded verbatim to the Bedrock API, which would produce + # a malformed URL and cause botocore's EventStreamBuffer to receive + # a JSON error body instead of a binary event-stream — surfaced as a + # misleading ChecksumMismatch (0x223a7b22 == ':{"'). + # Use strip_bedrock_routing_prefix (no break) so compound prefixes + # like "bedrock/invoke/arn:..." are fully stripped in one call. + from litellm.llms.bedrock.common_utils import strip_bedrock_routing_prefix + + model_id = strip_bedrock_routing_prefix(model_id) + # URL-encode ARNs so colons and slashes are safe in the URL path. + if model_id.startswith("arn:"): + model_id = BaseAWSLLM.encode_model_id(model_id=model_id) + return model_id model_id = model_id.replace("invoke/", "", 1) if provider == "llama" and "llama/" in model_id: diff --git a/tests/test_litellm/llms/bedrock/test_base_aws_llm.py b/tests/test_litellm/llms/bedrock/test_base_aws_llm.py index a4969e5dacc..10fc358e3a5 100644 --- a/tests/test_litellm/llms/bedrock/test_base_aws_llm.py +++ b/tests/test_litellm/llms/bedrock/test_base_aws_llm.py @@ -2112,3 +2112,102 @@ def test_is_already_running_as_role_ssl_verify_passed(): mock_boto3_client.assert_called_once_with( "sts", verify="/path/to/ca-bundle.crt" ) + + +# --------------------------------------------------------------------------- +# LIT-3274: get_bedrock_model_id must strip "bedrock/" prefix and URL-encode +# ARNs for the invoke path (invoke-with-response-stream). Without this fix +# the Bedrock API receives a malformed URL, returns a JSON error body, and +# botocore's EventStreamBuffer raises ChecksumMismatch instead of the real +# error. 0x223a7b22 == ':{\"' — the start of a JSON object. +# --------------------------------------------------------------------------- + + +class TestGetBedrockModelIdArnHandling: + """Unit tests for get_bedrock_model_id with inference-profile ARNs.""" + + ARN = "arn:aws:bedrock:us-east-1:086734376398:inference-profile/global.anthropic.claude-sonnet-4-5-20250929-v1:0" + + def _call(self, model: str, optional_params: dict | None = None) -> str: + from litellm.llms.bedrock.base_aws_llm import BaseAWSLLM + + provider = BaseAWSLLM.get_bedrock_invoke_provider(model) + return BaseAWSLLM.get_bedrock_model_id( + model=model, + provider=provider, + optional_params=optional_params or {}, + ) + + def test_arn_with_bedrock_prefix_is_stripped_and_encoded(self): + """bedrock/arn:... must not appear verbatim in the model_id.""" + model_id = self._call(f"bedrock/{self.ARN}") + assert ( + "bedrock/arn" not in model_id + ), f"'bedrock/' prefix not stripped; got: {model_id}" + # Must be URL-encoded (colons → %3A) + assert "%3A" in model_id, f"ARN not URL-encoded; got: {model_id}" + assert "%2F" in model_id, f"ARN slashes not URL-encoded; got: {model_id}" + + def test_arn_with_compound_bedrock_invoke_prefix_is_fully_stripped_and_encoded( + self, + ): + """bedrock/invoke/arn:... — compound prefix — must be fully stripped. + + The old fix used ``break`` after the first matched prefix, so + ``bedrock/invoke/arn:...`` would only strip ``bedrock/``, leaving + ``invoke/arn:...``. The subsequent ``.replace('invoke/', '')`` call + then returned the bare unencoded ARN, reproducing the same + malformed-URL bug the fix aimed to prevent. + + strip_bedrock_routing_prefix() has no break and handles this correctly. + """ + model_id = self._call(f"bedrock/invoke/{self.ARN}") + assert ( + "invoke/" not in model_id + ), f"'invoke/' prefix not stripped; got: {model_id}" + assert ( + "bedrock/" not in model_id + ), f"'bedrock/' prefix not stripped; got: {model_id}" + assert "%3A" in model_id, f"ARN not URL-encoded; got: {model_id}" + assert "%2F" in model_id, f"ARN slashes not URL-encoded; got: {model_id}" + + def test_bare_arn_is_encoded(self): + """Direct ARN without routing prefix must also be URL-encoded.""" + model_id = self._call(self.ARN) + assert "%3A" in model_id, f"ARN not URL-encoded; got: {model_id}" + assert "%2F" in model_id, f"ARN slashes not URL-encoded; got: {model_id}" + + def test_arn_url_matches_expected(self): + """Full URL built from messages config must match expected encoded form.""" + import urllib.parse + from litellm.llms.bedrock.messages.invoke_transformations.anthropic_claude3_transformation import ( + AmazonAnthropicClaudeMessagesConfig, + ) + + config = AmazonAnthropicClaudeMessagesConfig() + url = config.get_complete_url( + api_base=None, + api_key=None, + model=f"bedrock/{self.ARN}", + optional_params={"aws_region_name": "us-east-1"}, + litellm_params={}, + stream=True, + ) + encoded_arn = urllib.parse.quote(self.ARN, safe="") + expected = ( + f"https://bedrock-runtime.us-east-1.amazonaws.com" + f"/model/{encoded_arn}/invoke-with-response-stream" + ) + assert ( + url == expected + ), f"URL mismatch:\n got: {url}\n expected: {expected}" + + def test_regular_model_id_unaffected(self): + """Non-ARN model IDs must continue to work as before.""" + model_id = self._call("anthropic.claude-3-sonnet-20240229-v1:0") + assert model_id == "anthropic.claude-3-sonnet-20240229-v1:0" + + def test_invoke_prefixed_model_unaffected(self): + """invoke/ prefix stripping still works after the fix.""" + model_id = self._call("invoke/anthropic.claude-3-sonnet-20240229-v1:0") + assert model_id == "anthropic.claude-3-sonnet-20240229-v1:0" From 1b141bc588cf1d759975470ec69717e42ff1d1b0 Mon Sep 17 00:00:00 2001 From: milan-berri Date: Sat, 23 May 2026 00:39:24 +0300 Subject: [PATCH 10/13] fix(bedrock): decouple STS region from Bedrock aws_region_name (#28245) * fix(bedrock): decouple STS region from Bedrock aws_region_name STS AssumeRole now resolves signing region from aws_sts_endpoint (parsed host) or AWS_REGION/AWS_DEFAULT_REGION instead of aws_region_name, fixing air-gapped cross-region Bedrock setups and endpoint/signature mismatches. Co-authored-by: Cursor * test(bedrock): add regression coverage for _build_sts_client_kwargs Parametrize _resolve_sts_region and _build_sts_client_kwargs matrix cases, and assert IRSA/web-identity paths use aligned STS endpoint and region_name. Co-authored-by: Cursor * refactor(bedrock): tighten STS region helpers and drop redundant web-identity endpoint synthesis Co-authored-by: Cursor * test(bedrock): cover FIPS, GovCloud, and China STS endpoints Addresses greptile P2: regex sts(?:-fips)? supported sts-fips hosts but was not exercised by the parametrized parse test. Co-authored-by: Cursor --------- Co-authored-by: Cursor --- litellm/llms/bedrock/base_aws_llm.py | 99 +++--- .../llms/bedrock/test_base_aws_llm.py | 324 +++++++++++++++++- 2 files changed, 377 insertions(+), 46 deletions(-) diff --git a/litellm/llms/bedrock/base_aws_llm.py b/litellm/llms/bedrock/base_aws_llm.py index 8b316a587b4..b659c1b0a0a 100644 --- a/litellm/llms/bedrock/base_aws_llm.py +++ b/litellm/llms/bedrock/base_aws_llm.py @@ -44,6 +44,12 @@ else: # (e.g. "us-east-1", "eu-west-2", "us-gov-west-1", "cn-north-1"). _VALID_AWS_REGION_PATTERN = re.compile(r"\A[a-z0-9-]+\Z") +# Regional STS hostnames, e.g. sts.eu-west-1.amazonaws.com or +# vpce-xxx.sts.eu-west-1.vpce.amazonaws.com +_STS_REGION_FROM_ENDPOINT_PATTERN = re.compile( + r"(?:^|\.)sts(?:-fips)?\.([a-z0-9-]+)\.(?:amazonaws\.com(?:\.cn)?|vpce\.amazonaws\.com)" +) + class Boto3CredentialsInfo(BaseModel): credentials: Credentials @@ -651,6 +657,40 @@ class BaseAWSLLM: "Region names must contain only lowercase letters, digits, and hyphens." ) + @staticmethod + def _parse_sts_region_from_endpoint( + aws_sts_endpoint: Optional[str], + ) -> Optional[str]: + """Extract region from sts.{region}.amazonaws.com or vpce-x.sts.{region}.vpce.amazonaws.com.""" + if not aws_sts_endpoint: + return None + host = urllib.parse.urlparse(aws_sts_endpoint).hostname or "" + match = _STS_REGION_FROM_ENDPOINT_PATTERN.search(host) + return match.group(1) if match else None + + @staticmethod + def _resolve_sts_region(aws_sts_endpoint: Optional[str] = None) -> Optional[str]: + """STS signing region: parsed from aws_sts_endpoint else AWS_REGION / AWS_DEFAULT_REGION.""" + return ( + BaseAWSLLM._parse_sts_region_from_endpoint(aws_sts_endpoint) + or os.getenv("AWS_REGION") + or os.getenv("AWS_DEFAULT_REGION") + ) + + def _build_sts_client_kwargs( + self, + aws_sts_endpoint: Optional[str] = None, + ssl_verify: Optional[Union[bool, str]] = None, + ) -> dict: + """STS client kwargs with aligned endpoint_url and region_name (SigV4).""" + kwargs: dict = {"verify": self._get_ssl_verify(ssl_verify)} + if aws_sts_endpoint is not None: + kwargs["endpoint_url"] = aws_sts_endpoint + sts_region = self._resolve_sts_region(aws_sts_endpoint) + if sts_region is not None: + kwargs["region_name"] = sts_region + return kwargs + def get_aws_region_name_for_non_llm_api_calls( self, aws_region_name: Optional[str] = None, @@ -805,11 +845,6 @@ class BaseAWSLLM: f"IN Web Identity Token: {aws_web_identity_token} | Role Name: {aws_role_name} | Session Name: {aws_session_name}" ) - if aws_sts_endpoint is None: - sts_endpoint = f"https://sts.{aws_region_name}.amazonaws.com" - else: - sts_endpoint = aws_sts_endpoint - oidc_token = get_secret(aws_web_identity_token) if oidc_token is None: @@ -818,13 +853,13 @@ class BaseAWSLLM: status_code=401, ) + sts_client_kwargs = self._build_sts_client_kwargs( + aws_sts_endpoint=aws_sts_endpoint, + ssl_verify=ssl_verify, + ) + with tracer.trace("boto3.client(sts)"): - sts_client = boto3.client( - "sts", - region_name=aws_region_name, - endpoint_url=sts_endpoint, - verify=self._get_ssl_verify(ssl_verify), - ) + sts_client = boto3.client("sts", **sts_client_kwargs) # https://docs.aws.amazon.com/STS/latest/APIReference/API_AssumeRoleWithWebIdentity.html # https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sts/client/assume_role_with_web_identity.html @@ -865,7 +900,6 @@ class BaseAWSLLM: irsa_role_arn: str, aws_role_name: str, aws_session_name: str, - region: str, web_identity_token_file: str, aws_external_id: Optional[str] = None, aws_sts_endpoint: Optional[str] = None, @@ -880,12 +914,10 @@ class BaseAWSLLM: with open(web_identity_token_file, "r") as f: web_identity_token = f.read().strip() - irsa_sts_kwargs: dict = { - "region_name": region, - "verify": self._get_ssl_verify(ssl_verify), - } - if aws_sts_endpoint is not None: - irsa_sts_kwargs["endpoint_url"] = aws_sts_endpoint + irsa_sts_kwargs = self._build_sts_client_kwargs( + aws_sts_endpoint=aws_sts_endpoint, + ssl_verify=ssl_verify, + ) # Create an STS client without credentials with tracer.trace("boto3.client(sts) for manual IRSA"): @@ -942,7 +974,6 @@ class BaseAWSLLM: self, aws_role_name: str, aws_session_name: str, - region: str, aws_external_id: Optional[str] = None, aws_sts_endpoint: Optional[str] = None, ssl_verify: Optional[Union[bool, str]] = None, @@ -950,12 +981,10 @@ class BaseAWSLLM: """Handle same-account role assumption for IRSA.""" import boto3 - irsa_sts_kwargs: dict = { - "region_name": region, - "verify": self._get_ssl_verify(ssl_verify), - } - if aws_sts_endpoint is not None: - irsa_sts_kwargs["endpoint_url"] = aws_sts_endpoint + irsa_sts_kwargs = self._build_sts_client_kwargs( + aws_sts_endpoint=aws_sts_endpoint, + ssl_verify=ssl_verify, + ) verbose_logger.debug("Same account role assumption, using automatic IRSA") with tracer.trace("boto3.client(sts) with automatic IRSA"): @@ -1028,12 +1057,6 @@ class BaseAWSLLM: web_identity_token_file = os.getenv("AWS_WEB_IDENTITY_TOKEN_FILE") irsa_role_arn = os.getenv("AWS_ROLE_ARN") - region = ( - aws_region_name - or os.getenv("AWS_REGION") - or os.getenv("AWS_DEFAULT_REGION") - ) - # If we have IRSA environment variables and no explicit credentials, # we need to use the web identity token flow if ( @@ -1049,16 +1072,12 @@ class BaseAWSLLM: ) try: - # Use passed-in region when set, else env, else default (align with AssumeRole path) - region = region or "us-east-1" - # Check if we need to do cross-account role assumption if aws_role_name != irsa_role_arn: sts_response = self._handle_irsa_cross_account( irsa_role_arn, aws_role_name, aws_session_name, - region, web_identity_token_file, aws_external_id, aws_sts_endpoint=aws_sts_endpoint, @@ -1068,7 +1087,6 @@ class BaseAWSLLM: sts_response = self._handle_irsa_same_account( aws_role_name, aws_session_name, - region, aws_external_id, aws_sts_endpoint=aws_sts_endpoint, ssl_verify=ssl_verify, @@ -1092,11 +1110,10 @@ class BaseAWSLLM: # In EKS/IRSA environments, use ambient credentials (no explicit keys needed) # This allows the web identity token to work automatically - sts_client_kwargs: dict = {"verify": self._get_ssl_verify(ssl_verify)} - if region is not None: - sts_client_kwargs["region_name"] = region - if aws_sts_endpoint is not None: - sts_client_kwargs["endpoint_url"] = aws_sts_endpoint + sts_client_kwargs = self._build_sts_client_kwargs( + aws_sts_endpoint=aws_sts_endpoint, + ssl_verify=ssl_verify, + ) if aws_access_key_id is None and aws_secret_access_key is None: with tracer.trace("boto3.client(sts)"): sts_client = boto3.client("sts", **sts_client_kwargs) diff --git a/tests/test_litellm/llms/bedrock/test_base_aws_llm.py b/tests/test_litellm/llms/bedrock/test_base_aws_llm.py index 10fc358e3a5..3f91f6ac26e 100644 --- a/tests/test_litellm/llms/bedrock/test_base_aws_llm.py +++ b/tests/test_litellm/llms/bedrock/test_base_aws_llm.py @@ -869,14 +869,18 @@ def test_different_roles_without_session_names_should_not_share_cache(): ({}, {"verify": True}), ( {"aws_region_name": "us-east-1"}, - {"region_name": "us-east-1", "verify": True}, + {"verify": True}, ), ( {"aws_sts_endpoint": "https://sts.eu-west-1.amazonaws.com"}, - {"endpoint_url": "https://sts.eu-west-1.amazonaws.com", "verify": True}, + { + "endpoint_url": "https://sts.eu-west-1.amazonaws.com", + "region_name": "eu-west-1", + "verify": True, + }, ), ], - ids=["no_region_or_endpoint", "regional_sts", "explicit_sts_endpoint"], + ids=["no_region_or_endpoint", "bedrock_region_ignored_for_sts", "explicit_sts_endpoint"], ) def test_eks_irsa_ambient_credentials_used(role_kwargs, expected_client_kwargs): """ @@ -925,6 +929,316 @@ def test_eks_irsa_ambient_credentials_used(role_kwargs, expected_client_kwargs): assert ttl is not None +@pytest.mark.parametrize( + "endpoint,expected_region", + [ + ("https://sts.eu-west-1.amazonaws.com", "eu-west-1"), + ("https://sts.us-east-1.amazonaws.com", "us-east-1"), + ("https://sts-fips.us-east-1.amazonaws.com", "us-east-1"), + ("https://sts-fips.us-gov-west-1.amazonaws.com", "us-gov-west-1"), + ("https://sts.us-gov-west-1.amazonaws.com", "us-gov-west-1"), + ("https://sts.cn-north-1.amazonaws.com.cn", "cn-north-1"), + ( + "https://vpce-abc123.sts.eu-west-1.vpce.amazonaws.com", + "eu-west-1", + ), + ("https://sts.amazonaws.com", None), + ("https://invalid.example.com", None), + ], +) +def test_parse_sts_region_from_endpoint(endpoint, expected_region): + assert BaseAWSLLM._parse_sts_region_from_endpoint(endpoint) == expected_region + + +@pytest.mark.parametrize( + "env,aws_sts_endpoint,expected_region", + [ + ({}, None, None), + ({"AWS_REGION": "us-east-1"}, None, "us-east-1"), + ({"AWS_DEFAULT_REGION": "ap-southeast-1"}, None, "ap-southeast-1"), + ({}, "https://sts.eu-west-1.amazonaws.com", "eu-west-1"), + ( + {"AWS_REGION": "us-east-1"}, + "https://sts.eu-west-1.amazonaws.com", + "eu-west-1", + ), + ({}, "https://sts.amazonaws.com", None), + ( + {}, + "https://vpce-abc.sts.eu-central-1.vpce.amazonaws.com", + "eu-central-1", + ), + ], + ids=[ + "no_env_no_endpoint", + "env_region", + "env_default_region", + "parsed_from_endpoint", + "parsed_endpoint_over_env", + "global_endpoint", + "vpce_endpoint", + ], +) +def test_resolve_sts_region(env, aws_sts_endpoint, expected_region): + with patch.dict(os.environ, env, clear=True): + assert ( + BaseAWSLLM._resolve_sts_region(aws_sts_endpoint=aws_sts_endpoint) + == expected_region + ) + + +@pytest.mark.parametrize( + "env,aws_sts_endpoint,ssl_verify,expected", + [ + ({}, None, None, {"verify": True}), + ( + {"AWS_REGION": "us-east-1"}, + None, + None, + {"verify": True, "region_name": "us-east-1"}, + ), + ( + {}, + "https://sts.eu-west-1.amazonaws.com", + None, + { + "verify": True, + "endpoint_url": "https://sts.eu-west-1.amazonaws.com", + "region_name": "eu-west-1", + }, + ), + ( + {"AWS_REGION": "us-east-1"}, + "https://sts.eu-west-1.amazonaws.com", + None, + { + "verify": True, + "endpoint_url": "https://sts.eu-west-1.amazonaws.com", + "region_name": "eu-west-1", + }, + ), + ( + {}, + "https://sts.amazonaws.com", + None, + {"verify": True, "endpoint_url": "https://sts.amazonaws.com"}, + ), + ( + {}, + "https://vpce-abc.sts.eu-central-1.vpce.amazonaws.com", + None, + { + "verify": True, + "endpoint_url": "https://vpce-abc.sts.eu-central-1.vpce.amazonaws.com", + "region_name": "eu-central-1", + }, + ), + ({}, None, False, {"verify": False}), + ( + {"AWS_DEFAULT_REGION": "ap-southeast-1"}, + None, + None, + {"verify": True, "region_name": "ap-southeast-1"}, + ), + ], + ids=[ + "default_verify_only", + "env_region", + "endpoint_with_parsed_region", + "endpoint_parsed_over_env", + "global_endpoint_no_region", + "vpce_endpoint", + "ssl_verify_false", + "env_default_region", + ], +) +def test_build_sts_client_kwargs(env, aws_sts_endpoint, ssl_verify, expected): + base_aws_llm = BaseAWSLLM() + with patch.dict(os.environ, env, clear=True): + assert ( + base_aws_llm._build_sts_client_kwargs( + aws_sts_endpoint=aws_sts_endpoint, + ssl_verify=ssl_verify, + ) + == expected + ) + + +def test_irsa_cross_account_sts_client_uses_resolved_region(): + """IRSA cross-account path must use _build_sts_client_kwargs (env region, not Bedrock).""" + base_aws_llm = BaseAWSLLM() + import tempfile + + with tempfile.NamedTemporaryFile(mode="w", delete=False) as f: + f.write("test-web-identity-token") + token_file = f.name + + try: + with patch.dict( + os.environ, + { + "AWS_WEB_IDENTITY_TOKEN_FILE": token_file, + "AWS_ROLE_ARN": "arn:aws:iam::111111111111:role/eks-service-account-role", + "AWS_REGION": "eu-west-1", + }, + clear=True, + ): + mock_sts_client = MagicMock() + mock_sts_client.assume_role_with_web_identity.return_value = { + "Credentials": { + "AccessKeyId": "temp-key", + "SecretAccessKey": "temp-secret", + "SessionToken": "temp-token", + "Expiration": datetime.now(timezone.utc) + timedelta(hours=1), + } + } + mock_sts_client.assume_role.return_value = { + "Credentials": { + "AccessKeyId": "assumed-key", + "SecretAccessKey": "assumed-secret", + "SessionToken": "assumed-token", + "Expiration": datetime.now(timezone.utc) + timedelta(hours=1), + } + } + + with patch( + "boto3.client", return_value=mock_sts_client + ) as mock_boto3_client: + base_aws_llm._auth_with_aws_role( + aws_access_key_id=None, + aws_secret_access_key=None, + aws_session_token=None, + aws_role_name="arn:aws:iam::222222222222:role/target-role", + aws_session_name="test-session", + aws_region_name="eu-central-1", + ) + + for call in mock_boto3_client.call_args_list: + assert call.args == ("sts",) + assert call.kwargs["region_name"] == "eu-west-1" + assert call.kwargs["verify"] is True + finally: + os.unlink(token_file) + + +def test_web_identity_token_sts_client_uses_build_sts_client_kwargs(): + base_aws_llm = BaseAWSLLM() + mock_sts_client = MagicMock() + mock_sts_client.assume_role_with_web_identity.return_value = { + "Credentials": { + "AccessKeyId": "key", + "SecretAccessKey": "secret", + "SessionToken": "token", + "Expiration": datetime.now(timezone.utc) + timedelta(hours=1), + }, + "PackedPolicySize": 0, + } + + with patch.dict(os.environ, {"AWS_REGION": "eu-west-1"}, clear=True): + with patch("boto3.client", return_value=mock_sts_client) as mock_boto3_client: + with patch( + "litellm.llms.bedrock.base_aws_llm.get_secret", + return_value="oidc-token", + ): + base_aws_llm._auth_with_web_identity_token( + aws_web_identity_token="my-token", + aws_role_name="arn:aws:iam::111111111111:role/target", + aws_session_name="test-session", + aws_region_name="eu-central-1", + aws_sts_endpoint="https://sts.eu-west-1.amazonaws.com", + ) + + mock_boto3_client.assert_called_once_with( + "sts", + verify=True, + endpoint_url="https://sts.eu-west-1.amazonaws.com", + region_name="eu-west-1", + ) + + +def test_sts_uses_workload_region_not_bedrock_region(): + """Air-gapped: Bedrock in eu-central-1, STS VPC endpoint in eu-west-1 via AWS_REGION.""" + base_aws_llm = BaseAWSLLM() + mock_expiry = MagicMock() + mock_expiry.tzinfo = timezone.utc + time_diff = MagicMock() + time_diff.total_seconds.return_value = 3600 + mock_expiry.__sub__ = MagicMock(return_value=time_diff) + mock_sts_client = MagicMock() + mock_sts_client.assume_role.return_value = { + "Credentials": { + "AccessKeyId": "assumed-access-key", + "SecretAccessKey": "assumed-secret-key", + "SessionToken": "assumed-session-token", + "Expiration": mock_expiry, + } + } + + with patch.dict(os.environ, {"AWS_REGION": "eu-west-1"}, clear=True): + with patch("boto3.client", return_value=mock_sts_client) as mock_boto3_client: + base_aws_llm._auth_with_aws_role( + aws_access_key_id=None, + aws_secret_access_key=None, + aws_session_token=None, + aws_role_name="arn:aws:iam::2222222222222:role/LitellmEvalBedrockRole", + aws_session_name="test-session", + aws_region_name="eu-central-1", + ) + mock_boto3_client.assert_called_with( + "sts", + region_name="eu-west-1", + verify=True, + ) + + +def test_sts_endpoint_region_matches_bedrock_region_param(): + """aws_sts_endpoint signing region must not follow aws_region_name when they differ.""" + base_aws_llm = BaseAWSLLM() + mock_expiry = MagicMock() + mock_expiry.tzinfo = timezone.utc + time_diff = MagicMock() + time_diff.total_seconds.return_value = 3600 + mock_expiry.__sub__ = MagicMock(return_value=time_diff) + mock_sts_client = MagicMock() + mock_sts_client.assume_role.return_value = { + "Credentials": { + "AccessKeyId": "assumed-access-key", + "SecretAccessKey": "assumed-secret-key", + "SessionToken": "assumed-session-token", + "Expiration": mock_expiry, + } + } + + env_without_irsa = { + k: v + for k, v in os.environ.items() + if k + not in ( + "AWS_ROLE_ARN", + "AWS_WEB_IDENTITY_TOKEN_FILE", + "AWS_REGION", + "AWS_DEFAULT_REGION", + ) + } + with patch.dict(env_without_irsa, clear=True): + with patch("boto3.client", return_value=mock_sts_client) as mock_boto3_client: + base_aws_llm._auth_with_aws_role( + aws_access_key_id=None, + aws_secret_access_key=None, + aws_session_token=None, + aws_role_name="arn:aws:iam::2222222222222:role/LitellmEvalBedrockRole", + aws_session_name="test-session", + aws_region_name="eu-central-1", + aws_sts_endpoint="https://sts.eu-west-1.amazonaws.com", + ) + mock_boto3_client.assert_called_with( + "sts", + endpoint_url="https://sts.eu-west-1.amazonaws.com", + region_name="eu-west-1", + verify=True, + ) + + @pytest.mark.parametrize( "role_kwargs,expected_client_kwargs", [ @@ -940,7 +1254,6 @@ def test_eks_irsa_ambient_credentials_used(role_kwargs, expected_client_kwargs): ( {"aws_region_name": "us-east-1"}, { - "region_name": "us-east-1", "aws_access_key_id": "explicit-access-key", "aws_secret_access_key": "explicit-secret-key", "aws_session_token": "assumed-session-token", @@ -951,6 +1264,7 @@ def test_eks_irsa_ambient_credentials_used(role_kwargs, expected_client_kwargs): {"aws_sts_endpoint": "https://sts.eu-west-1.amazonaws.com"}, { "endpoint_url": "https://sts.eu-west-1.amazonaws.com", + "region_name": "eu-west-1", "aws_access_key_id": "explicit-access-key", "aws_secret_access_key": "explicit-secret-key", "aws_session_token": "assumed-session-token", @@ -958,7 +1272,7 @@ def test_eks_irsa_ambient_credentials_used(role_kwargs, expected_client_kwargs): }, ), ], - ids=["no_region_or_endpoint", "regional_sts", "explicit_sts_endpoint"], + ids=["no_region_or_endpoint", "bedrock_region_ignored_for_sts", "explicit_sts_endpoint"], ) def test_explicit_credentials_used_when_provided(role_kwargs, expected_client_kwargs): """ From 574ee7526db6808be3b2e3649da9a69bd2fb40b6 Mon Sep 17 00:00:00 2001 From: yuneng-jiang Date: Fri, 22 May 2026 15:57:29 -0700 Subject: [PATCH 11/13] test(streaming): tolerate Vertex 429 wrapped in MidStreamFallbackError (#28669) Streaming 429s are wrapped in MidStreamFallbackError so the Router can fall back; the existing 'except litellm.RateLimitError: pass' in test_vertex_ai_stream no longer matches, causing the generic pytest.fail branch to fire when upstream Vertex returns 429. Add a sibling except for MidStreamFallbackError that only swallows it when e.original_exception is a RateLimitError, so unrelated streaming failures still fail the test. --- tests/local_testing/test_streaming.py | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/tests/local_testing/test_streaming.py b/tests/local_testing/test_streaming.py index b1a93c380b2..10f351714e1 100644 --- a/tests/local_testing/test_streaming.py +++ b/tests/local_testing/test_streaming.py @@ -993,6 +993,11 @@ def test_vertex_ai_stream(provider): except litellm.RateLimitError as e: pass + except litellm.exceptions.MidStreamFallbackError as e: + # Streaming 429s are wrapped in MidStreamFallbackError so the + # Router can fall back; treat as a transient rate-limit pass. + if not isinstance(e.original_exception, litellm.RateLimitError): + pytest.fail(f"Error occurred: {e}") except Exception as e: pytest.fail(f"Error occurred: {e}") From f35e7eb2f6ac0ac84b3b470cadeb3fc5b0b379a3 Mon Sep 17 00:00:00 2001 From: Sameer Kankute Date: Sat, 23 May 2026 04:29:04 +0530 Subject: [PATCH 12/13] feat(guardrails): add Microsoft Purview DLP guardrail (#24966) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * feat(guardrails): add Microsoft Purview DLP guardrail * fix(guardrails/purview): raise_for_status on HTTP errors, cap scope cache, reuse executor * fix(guardrails/purview): propagate litellm_call_id as correlation_id to Purview * chore: fixes * refactor(guardrails): delegate get_user_prompt to get_last_user_message PurviewGuardrailBase duplicated AzureGuardrailBase (and OpenAIGuardrailBase) user-prompt extraction. The same logic already lived in common_utils.get_last_user_message; wire guardrail bases to that helper, fix the helper docstring, and drop its redundant self-import of convert_content_list_to_str. Co-authored-by: Sameer Kankute * fix(purview): make protection scope cache true LRU on hits OrderedDict.get() does not update insertion order; call move_to_end on TTL-valid cache hits so popitem(last=False) evicts least-recently-used users instead of FIFO by first insert. Add a regression test with a small max cache size. Co-authored-by: Sameer Kankute * Fix mypy * fix(guardrails/purview): harden user-id resolution and broaden DLP text Prefer API key and proxy-injected metadata over client metadata for Entra identity. Scan full message transcript pre-call and all completion choices post-call. Align logging-only hook with the same user-id rules. Co-authored-by: Cursor * fix(guardrails/purview): scan /v1/completions prompt and TextChoices Normalize text-completion prompts (string or list of strings); skip token-id-only prompts. Run post-call DLP on TextCompletionResponse choices. Extend logging_only hook for text_completion. Add tests and completion_prompt_to_str helper. Co-authored-by: Cursor * fix(purview-dlp): return data after DLP pass; per-call executor; dedupe text extraction async_pre_call_hook now returns the request dict after a successful check so callers match skip-path behavior. logging_hook uses a fresh ThreadPoolExecutor per invocation like Presidio to avoid single-worker starvation. Response text extraction is centralized in _completion_response_text_parts. Co-authored-by: Sameer Kankute * fix(purview): fix LRU cache refresh position and add Responses API scanning Two fixes to the Microsoft Purview DLP guardrail: 1. LRU cache bug (base.py): When a stale scope cache entry was re-fetched, the assignment updated the value but Python's OrderedDict.__setitem__ preserves the original insertion order for existing keys. This left the refreshed entry near the front of the dict, making it the first candidate for LRU eviction via popitem(last=False). Fix: call move_to_end(user_id) after every write to an existing key. 2. Responses API coverage gap (purview_dlp.py): Requests to /v1/responses use an 'input' field instead of 'messages' or 'prompt', so the pre-call hook returned without scanning the content. Similarly, post-call hook did not handle ResponsesAPIResponse.output. Fix: add _responses_api_input_to_str() helper and handle 'responses'/'aresponses' call types in async_pre_call_hook, async_post_call_success_hook (via _completion_response_text_parts), and async_logging_hook. Co-authored-by: Sameer Kankute * fix(purview): message separator, non-blocking logging_hook, TextChoices type error Three bugs fixed in the Microsoft Purview DLP guardrail: 1. get_prompt_text_for_dlp message separator (base.py) - Previously called get_str_from_messages() which concatenated all message texts with NO separator, so 'end of msg1' + 'start of msg2' became 'end of msg1start of msg2'. - Now joins per-message text with '\n\n' via convert_content_list_to_str(), preserving DLP pattern detection accuracy across message boundaries. 2. logging_hook blocking the event loop thread (purview_dlp.py) - Previously called future.result() which blocked the calling thread (often the event loop thread) for the entire round-trip of two sequential Microsoft Graph API calls (_compute_protection_scopes + _process_content). - Now fires and forgets: when called inside a running loop, schedules the coroutine with loop.create_task(); otherwise spawns a daemon thread. Returns (kwargs, result) immediately in both cases. - Removes unused concurrent.futures.ThreadPoolExecutor import; adds threading. 3. Incompatible assignment type error (purview_dlp.py:180) - mypy inferred 'choice' as TextChoices from the first loop body, then flagged the assignment in the second loop as incompatible with Choices. - Fixed by using distinct loop variable names: text_choice (TextChoices) and chat_choice (Choices). Tests: 7 new tests added covering the separator fix (TestGetPromptTextForDlp) and the non-blocking logging_hook (TestLoggingHookNonBlocking). Co-authored-by: Sameer Kankute * fix(purview): suppress API errors in logging-only mode and scan tool-call arguments Three issues fixed: 1. _check_content except block re-raised unconditionally even when block_on_violation=False. The docstring promised 'log only - do not raise' but network/API errors always propagated. Fixed by checking block_on_violation before re-raising; when False, log a warning and continue. 2. async_logging_hook used a single try/except wrapping both the prompt and response audit calls. When the first _check_content (uploadText) raised due to an API error the second call (downloadText) was silently skipped. Fixed by giving each audit call its own try/except so both always run independently. 3. convert_content_list_to_str() only reads message.content, so tool_calls[].function.arguments and function_call.arguments were invisible to the Purview pre-call and post-call scans. An authenticated caller could embed sensitive text in tool-call arguments and bypass DLP. Fixed by: - Adding PurviewGuardrailBase._extract_tool_call_args_from_message() which handles both dict and object-style messages, covering both tool_calls[] arrays and the legacy function_call field. - Updating get_prompt_text_for_dlp() to include those arguments alongside message content (request/prompt path). - Changing _completion_response_text_parts() from @staticmethod to an instance method and adding tool-call argument extraction for ModelResponse choices (response path). Co-authored-by: Sameer Kankute * chore(ui): restructure pre-built Next.js output to directory-based routing Flat page files (e.g. guardrails.html) replaced by directory-based index.html equivalents (e.g. guardrails/index.html) matching the Next.js App Router output format. Co-authored-by: Sameer Kankute * fix(purview): comprehensive security hardening — identity spoofing, streaming bypass, token-id gap Four security issues addressed: 1. end_user_id kwargs fallback missing in _resolve_user_id_from_logging_kwargs user_id already fell back to kwargs.get("user_api_key_user_id") when absent from metadata, but end_user_id only checked md.get("user_api_key_end_user_id") with no kwargs-level fallback. Added or kwargs.get("user_api_key_end_user_id"). 2. Streaming responses bypassed post_call blocking async_post_call_success_hook only runs on assembled non-streaming responses. For streaming requests the proxy already delivered all content before the hook ran, so raising HTTPException there had no effect. Added async_post_call_streaming_iterator_hook which buffers the entire stream, assembles it via stream_chunk_builder, runs the Purview DLP check, and only then re-yields chunks via MockResponseIterator. If a violation is detected the exception is raised before any bytes reach the client. The proxy automatically skips async_post_call_success_hook for guardrails that define this method, preventing duplicate scans. 3. Caller-controlled Purview user identity in blocking modes When a LiteLLM API key has no bound user_id the guardrail fell back to metadata[user_id_field], which is supplied by the caller. A caller could set this to any Entra object ID whose Purview policies are more permissive and bypass DLP. Added _resolve_trusted_user_id() that only returns identities from the proxy auth system (user_api_key_dict.user_id, end_user_id, or proxy-injected metadata["user_api_key_user_id"]). Added _resolve_user_id_for_blocking() used by all blocking-mode hooks: tries trusted sources first; if only caller-supplied is available, logs a SECURITY WARNING and still proceeds (backward compat); if nothing resolves, skips with a warning. 4. Token-id prompt DLP bypass When /v1/completions received a pure token-id array prompt, completion_prompt_to_str() returned None and the pre_call hook silently skipped the Purview scan. An authenticated caller could tokenize blocked text and send it without DLP evaluation. The hook now detects this case (raw_prompt present but prompt_text None) and logs a WARNING while letting the request pass through — token-id payloads are opaque at the text layer and cannot be scanned. This makes the gap explicit rather than silent. Tests: 94 total, all passing. Co-authored-by: Sameer Kankute * Revert "chore(ui): restructure pre-built Next.js output to directory-based routing" This reverts commit c70c4303b735bb3885732bd4a0e01997e9571f56. * fix(purview): fail closed on identity spoofing, token prompts, and path encoding Encode Entra user IDs in Graph paths, guard caches with asyncio.Lock, scan Responses API instructions with string input, reject caller-only metadata and token-id completion prompts in blocking mode, and revert unrelated UI HTML restructure from the PR branch. Co-authored-by: Cursor * fix(purview): use threading.Lock and getattr for LitellmParams - Replace asyncio.Lock with threading.Lock in PurviewGuardrailBase. The cache lock is acquired both from the proxy's main event loop and from short-lived event loops created by the logging_hook thread fallback. In Python 3.10+ an asyncio.Lock is bound to the first event loop that acquires it, so the second loop would silently break audit logging with RuntimeError. All critical sections are in-memory dict ops with no awaits, so a synchronous lock is safe. - Use getattr() on LitellmParams in initialize_guardrail() instead of .get(), which does not exist on Pydantic BaseModel instances and would raise AttributeError at runtime. Tests updated to construct Mock objects with spec= so they reflect the real interface. Co-authored-by: Yassin Kortam * refactor(purview): dedupe trust-level user resolution and drop dead code - _resolve_user_id now delegates levels 1-3 to _resolve_trusted_user_id so blocking and non-blocking paths share a single source of truth. - Drop redundant event_hook override in MicrosoftPurviewDLPGuardrail.__init__ (initialize_guardrail already forwards event_hook=litellm_params.mode). - Drop unused self._logging_only attribute; blocking is controlled by the block_on_violation argument passed to _check_content. Co-authored-by: Yassin Kortam * fix(purview): fail-closed on responses API transform error; avoid duplicate audit calls Co-authored-by: Yassin Kortam * fix(purview): fail-closed blocking DLP; revert directory-based UI HTML Blocking hooks now require UserAPIKeyAuth user_id/end_user_id only (no spoofable metadata), re-raise Responses API transform errors, scan streamed text completions, and reject requests with no bound identity. Reverts the accidental directory-based Next.js output from cc47081 (c70c4303b7). Co-authored-by: Cursor * Remove dead code in purview_dlp: _resolve_user_id_for_blocking never returns falsy The method either returns a non-empty trusted user id or raises HTTPException, so the 'if not user_id' guards in async_pre_call_hook and async_post_call_success_hook were unreachable. Tighten the return type to str and drop the dead checks to make the fail-closed behavior explicit. Co-authored-by: Yassin Kortam * fix(purview): exclude caller-controlled end_user_id from blocking DLP Blocking Purview checks now use only API-key/JWT-bound user_id, not end_user_id populated from request user/metadata/safety_identifier. Co-authored-by: Cursor * style(purview): apply Black formatting to base.py Co-authored-by: Cursor * fix(purview): use post-await timestamp for cache TTL Capture the timestamp after the network call completes when storing it as the cache freshness marker, so the effective TTL reflects when the response was actually received rather than when the request started. Under high network latency the previous behavior shortened the effective cache lifetime. Co-authored-by: Yassin Kortam * fix(purview_dlp): fail closed when stream_chunk_builder returns None stream_chunk_builder can return None (e.g., when ChunkProcessor filters all chunks), causing both isinstance checks to fail and the buffered chunks to be released without DLP scanning. Explicitly fail closed in that case by raising an HTTPException so the streaming DLP guardrail does not bypass policy enforcement. Co-authored-by: Yassin Kortam * fix(purview_dlp): resolve user_id before buffering stream Co-authored-by: Yassin Kortam * merge main (#28629) * test(vcr): classify cache verdicts, detect live calls, surface cost leaks Convert the per-test VCR verdict line from a single 'NOOP / HIT / MISS / PARTIAL' tag into a classified outcome that distinguishes the cases that silently bill the live API on every CI run from the ones that don't: HIT pure replay PARTIAL mixed replay + new recordings MISS:RECORDED new cassette saved to Redis (cached next run) MISS:OVERFLOW cassette > MAX_EPISODES_PER_CASSETTE; persister refused to save; re-bills every run MISS:NOT_PERSISTED test failed; save_cassette skipped; re-bills NOOP VCR-marked but no HTTP traffic (mocked elsewhere) UNMARKED:LIVE_CALL test bypassed VCR AND opened a TCP connection to a known LLM provider host -> wasted spend UNMARKED:NO_TRAFFIC test bypassed VCR but didn't call out The UNMARKED:LIVE_CALL signal is what converts 'this test probably hits live' into 'this test connected to api.openai.com'. We install a socket.connect / socket.create_connection wrapper for the duration of each non-VCR-marked test and record any outbound TCP to a known LLM provider hostname. The probe sits below the httpx layer so vcrpy and respx (which both patch above the socket) are unaffected. Replace the file-level _RESPX_CONFLICTING_FILES blacklists in the llm_translation and local_testing conftests with per-item respx detection in apply_vcr_auto_marker_to_items. A test now skips VCR when it actually carries @pytest.mark.respx or has respx_mock in its fixture chain - not just because some other test in the same file imports MockRouter. Items skipped by skip_files are split into respx_conflict (real conflict, the module wires up respx) vs file_opt_out (dead skip- list entry whose module never touches respx) so the session summary makes pruning obvious. Stabilize the AWS SigV4 fingerprint: the Authorization header on Bedrock requests rotates its Credential date and Signature on every call, which previously pushed every Bedrock test past the 50-episode overflow threshold. Extract the access-key id only ('aws-sigv4:AKIA...') so two requests with the same identity match. Always emit verdict logging when VCR is active (set LITELLM_VCR_VERBOSE=0 to opt back into the legacy quiet mode). Add a session-end classification summary that lists overflow tests, unmarked live-call tests, and the skip-reason breakdown. Wire the live-call probe + summary hook into every test directory that already uses the Redis-backed VCR cache (audio_tests, guardrails_tests, image_gen_tests, litellm_utils_tests, llm_responses_api_testing, llm_translation, local_testing, logging_callback_tests, ocr_tests, pass_through_unit_tests, router_unit_tests, search_tests, unified_google_tests). Add tests/llm_translation/test_vcr_classification.py covering the verdict classifier, skip-reason tagging, AWS SigV4 fingerprint stability, live-host classification, and session summary rendering. Co-authored-by: Mateo Wang * test(vcr): drop dead 'from respx import MockRouter' imports These seven test files were on _RESPX_CONFLICTING_FILES, which made the auto-marker skip them entirely. Inspecting the source shows the only respx artifact is a top-level 'from respx import MockRouter' that no test ever uses - no @pytest.mark.respx, no respx_mock fixture, no respx.mock context manager. The import is dead code left over from a previous mocking pattern. Now that apply_vcr_auto_marker_to_items detects respx per-item via the marker / fixture chain (b637d9f64a), the file-level skip is no longer needed for these files - they were the reason the OpenAI tests (test_o3_reasoning_effort, test_streaming_response[o1/o3-mini], TestOpenAIO1::test_streaming, TestOpenAIChatCompletion::test_web_search, TestOpenAIO3::test_web_search, etc.) ran live every CI build despite the cassette cache being healthy. Co-authored-by: Mateo Wang * test(image_edits): regenerate fixtures per call instead of holding open module-level file handles Module-level TEST_IMAGES = [ open(os.path.join(pwd, 'ishaan_github.png'), 'rb'), open(os.path.join(pwd, 'litellm_site.png'), 'rb'), ] SINGLE_TEST_IMAGE = open(...) opens the file once at import. After the first multipart upload, the file pointer is at EOF, so every subsequent test in the same xdist worker sends an empty multipart body. That non-determinism (a) blows the recorded cassette past MAX_EPISODES_PER_CASSETTE (50) so _RedisPersister.save_cassette refuses to save it, and (b) re-bills the live image edit endpoint on every CI run. Recent CI runs confirm the leak: tests/image_gen_tests/test_image_edits.py shows six tests parking at 51-52 cassette entries (TestOpenAIImageEditGPTImage1::test_openai_image_edit_litellm_sdk[False], TestOpenAIImageEditDallE2::..., test_openai_image_edit_with_bytesio, test_openai_image_edit_litellm_router, test_multiple_vs_single_image_edit[False], test_multiple_image_edit_with_different_formats). Replace the module-level file handles with _make_test_images() / _make_single_test_image() factories that return fresh _RewindableImage (BytesIO subclass) objects whose pointer always starts at 0. The image bytes are read once at import into module-level constants (_ISHAAN_GITHUB_BYTES, _LITELLM_SITE_BYTES), so disk I/O cost is unchanged. Co-authored-by: Mateo Wang * fix(vcr): match real Bedrock hostnames in live-call probe The suffix '.bedrock-runtime.amazonaws.com' never matched real Bedrock endpoints, which use the format 'bedrock-runtime[-fips].{region}.amazonaws.com' (region between 'bedrock-runtime' and 'amazonaws.com'). Add an explicit host check for that pattern so Bedrock live calls are visible to the probe, and update the unit test accordingly. Also drop the unused '_LIVE_CALL_PROBE_INSTALLED' module variable. * fix(vcr): cover full RFC1918 172.16.0.0/12 range in local prefixes * fix(image_edits): drop _RewindableImage to prevent infinite multipart upload The _RewindableImage(BytesIO) wrapper auto-rewound on every read after EOF, which made the OpenAI SDK's multipart upload writer read the same bytes forever instead of seeing EOF. Workers OOM'd / SIGKILL'd: [gw0] node down: Not properly terminated replacing crashed worker gw0 ... worker 'gw1' crashed while running 'tests/image_gen_tests/test_image_edits.py::TestOpenAIImageEditGPTImage1::test_openai_image_edit_litellm_sdk[False]' The auto-rewind was added defensively for parametrized + flaky-retried tests, but BaseLLMImageEditTest::test_openai_image_edit_litellm_sdk already calls get_base_image_edit_call_args() once per invocation and that helper now constructs fresh streams via _make_test_images(), so rewinding inside the stream is unnecessary. Replace with plain BytesIO seeded with the cached image bytes. Co-authored-by: Mateo Wang * test(vcr): mark Bedrock prompt-caching cross-call tests VCR-incompatible The pass_through prompt-caching tests (test_prompt_caching_returns_cache_read_tokens_on_second_call, test_prompt_caching_streaming_second_call_returns_cache_read) make a warm-up call and then assert the *second* call sees a non-zero cache_read_input_tokens count from the upstream's prompt-cache. VCR replay can't model cross-call provider state — both calls match the same cassette episode, so the second call returns the first call's pre-warmup response and the assertion fails: AssertionError: Expected cache_read_input_tokens > 0 on second call, but got 0. Full usage: {'input_tokens': 4986, 'cache_creation_input_tokens': 4974, 'cache_read_input_tokens': 0} This started biting after the AWS SigV4 fingerprint stabilization (b637d9f64a): Bedrock requests now produce a stable per-access-key fingerprint instead of a per-request signature, so cassettes successfully replay where they previously always missed and re-recorded live. Opt these tests out via skip_nodeid_suffixes so they run live and match the existing pattern in tests/llm_translation/conftest.py (::test_prompt_caching). Co-authored-by: Mateo Wang * test(vcr): tighten OVERFLOW classification and switch respx detection to AST Address two greptile P2 review concerns on PR #27795: 1. MISS:OVERFLOW was firing whenever total > MAX_EPISODES_PER_CASSETTE regardless of cassette state. A cassette that grew past the cap historically but this run only *replayed* (dirty=False) is healthy — the persister never tries to save, so the cache state is stable and the next run will replay too. Only flag OVERFLOW when dirty=True (new episodes were recorded that the persister would refuse to save). Add a regression test covering the dirty=False + large-total case. 2. _module_uses_respx did substring matching on the module source, which false-positives on comments / docstrings / string literals. A comment like # Previously tried respx.mock but switched to vcrpy would keep a file pinned on the opt-out list, defeating the dead-import pruning goal of this PR. Replace the substring scan with an ast.NodeVisitor (_RespxUsageVisitor) that only counts: - @pytest.mark.respx / @respx.mock decorators - with respx.mock(): ... (sync + async) context managers - respx.mock(...) calls outside a with/decorator - function parameters / fixture names equal to respx_mock Add tests for the comment / docstring / string-literal cases plus each real-usage pattern. Co-authored-by: Mateo Wang * fix(vcr): aggregate worker stats on the controller so the session summary actually renders under xdist `_session_stats` is a module-level dict mutated inside `_vcr_outcome_gate` — which runs in each xdist worker process. The controller's `pytest_terminal_summary` then reads its own empty `_session_stats` and bails on `if not counts: return`, so the OVERFLOW / LIVE_CALL sections the rest of this PR adds never make it into CI logs in the dist mode CI actually uses. Ship a structured `vcr_outcome` payload via `user_properties` (which xdist round-trips) and add `aggregate_report_outcome` on the controller to fold worker outcomes into `_session_stats`. The recording process tags `vcr_recorded_by` with `PYTEST_XDIST_WORKER` so the controller can tell "single-process — already counted locally" apart from "produced by a worker — needs aggregation here", and not double-count when there's no xdist. Covered by 9 new unit tests in test_vcr_classification.py including the end-to-end summary render path. * fix(guardrails): improve CrowdStrike AIDR input handling (#26658) * feat(lasso): add tool-calling support to LassoGuardrail (#27648) * feat(lasso): extend LassoGuardrail to support tool calling (RND-5748) * fix(lasso): PR review followups for tool-calling guardrail (RND-5748) * fix(lasso): handle object-style tool_calls in _update_tool_calls_from_masked (RND-5748) * fix(lasso): use model role for tool_use blocks (RND-5748) * test(lasso): add round-trip tests for message transformation (RND-5748) * fix(lasso): remove unused imports, handle Responses-API input masking, flatten multimodal content (RND-5748) * fix(lasso): inspect Responses-API input field (RND-5748) * fix(lasso): guard text-cursor remap against Lasso count mismatch (RND-5748) * fix(lasso): flatten list content in tool_result.content (RND-5748) * fix(lasso): remap multimodal list content during masking (RND-5748) Bug: _map_masked_messages_back counted list-content messages in original_text_count but the remap loop only handled isinstance(str). The positional text_cursor never advanced for list messages, causing all subsequent masked texts to be written onto the wrong messages. Fix: added elif isinstance(content, list) branch that replaces the list with the masked text string and advances the cursor — mirrors the existing string-content branch. Also handles the assistant + tool_calls combo for list-content messages. Test: test_map_masked_messages_back_list_content verifies a user message with [text + image_url] followed by an assistant message gets correct masked content on both (cursor stays aligned). * refactor(lasso): extract _get_field and _extract_tool_call_fields helpers (RND-5748) The dict-vs-object access pattern (x.get('y') if isinstance(x, dict) else getattr(x, 'y', None)) was duplicated 14 times across 5 methods. _get_field(obj, field) — single-point dict/Pydantic field access. _extract_tool_call_fields(call) — returns (call_id, name, parsed_input) with JSON argument parsing, replacing ~30 duplicate lines in both async_post_call_success_hook and _expand_messages_for_classification. Also simplified _update_tool_calls_from_masked, _prepare_payload tool mapping, and _apply_masking_to_model_response call_id extraction. Net ~60 lines removed. No behavior change — all 32 tests pass. * fix(lasso): add count guard to _apply_masking_to_model_response (RND-5748) _apply_masking_to_model_response used a bare text_cursor without verifying 1:1 correspondence between text-bearing choices and masked text entries. If Lasso returned a different number of text messages than choices with content, masked text would be applied to the wrong choice or silently skip choices. Added the same count-mismatch guard pattern already used in _map_masked_messages_back: count original text-bearing choices, compare to masked_text length, skip text remap on mismatch with a warning log. Tool_call masking via id-based lookup is unaffected. Tests: - test_apply_masking_to_model_response_multiple_choices: verifies correct per-choice masked text with 2 choices - test_apply_masking_to_model_response_count_mismatch: verifies content is left unchanged when counts disagree * fix(lasso): close two guardrail-bypass paths flagged in review (RND-5748) * tool-call args: when function.arguments is malformed JSON or parses to a non-object, preserve the raw string as {"arguments": } so Lasso still inspects it instead of receiving input=None. Covers both pre-call and post-call extraction (shared helper). Also resolves the CodeQL empty-except warning since the except body now assigns parsed=None. * Responses-API input: when a request carries both "messages" and "input", inspect both. Previously a benign messages array let the guardrail skip data["input"] entirely. The masking write-back is split via a count boundary so masked messages flow back to data["messages"] and masked input flows back to data["input"] without cross-contamination. Tests: malformed/non-object args round-trip, dual-field classification, dual-field masking write-back split. * chore(lasso): black formatting + comment on expand skip branch (RND-5748) * black: wrap two long expressions in lasso.py and reformat dict literals in test_lasso.py to satisfy CI lint. * add a short comment in _expand_messages_for_classification explaining why empty string and None content are intentionally skipped (None is the OpenAI shape for a pure tool-call turn). * fix(lasso): satisfy mypy in _handle_masking, _update_tool_calls_from_masked, _apply_masking_to_model_response (RND-5748) * Narrow `response.get("messages")` into a local before slicing so mypy doesn't see `Optional[List[Dict[str, str]]]` as non-indexable. * Rename the two write-side `func` bindings in `_update_tool_calls_from_masked` to `func_dict` / `func_obj` so mypy doesn't unify the dict and Any|None branches. * Rename the inner loop variable in `_apply_masking_to_model_response` from `msg` to `masked_msg` to avoid clashing with the `msg = choice.message` rebinding below. No behavior change; resolves the 7 mypy errors from the CI lint job. * perf: eliminate per-request callback scanning on proxy hot path (#27858) - Introduce `_CallbackCapabilities` dataclass and `ProxyLogging._callback_capabilities()` static method that inspects `litellm.callbacks` once and caches capability flags keyed on (list length, member ids); invalidates automatically when the callback list mutates without per-request iteration overhead - Replace O(n) `litellm.callbacks` walks in `async_pre_call_hook`, `during_call_hook`, `async_post_call_streaming_iterator_hook`, `async_post_call_streaming_hook`, and `post_call_response_headers_hook` with fast-path exits when no relevant callbacks are registered - Add `needs_iterator_wrap()` and `needs_per_chunk_streaming_hook()` instance methods to decouple iterator-level wrapping from per-chunk hook execution; avoids `get_response_string` materialization per chunk when no guardrail or chunk-hook callback is active - Introduce `_fast_serialize_simple_model_response_stream()` using `orjson` for common single-choice text streaming chunks, bypassing the full Pydantic serializer; falls back to `model_dump_json` for tool calls, logprobs, usage, and provider-specific fields - Add early-return in `_restamp_streaming_chunk_model` when downstream model already matches the requested model, avoiding unnecessary string comparisons on every chunk - Fix stale zero-cost cache bug in `_is_model_cost_zero`: move the per-router `_zero_cost_cache` dict onto the `Router` instance and clear it in `_invalidate_model_group_info_cache` so in-place pricing updates via `upsert_deployment` immediately resume budget enforcement - Add `scripts/benchmark_chat_completions_perf.py`: standalone async benchmarking tool with a mock OpenAI provider, LiteLLM proxy process management, non-streaming RPS, streaming TTFT, and full-stream latency measurements with repeat/median run support - Add comprehensive unit tests covering capability detection, cache invalidation, fast-path correctness, zero-cost cache regression, and the no-callback streaming fast path Co-authored-by: Yassin Kortam * ci(mutmut): enable mutate_only_covered_lines to fit in CI budget (#27910) The mutation-test workflow timed out at the 350-minute job cap when running whole-folder mutation against litellm/proxy/management_endpoints/ (~30 files, ~1.5 MB of source). Every mutant was running the full test suite, and mutants were generated for lines no test covers — which would survive regardless, just wasting compute. mutmut 3.x's mutate_only_covered_lines setting runs the suite once up front to compute coverage, then skips mutating uncovered lines. This cuts the mutant count dramatically and is the right semantic for the score (no test → no kill possible → uncountable). Per-mutant test filtering by function name is already automatic in mutmut 3.x; no external coverage step is needed. * fix(rate-limit): stop v3 limiter from leaking internal stash to provider body (#27913) * fix(rate-limit): stop v3 limiter from leaking internal stash to provider body PR #27001 (atomic TPM rate limit) introduced a reservation flow that writes four LiteLLM-internal keys onto the request data dict: _litellm_rate_limit_descriptors _litellm_tpm_reserved_tokens _litellm_tpm_reserved_model _litellm_tpm_reserved_scopes _litellm_tpm_reservation_released These keys are forwarded as request body params to the upstream provider, which rejects them as unknown fields: OpenAI -> 400 'Unknown parameter: _litellm_rate_limit_descriptors' (mapped by litellm to RateLimitError / 429, hiding the bug behind a misleading 'throttling_error' code) Anthropic -> 400 '_litellm_rate_limit_descriptors: Extra inputs are not permitted' Net effect: every chat completion against any real provider fails the moment a virtual key has any tpm_limit / rpm_limit set — i.e. v3-enforced key-level TPM/RPM limits are broken end-to-end. The v3 RPM/TPM check itself still runs (raises 429 on over-limit), but the success path poisons the upstream body. Reproduced on litellm_internal_staging HEAD (410ce761dc) against gpt-4o-mini and claude-haiku-4-5 with a 1-RPM/1-TPM key — first request fails with the provider's unknown-field error. Fix: the stash is metadata only. - Add RATE_LIMIT_DESCRIPTORS_KEY constant and a _LITELLM_STASH_KEYS registry so we have a single source of truth for stash keys. - New helper _stash_value_in_metadata_channels writes to data['metadata'] / data['litellm_metadata'] without touching the top level. - _stash_reservation_in_data and the descriptor stash now route through that helper. _mark_reservation_released stops writing top-level. - _lookup_stashed_value also checks kwargs['metadata'] / kwargs['litellm_metadata'] (raw request_data shape) in addition to kwargs['litellm_params']['metadata'] (completion kwargs shape). - async_post_call_failure_hook now reads descriptors via the unified metadata lookup instead of request_data.get(top-level). - Defense in depth: async_pre_call_hook strips any stash key that somehow surfaced at the top level (stale cache, future refactor, test fixture) before returning. Tests: - New regression test asserts no _litellm_* stash key is present at the top level of data after async_pre_call_hook, and that the metadata channel still carries the reservation + descriptors so success / failure reconciliation works. - Existing test_tpm_concurrent.py tests that asserted top-level presence are updated to read from data['metadata'] — the location is an implementation detail; the spec is that post-call callbacks can resolve the stash. Verified end-to-end against OpenAI gpt-4o-mini and Anthropic claude-haiku-4-5 via /v1/chat/completions on a low-rpm key: - With limits not exceeded: HTTP 200, valid completion response, no leaked fields in body. - With RPM exceeded: HTTP 429 from v3 enforcement ('Rate limit exceeded ... Limit type: requests'). - With TPM exceeded: HTTP 429 from v3 enforcement ('Rate limit exceeded ... Limit type: tokens'). Full v3 hook test suite passes (171 tests). Co-authored-by: Mateo Wang * chore(rate-limit): use RATE_LIMIT_DESCRIPTORS_KEY constant in test, trim noisy comments Address greptile P2: test fixture now uses the imported constant. Drop comments that re-explain what well-named identifiers already convey. * fix(rate-limit): reject caller-supplied stash values to prevent TPM-refund abuse Strip _LITELLM_STASH_KEYS from data top-level and both metadata channels at the start of async_pre_call_hook. Without this, an authenticated caller can inject _litellm_rate_limit_descriptors plus _litellm_tpm_reserved_tokens in body metadata, trigger a proxy-side rejection, and cause async_post_call_failure_hook to refund TPM counters against attacker-named scopes (e.g. another tenant's api_key). --------- Co-authored-by: Cursor Agent Co-authored-by: Mateo Wang * fix: allow for allowlisted redirect URIs (#27761) * fix: allow for allowlisted redirect URIs * github comment addressing * Update litellm/proxy/_experimental/mcp_server/oauth_utils.py Co-authored-by: veria-ai[bot] <224490171+veria-ai[bot]@users.noreply.github.com> * harden oauth wildcard further * test: cover wildcard entry with dot-leading suffix rejection --------- Co-authored-by: veria-ai[bot] <224490171+veria-ai[bot]@users.noreply.github.com> * Emit native web_search_tool_result blocks for Anthropic clients (Claude Desktop / Cowork citations) (#27886) * feat(custom_logger): add async_post_agentic_loop_response_hook Lets a CustomLogger shape the response returned by the agentic-loop follow-up call without bypassing the loop's safety / observability machinery (depth tracking, fingerprinting, etc.). Default returns the response unchanged. Used by websearch_interception to inject Anthropic-native web_search_tool_result blocks when the originating client requested a native web_search_* tool. * feat(llm_http_handler): call post-agentic-loop hook on the originating callback In _execute_anthropic_agentic_plan, after anthropic_messages.acreate returns, call the originating callback's async_post_agentic_loop_response_hook so it can mutate the final response (e.g. inject native tool_result blocks). Pass the callback through from _call_agentic_completion_hooks. Exceptions in the post-hook are caught and logged so a buggy callback can't kill the request. * feat(websearch_interception): add is_anthropic_native_web_search_tool Identifies tools the Anthropic-native clients (Claude Desktop, the Anthropic SDK, the Anthropic Console) use to request native search: type starts with "web_search_" (e.g. web_search_20250305). Rejects the LiteLLM standard tool, the OpenAI-function variant, the bare "WebSearch" legacy name, and the bare "web_search" Claude Code shape. This lets us decide per-request whether the client expects web_search_tool_result content blocks in the response, without renaming any existing constants or touching native-provider skip logic. * feat(websearch_interception): add build_web_search_tool_result_block Produces the Anthropic-native web_search_tool_result content block from a structured SearchResponse. Anthropic-native clients use this block to populate citations / source links — the existing text-blob flatten path only feeds readable evidence to the model and discards the structure, so this builder gives us the missing piece. Shape matches https://docs.anthropic.com/en/api/web-search-tool — web_search_result items carry url, title, page_age, encrypted_content (empty string when the search provider doesn't supply one). * feat(websearch_interception): emit native web_search_tool_result blocks When the originating client request carried a native Anthropic web_search_* tool, the final response now also carries web_search_tool_result content blocks alongside the model's text answer — so Claude Desktop / Anthropic SDK clients can populate the citations panel and replay conversation history with structured search evidence. Wiring: - Pre-request hooks (both deployment + Anthropic path) set a flag on kwargs when they see a native web_search_* tool, so the signal survives the conversion-to-litellm_web_search step regardless of which hook fires first. - _execute_search now returns (text, SearchResponse) so the structured results aren't lost when the text is flattened for the follow-up model call. - _build_anthropic_request_patch returns the parallel list of SearchResponse objects. - async_build_agentic_loop_plan pre-builds the web_search_tool_result blocks (one per tool_use_id) and stashes them on plan.metadata when the flag is set. - async_post_agentic_loop_response_hook reads the metadata and prepends the blocks to response.content. - _execute_agentic_loop mirrors the injection for the legacy path so both paths behave identically. Clients that send the LiteLLM standard tool keep the existing text-only behavior — no regression. * test(websearch_interception): cover native web_search_tool_result emission 18 tests across: - detector branches (native vs litellm-standard, OpenAI-function shape, Claude Desktop builtin WebSearch, bare web_search, missing type) - block-builder shape (results, none, empty) - pre-request hook flag-setting (native sets, standard does not) - async_build_agentic_loop_plan attaches blocks to plan.metadata when the flag is present, leaves metadata untouched when absent - post-hook injection into dict and object responses - legacy _execute_agentic_loop mirrors the injection so both paths return the same shape * test(websearch_short_circuit): keep _execute_search mocks in sync with new tuple return * test(websearch_thinking_constraint): keep _execute_search mocks in sync with new tuple return * feat(websearch_interception): emit native blocks from try_short_circuit_search The agentic-loop post-hook only fires when the model returns a tool_use block. Cowork / Claude Desktop on Bedrock actually make TWO requests per user turn: the main /v1/messages with their builtin tool, and a separate standalone /v1/messages whose only tool is web_search_20250305. That second request hits try_short_circuit_search — no agentic loop, no post-hook — and was returning text-only, leaving the citations panel empty. When the short-circuit input carries a native web_search_* tool, build a synthetic server_tool_use + web_search_tool_result pair (using the structured SearchResponse already returned by _execute_search) so the client gets the native shape it expects. The legacy text block is preserved so non-native short-circuit callers (Claude Code, github_copilot, etc.) see the same payload as before. Failure path still emits the native block pair (with empty results) plus the text-error block, so the client gets a well-formed response rather than a malformed half-shape. * test(websearch_native_blocks): cover short-circuit native-block emission Three new cases on top of the existing 18: - native web_search_20250305 short-circuit → [server_tool_use, web_search_tool_result, text], ids paired, urls/titles carried. - litellm_web_search short-circuit → text-only (no regression). - native short-circuit on search failure → still emits the native block pair (empty results) plus the text-error block, so the client never sees a malformed half-shape. * test(websearch_short_circuit): index assertions by block type, not by position Native short-circuit responses now have [server_tool_use, web_search_tool_result, text] when the input carries web_search_20250305 — find the text block by type rather than relying on content[0]. * fix(websearch_interception): gate legacy WebSearch name on schema absence Clients like Cowork / Claude Desktop ship a client-side tool named "WebSearch" with a full input_schema — they handle it themselves and expect to make a separate native web_search_20250305 sub-request for the actual search. Today is_web_search_tool matches the bare name regardless of other fields, which hijacks the client's tool server-side. The agentic loop fires on the main request, the model never gets to emit the client-side tool_use, and the separate native sub-request (where citation data flows) is never made. Net: citations panel empty. Real Anthropic client tools always carry input_schema (the API rejects them otherwise), so a bare {name: "WebSearch"} with no schema is the only thing that could be a legacy interception marker. Gate the match on schema absence: legacy callers (if any) keep working, real client-side WebSearch tools pass through untouched. * fix(websearch_interception): drop "WebSearch" from response-detection lists Post-conversion the model always sees ``litellm_web_search``, so the "WebSearch" entry in the response-side tool_use detection lists was dead at best. If a model ever did return ``tool_use(name="WebSearch")`` it would now (incorrectly) hijack the client's own ``WebSearch`` tool again — same Cowork problem we just fixed on the input side. Drop it. * test(websearch_native_blocks): cover the WebSearch legacy-name schema gate Three new cases: - {name: "WebSearch"} (bare interception marker) → still matched - {name: "WebSearch", input_schema: {...}} (Cowork client tool) → passes through untouched - {name: "WebSearch", description: "..."} (no schema) → still matched on the assumption it's a legacy marker rather than a malformed real client tool. --------- Co-authored-by: Ishaan Jaffer * ci(codecov): restore litellm/ prefix on uploaded coverage paths pytest-cov runs with --cov=litellm, which makes coverage.xml store paths relative to the package root (e.g. `proxy/proxy_server.py` instead of `litellm/proxy/proxy_server.py`). Codecov auto-resolves these only when the basename is unique in the repo. Files like proxy_server.py, router.py, utils.py, main.py, and constants.py — which have duplicates under enterprise/ or other subpackages — get silently dropped during ingest. The `fixes: ["::litellm/"]` rule prepends `litellm/` to every uploaded path so they resolve unambiguously. Confirmed against multiple recent coverage.xml artifacts that no uploader currently emits paths already prefixed with `litellm/`, so the rule is safe to apply universally. This restores Codecov visibility for the highest-fix-rate hotspots: proxy_server.py, router.py, proxy/utils.py, litellm_logging.py, constants.py, key_management_endpoints.py, utils.py, main.py, user_api_key_auth.py, team_endpoints.py, and litellm_pre_call_utils.py. * chore(ci): remove unused GitHub Actions workflows and orphan files Audit of .github/workflows/ via gh run history shows the following have either never run or have been dormant for 10+ weeks. CI coverage that still matters is preserved on CircleCI (e.g. llm_translation_testing). Removed workflows: - test-litellm.yml — workflow_dispatch only, last run 2026-02-12 (cancelled); CCI local_testing_part1/2 covers the same tests - llm-translation-testing.yml — last run 2025-07-10; replaced by CCI llm_translation_testing job (run_llm_translation_tests.py kept for the make test-llm-translation target) - run_observatory_tests.yml — last run 2026-03-03 (cancelled) - scan_duplicate_issues.yml — last run 2026-03-02 (failure) - publish_to_pypi.yml — never run - read_pyproject_version.yml — fires on every push to main but its echoed version output is not consumed by any downstream step Removed orphan files (no callers in workflows, CCI, or Makefile): - .github/workflows/README.md — documented only publish_to_pypi.yml - .github/workflows/update_release.py + results_stats.csv - .github/actions/helm-oci-chart-releaser/ * Revert "ci(codecov): restore litellm/ prefix on uploaded coverage paths" This reverts commit e25a988a3feb4a31843a67274a3a64fea2fed805. The `fixes: ["::litellm/"]` rule turned out to be applied *after* Codecov's auto-resolution, not before. Files with unique basenames (which were auto-resolving correctly to `litellm/`) got an extra `litellm/` prepended, producing `litellm/litellm/` storage. Files with ambiguous basenames (the actual target of the fix) continued to be dropped because the auto-resolution still failed for them. Net result on the verification run: 1375 files now stored under unresolvable `litellm/litellm/...` paths, and the 11 originally-missing hotspots are still missing. Reverting before piling on further changes. * test(ui): preserve global Button/Tooltip mocks in per-file @tremor/react vi.mock Per-file `vi.mock("@tremor/react", ...)` factories fully replace the setup-level mock from `tests/setupTests.ts`, so the global Button/Tooltip overrides are lost in any file that re-mocks `@tremor/react`. Without them, the real Tremor `