* fix aviation safety topic filter: remove overly broad exceptions, add cockpit access block words
* fix airline brand protection filter: add identifier words, competitor/ops block words, tighten exceptions
* update policy templates with competitor pre/post guardrails and streaming enrichment
* sync policy_templates_backup.json with policy_templates.json
* add streaming enrichment endpoint, competitor variations, and model selection
* add streaming enrichPolicyTemplate networking function
* rewrite template parameter modal with streaming tags, AI/manual toggle, model selector
* update policies panel to pass enrichment options and show discovered competitors
* show discovered competitors as tags in guardrail selection modal
* use llm_router instead of litellm.acompletion, extract helpers, move constants
* validate competitors list size, cap variation prompt input
* add refinement instruction support for competitor discovery
* add instruction and existingCompetitors params to streaming enrichment
* add refinement input for iterating on competitor list with AI
* emit status events during variation generation so UI shows progress
* add onStatus callback to streaming enrichment
* show status spinner during variation generation, widen modal to 700px
* add tests for competitor enrichment helper functions
Add a new team member permission `/team/daily/activity` that allows
non-admin team members to see all team usage data, not just their own.
## Changes
- Add `TEAM_DAILY_ACTIVITY` to `KeyManagementRoutes` enum and available
team member permissions
- Add `_team_member_has_permission` helper in `common_utils.py`
- Modify `/team/daily/activity` endpoint to skip API key filtering when
the member has this permission
- Add permission description and method detection in the UI
- Add backend and frontend tests
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Four tests were broken by commit e00c181f0c (Mcp user permissions #21462):
1. test_list_tools_single_server_unprefixed_names: The commit changed
_get_tools_from_mcp_servers to always add server prefixes (add_prefix=True),
removing the conditional that skipped prefixing for single servers.
Updated assertion from "toolA" → "zapier-toolA".
2. test_mcp_get_prompt_success: mcp_get_prompt now extracts the server name
from a prefixed prompt name via split_server_prefix_from_name(). Passing
unprefixed "hello" returns server_name="" which matches no server → 403.
Updated call to use "server_a-hello" so the server lookup succeeds.
3. test_e2e_jwt_team_mcp_permissions_enforced &
4. test_e2e_jwt_team_mcp_key_intersection:
The commit replaced `from typing import List` with
`from litellm.proxy.proxy_server import general_settings` in
MCPRequestHandler.get_allowed_mcp_servers(). Both tests mock
litellm.proxy.proxy_server with a types.ModuleType that lacked
general_settings, causing ImportError. Added general_settings={} to
both mock modules.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Cast `guardrail` to `Dict[str, Any]` in `mcp_end_user_permission/__init__.py`
to fix "No overload variant of 'get' matches argument type 'str'" — the
`Guardrail` TypedDict's `.get()` overloads resolve the key type to `Never`
when chained with a `{}` default, making further `.get()` calls fail.
- Cast `fn` to `Dict[str, Any]` in `transformation.py`
`transform_chat_completion_tool_params_to_responses_api_tools` to fix
'"object" has no attribute "get"' — `tool.get("function")` on
`Union[ChatCompletionFunctionToolParam, OpenAIMcpServerTool]` returns
`object` because `"function"` is not a key in `OpenAIMcpServerTool`.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
test_complete_reload_flow and test_distributed_reload_check_function both
trigger code paths that assign a minimal stub dict to litellm.model_cost
(via the /reload/model_cost_map endpoint and _check_and_reload_model_cost_map).
Without restoring, subsequent tests in the same worker can't find gpt-4o
pricing and calculate spend=0.0 instead of the expected value.
Added try/finally save-and-restore of litellm.model_cost in both tests,
matching the pattern used in test_reload_model_cost_map_admin_access.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(schema.prisma): add object permissions for end users
allows controlling if end user can call specific mcp servers
* feat: cleanup for customer_endpoints support of object permission id
* fix: cleanup str
* feat(customers/): enforce end user can only call allowed mcps - if configured
* docs: document customer/end user object permission usage
* feat: enforce end user permissions on MCP tool calls
This commit implements end user permission enforcement for MCP servers:
1. Always add server prefixes to MCP tool names
- Removed conditional logic that only added prefixes when multiple servers existed
- Now always adds server prefix for consistent tool naming across all scenarios
- Updated 5 locations in server.py (list_tools, get_prompts, get_resources,
get_resource_templates, get_prompt)
2. Created MCP End User Permission Guardrail Hook
- New guardrail hook: litellm/proxy/guardrails/guardrail_hooks/mcp_end_user_permission.py
- Runs on post_call to validate tool calls in LLM responses
- Extracts MCP server name from tool names (splits on first '-')
- Checks if end_user_id has permissions for the MCP server
- Raises GuardrailRaisedException if end user lacks permission
- Supports both streaming and non-streaming responses
3. Added comprehensive tests
- Test file: tests/test_litellm/proxy/guardrails/guardrail_hooks/test_mcp_end_user_permission.py
- Tests cover: authorized/unauthorized tools, non-MCP tools, no end_user scenarios
- Tests permission checking logic and exception raising
The hook integrates with the existing MCPRequestHandler._get_allowed_mcp_servers_for_end_user
to fetch end user permissions and enforce access control at the response level.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* refactor: remove redundant add_prefix variable assignments
Simplified the code by removing intermediate `add_prefix` variable
assignments and passing `True` directly to function calls since
we now always add server prefixes.
Changes:
- Removed `add_prefix = True` variable assignments in 5 locations
- Changed `add_prefix=add_prefix` to `add_prefix=True` in function calls
- Added inline comments to clarify the behavior
This makes the code more concise and clearer in intent.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* feat(auth_utils.py): support safety_identifier as a valid way of passing the end user id for responses api
* feat(llms): ensure 'tools' is correctly updated for responses api
* fix: fix greptile feedback
* feat: transformation.py
proper responses api tool handling for guardrail translation layer
---------
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
* feat(ui/): initial commit adding a compliance testing playground
allow proxy admins to test policies and guardrails against datasets
* feat(ui/): make score more friendly
* feat(policy_endpoints.py): new helper function for testing policies
* feat(policy_endpoints.py): expose new endpoint for testing policies and guardrails
enables compliance playground to work as expected
* feat(complianceui.tsx): show returned text
* docs: add DATABRICKS_API_KEY to environment settings reference
* fix: streaming test usage check on Pydantic model
* fix: mock litellm.proxy.proxy_server in test_skip_server_startup
Users had to set store_model_in_db in the config YAML and restart the proxy,
causing service downtime. This change allows the value to be written to the
LiteLLM_Config table and read from the database at runtime, with DB values
overriding config file values.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Running `cd enterprise && poetry run pip install -e .` causes poetry to
create a separate venv in `enterprise/.venv` (since enterprise/ has its
own pyproject.toml). The main project's tests run with `.venv/bin/python`,
so the enterprise package installed in `enterprise/.venv` is never seen.
Fix: run `poetry run pip install -e enterprise/` from the repo root so
poetry uses the main project's venv. This ensures litellm_enterprise is
importable when tests run.
This explains why enterprise tests kept failing with:
AttributeError: '_PROXY_LiteLLMManagedFiles' object has no attribute
'_check_file_deletion_allowed'
even after --force-reinstall was added — the reinstall was going to the
wrong virtual environment.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(middleware): replace BaseHTTPMiddleware with pure ASGI middleware
BaseHTTPMiddleware wraps streaming responses with receive_or_disconnect
per chunk, blocking the event loop and causing severe throughput
degradation under concurrent streaming load (53% of CPU in profiling).
Converts PrometheusAuthMiddleware to a pure ASGI middleware using the
__call__(scope, receive, send) protocol.
* fix(streaming): remove expensive debug logging and optimize usage stripping
- Remove print_verbose calls that format chunk/response Pydantic objects,
triggering millions of __repr__ calls (8% of CPU in profiling)
- Guard remaining verbose_logger.debug with isEnabledFor(DEBUG) and use
lazy %s formatting instead of f-strings
- Replace usage stripping round-trip (model_dump + delete + reconstruct)
with a _usage_stripped flag, deferring exclusion to serialization time
* fix(proxy): remove per-chunk debug log and use _usage_stripped flag
- Remove verbose_proxy_logger.debug that formatted every streaming chunk
- Honor _usage_stripped flag from streaming handler to exclude usage
during model_dump_json serialization instead of reconstructing objects
* fix(proxy): remove per-chunk debug log in async_data_generator
Remove verbose_proxy_logger.debug that formatted every streaming chunk,
which triggered expensive Pydantic serialization on the hot path.
* fix indentation and add clarifying comment for usage stripping
* fix: guard calculate_total_usage against None usage in chunks
* fix: store chunk copy to preserve usage for calculate_total_usage
When `default_internal_user_params` was set, `insert_sso_user()` only
preserved SSO-provided roles if `role_mappings` was explicitly configured.
Roles from other valid SSO sources (Microsoft app_roles,
GENERIC_USER_ROLE_ATTRIBUTE, custom SSO handlers) were silently
overwritten with the default "internal_user" role, causing admin users
to be downgraded on first login or after user deletion.
Replace the `role_mappings_configured` gate with `_should_use_role_from_sso_response()`
which validates the role is a recognized LitellmUserRoles value regardless
of origin. Also removes an unnecessary DB round-trip to litellm_ssoconfig
on every new SSO user creation.
Fixes: admin users seeing internal-user UI after SSO login
Fixes: test_get_redirect_url_for_sso flaking due to local env vars
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When models are created via UI with pre-existing credentials, they use
litellm_credential_name to reference credentials stored separately.
The batch file upload endpoint uses get_deployment_credentials_with_provider()
to retrieve model credentials, but this method was returning the credential
reference name instead of resolving it to actual API keys and endpoints.
This fix adds credential resolution using CredentialAccessor.get_credential_values()
to ensure UI-created models work the same as YAML-configured models for batch
file uploads and other passthrough endpoints.
Fixes: Batch file uploads failing with 'Missing credentials' error for UI models
In Pydantic v2, `Optional[str]` without a default value is a required field
(callers must explicitly pass `None`). This caused `ValidationError` when
constructing `LiteLLM_ManagedFileTable` and `LiteLLM_ManagedVectorStoreTable`
without providing `created_by`/`updated_by`.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
