Add a prominent warning banner to the UI dashboard when detailed debug
mode (LITELLM_LOG=DEBUG) is enabled. This alerts users to significant
performance degradation caused by extensive diagnostic logging.
Backend changes:
- Enhanced /health/readiness endpoint to include log_level and
is_detailed_debug fields
- Added detection using verbose_logger.getEffectiveLevel()
- Backward compatible - old clients ignore new fields
Frontend changes:
- Updated useHealthReadiness TypeScript interface
- Created DebugWarningBanner component using Ant Design Alert
- Integrated banner into dashboard layout below navbar
- Banner only shows when DEBUG level is active
- Non-dismissible to ensure users are aware of performance impact
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat(schema.prisma): add object permissions for end users
allows controlling if end user can call specific mcp servers
* feat: cleanup for customer_endpoints support of object permission id
* fix: cleanup str
* feat(customers/): enforce end user can only call allowed mcps - if configured
* docs: document customer/end user object permission usage
* feat: enforce end user permissions on MCP tool calls
This commit implements end user permission enforcement for MCP servers:
1. Always add server prefixes to MCP tool names
- Removed conditional logic that only added prefixes when multiple servers existed
- Now always adds server prefix for consistent tool naming across all scenarios
- Updated 5 locations in server.py (list_tools, get_prompts, get_resources,
get_resource_templates, get_prompt)
2. Created MCP End User Permission Guardrail Hook
- New guardrail hook: litellm/proxy/guardrails/guardrail_hooks/mcp_end_user_permission.py
- Runs on post_call to validate tool calls in LLM responses
- Extracts MCP server name from tool names (splits on first '-')
- Checks if end_user_id has permissions for the MCP server
- Raises GuardrailRaisedException if end user lacks permission
- Supports both streaming and non-streaming responses
3. Added comprehensive tests
- Test file: tests/test_litellm/proxy/guardrails/guardrail_hooks/test_mcp_end_user_permission.py
- Tests cover: authorized/unauthorized tools, non-MCP tools, no end_user scenarios
- Tests permission checking logic and exception raising
The hook integrates with the existing MCPRequestHandler._get_allowed_mcp_servers_for_end_user
to fetch end user permissions and enforce access control at the response level.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* refactor: remove redundant add_prefix variable assignments
Simplified the code by removing intermediate `add_prefix` variable
assignments and passing `True` directly to function calls since
we now always add server prefixes.
Changes:
- Removed `add_prefix = True` variable assignments in 5 locations
- Changed `add_prefix=add_prefix` to `add_prefix=True` in function calls
- Added inline comments to clarify the behavior
This makes the code more concise and clearer in intent.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* feat(auth_utils.py): support safety_identifier as a valid way of passing the end user id for responses api
* feat(llms): ensure 'tools' is correctly updated for responses api
* fix: fix greptile feedback
* feat: transformation.py
proper responses api tool handling for guardrail translation layer
---------
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
* feat(ui/): initial commit adding a compliance testing playground
allow proxy admins to test policies and guardrails against datasets
* feat(ui/): make score more friendly
* feat(policy_endpoints.py): new helper function for testing policies
* feat(policy_endpoints.py): expose new endpoint for testing policies and guardrails
enables compliance playground to work as expected
* feat(complianceui.tsx): show returned text
* docs: add DATABRICKS_API_KEY to environment settings reference
* fix: streaming test usage check on Pydantic model
* fix: mock litellm.proxy.proxy_server in test_skip_server_startup
Users had to set store_model_in_db in the config YAML and restart the proxy,
causing service downtime. This change allows the value to be written to the
LiteLLM_Config table and read from the database at runtime, with DB values
overriding config file values.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Running `cd enterprise && poetry run pip install -e .` causes poetry to
create a separate venv in `enterprise/.venv` (since enterprise/ has its
own pyproject.toml). The main project's tests run with `.venv/bin/python`,
so the enterprise package installed in `enterprise/.venv` is never seen.
Fix: run `poetry run pip install -e enterprise/` from the repo root so
poetry uses the main project's venv. This ensures litellm_enterprise is
importable when tests run.
This explains why enterprise tests kept failing with:
AttributeError: '_PROXY_LiteLLMManagedFiles' object has no attribute
'_check_file_deletion_allowed'
even after --force-reinstall was added — the reinstall was going to the
wrong virtual environment.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(middleware): replace BaseHTTPMiddleware with pure ASGI middleware
BaseHTTPMiddleware wraps streaming responses with receive_or_disconnect
per chunk, blocking the event loop and causing severe throughput
degradation under concurrent streaming load (53% of CPU in profiling).
Converts PrometheusAuthMiddleware to a pure ASGI middleware using the
__call__(scope, receive, send) protocol.
* fix(streaming): remove expensive debug logging and optimize usage stripping
- Remove print_verbose calls that format chunk/response Pydantic objects,
triggering millions of __repr__ calls (8% of CPU in profiling)
- Guard remaining verbose_logger.debug with isEnabledFor(DEBUG) and use
lazy %s formatting instead of f-strings
- Replace usage stripping round-trip (model_dump + delete + reconstruct)
with a _usage_stripped flag, deferring exclusion to serialization time
* fix(proxy): remove per-chunk debug log and use _usage_stripped flag
- Remove verbose_proxy_logger.debug that formatted every streaming chunk
- Honor _usage_stripped flag from streaming handler to exclude usage
during model_dump_json serialization instead of reconstructing objects
* fix(proxy): remove per-chunk debug log in async_data_generator
Remove verbose_proxy_logger.debug that formatted every streaming chunk,
which triggered expensive Pydantic serialization on the hot path.
* fix indentation and add clarifying comment for usage stripping
* fix: guard calculate_total_usage against None usage in chunks
* fix: store chunk copy to preserve usage for calculate_total_usage
When `default_internal_user_params` was set, `insert_sso_user()` only
preserved SSO-provided roles if `role_mappings` was explicitly configured.
Roles from other valid SSO sources (Microsoft app_roles,
GENERIC_USER_ROLE_ATTRIBUTE, custom SSO handlers) were silently
overwritten with the default "internal_user" role, causing admin users
to be downgraded on first login or after user deletion.
Replace the `role_mappings_configured` gate with `_should_use_role_from_sso_response()`
which validates the role is a recognized LitellmUserRoles value regardless
of origin. Also removes an unnecessary DB round-trip to litellm_ssoconfig
on every new SSO user creation.
Fixes: admin users seeing internal-user UI after SSO login
Fixes: test_get_redirect_url_for_sso flaking due to local env vars
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The same PyPI-override issue existed in test-litellm.yml, test-mcp.yml,
and .circleci/config.yml. Also adds --no-deps (enterprise has no runtime
deps) to avoid redundant dependency resolution on every forced reinstall.
Addresses greptile review comments on PR #21481.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- test_litellm_pre_call_utils.py: wrap test body in try/finally so
litellm.callbacks is always restored even when an assertion fails,
addressing greptile review comment
- test_langfuse_otel.py: resolve trivial merge conflict in comment
("unpatched" vs "unpatch-ed"), keeping correct spelling
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
test_reload_model_cost_map_admin_access calls the /reload/model_cost_map
HTTP endpoint with get_model_cost_map mocked to return a single-entry
dict. The endpoint handler does a direct module-level assignment
(litellm.model_cost = new_model_cost_map) which persists after the
patch context manager exits, stripping all models except gpt-3.5-turbo
from the in-memory cost map and causing subsequent tests that rely on
models like gemini-1.5-flash, multimodalembedding@001, and gpt-4o to
fail with "model not mapped" errors or zero-cost spend payloads.
Fix: save litellm.model_cost before the test and restore it (along with
invalidating the case-insensitive lookup cache) in a finally block.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: guard against None metadata in prometheus metrics
Use get_litellm_metadata_from_kwargs and get_metadata_variable_name_from_kwargs
helpers to properly resolve metadata from both 'metadata' and 'litellm_metadata'
keys, with None safety.
* test: add test for None metadata in prometheus metrics
Three test isolation issues fixed:
1. test_mcp_debug.py: Replace deprecated asyncio.get_event_loop().run_until_complete()
with asyncio.run() in TestWrapSendWithDebugHeaders. In Python 3.10+,
get_event_loop() raises RuntimeError when no event loop is set in the
current thread, causing test_injects_headers and test_body_messages_unchanged
to fail in isolation.
2. test_mcp_server_manager.py: After _reload_mcp_manager_module() creates a new
global_mcp_server_manager instance, server.py still holds a stale reference
to the old instance. Tests in test_mcp_server.py that populate the new
manager's registry and then call server.py functions (e.g. _get_tools_from_mcp_servers)
get empty results because server.py reads from the old manager. Fix: update
server.py's module-level reference after each reload.
3. test_litellm_pre_call_utils.py: test_add_litellm_metadata_from_request_headers
sets litellm.callbacks without restoring it afterward. Add cleanup to restore
original callbacks after the test to prevent state leaking to subsequent tests.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Four finally blocks in test_internal_user_endpoints.py and one in
test_ui_sso.py used the pattern:
if original_default_params is not None:
litellm.default_internal_user_params = original_default_params
else:
delattr(litellm, "default_internal_user_params")
Since the attribute is defined in litellm/__init__.py with a default of
None, `getattr(litellm, "default_internal_user_params", None)` returns
None. The else branch then calls delattr(), permanently removing the
attribute from the module for the rest of the process.
Subsequent tests in the same pytest-xdist worker (e.g.
test_add_new_member_* in test_management_helpers_utils.py) then fail
with: AttributeError: module 'litellm' has no attribute
'default_internal_user_params'
Fix: replace all five flawed finally blocks with a simple assignment:
litellm.default_internal_user_params = original_default_params
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
poetry install includes litellm-enterprise from PyPI, then the editable
install step runs. When the same version is already installed, pip may
skip the editable install leaving the PyPI build in place - which may
lack methods added after the latest PyPI release. Adding
--force-reinstall ensures the local editable version always wins.
Fixes enterprise tests failing with AttributeError on methods that exist
locally but not in the cached PyPI-installed package.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Prisma's create_many() requires JSON fields to be wrapped in prisma.Json()
not passed as raw JSON strings. Lines 3099-3100 were using safe_dumps()
(which returns str) instead of prisma.Json(), causing Prisma validation
errors during master key rotation.
This is consistent with the existing pattern in the same file (line 3134
already uses prisma.Json for litellm_config env vars).
The regression test test_rotate_master_key_model_data_valid_for_prisma
was already correctly asserting isinstance(..., prisma.Json) — the test
exposed the mismatch.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The file had two unresolved git merge conflict markers from a merge of
litellm_oss_staging_02_17_2026 into main, causing a SyntaxError when
pytest tried to collect the test module.
Kept the instance-level mocking approach (from litellm_oss_staging) for
test_get_complete_url and test_validate_environment, which is consistent
with the rest of the file and avoids class-reference issues caused by
importlib.reload(litellm) in conftest.py.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The model_prices_and_context_window_backup.json file has 'inference_geo'
fields (e.g. on 'us/claude-sonnet-4-6') for geo-prefixed Anthropic models
used in cost calculation, but the JSON schema validator in test_utils.py
did not include 'inference_geo' as an allowed property.
This caused test_aaamodel_prices_and_context_window_json_is_valid to fail
with: Additional properties are not allowed ('inference_geo' was unexpected)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
