* feat(ui/): initial commit adding a compliance testing playground
allow proxy admins to test policies and guardrails against datasets
* feat(ui/): make score more friendly
* feat(policy_endpoints.py): new helper function for testing policies
* feat(policy_endpoints.py): expose new endpoint for testing policies and guardrails
enables compliance playground to work as expected
* feat(complianceui.tsx): show returned text
* docs: add DATABRICKS_API_KEY to environment settings reference
* fix: streaming test usage check on Pydantic model
* fix: mock litellm.proxy.proxy_server in test_skip_server_startup
Users had to set store_model_in_db in the config YAML and restart the proxy,
causing service downtime. This change allows the value to be written to the
LiteLLM_Config table and read from the database at runtime, with DB values
overriding config file values.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Running `cd enterprise && poetry run pip install -e .` causes poetry to
create a separate venv in `enterprise/.venv` (since enterprise/ has its
own pyproject.toml). The main project's tests run with `.venv/bin/python`,
so the enterprise package installed in `enterprise/.venv` is never seen.
Fix: run `poetry run pip install -e enterprise/` from the repo root so
poetry uses the main project's venv. This ensures litellm_enterprise is
importable when tests run.
This explains why enterprise tests kept failing with:
AttributeError: '_PROXY_LiteLLMManagedFiles' object has no attribute
'_check_file_deletion_allowed'
even after --force-reinstall was added — the reinstall was going to the
wrong virtual environment.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(middleware): replace BaseHTTPMiddleware with pure ASGI middleware
BaseHTTPMiddleware wraps streaming responses with receive_or_disconnect
per chunk, blocking the event loop and causing severe throughput
degradation under concurrent streaming load (53% of CPU in profiling).
Converts PrometheusAuthMiddleware to a pure ASGI middleware using the
__call__(scope, receive, send) protocol.
* fix(streaming): remove expensive debug logging and optimize usage stripping
- Remove print_verbose calls that format chunk/response Pydantic objects,
triggering millions of __repr__ calls (8% of CPU in profiling)
- Guard remaining verbose_logger.debug with isEnabledFor(DEBUG) and use
lazy %s formatting instead of f-strings
- Replace usage stripping round-trip (model_dump + delete + reconstruct)
with a _usage_stripped flag, deferring exclusion to serialization time
* fix(proxy): remove per-chunk debug log and use _usage_stripped flag
- Remove verbose_proxy_logger.debug that formatted every streaming chunk
- Honor _usage_stripped flag from streaming handler to exclude usage
during model_dump_json serialization instead of reconstructing objects
* fix(proxy): remove per-chunk debug log in async_data_generator
Remove verbose_proxy_logger.debug that formatted every streaming chunk,
which triggered expensive Pydantic serialization on the hot path.
* fix indentation and add clarifying comment for usage stripping
* fix: guard calculate_total_usage against None usage in chunks
* fix: store chunk copy to preserve usage for calculate_total_usage
When `default_internal_user_params` was set, `insert_sso_user()` only
preserved SSO-provided roles if `role_mappings` was explicitly configured.
Roles from other valid SSO sources (Microsoft app_roles,
GENERIC_USER_ROLE_ATTRIBUTE, custom SSO handlers) were silently
overwritten with the default "internal_user" role, causing admin users
to be downgraded on first login or after user deletion.
Replace the `role_mappings_configured` gate with `_should_use_role_from_sso_response()`
which validates the role is a recognized LitellmUserRoles value regardless
of origin. Also removes an unnecessary DB round-trip to litellm_ssoconfig
on every new SSO user creation.
Fixes: admin users seeing internal-user UI after SSO login
Fixes: test_get_redirect_url_for_sso flaking due to local env vars
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When models are created via UI with pre-existing credentials, they use
litellm_credential_name to reference credentials stored separately.
The batch file upload endpoint uses get_deployment_credentials_with_provider()
to retrieve model credentials, but this method was returning the credential
reference name instead of resolving it to actual API keys and endpoints.
This fix adds credential resolution using CredentialAccessor.get_credential_values()
to ensure UI-created models work the same as YAML-configured models for batch
file uploads and other passthrough endpoints.
Fixes: Batch file uploads failing with 'Missing credentials' error for UI models
In Pydantic v2, `Optional[str]` without a default value is a required field
(callers must explicitly pass `None`). This caused `ValidationError` when
constructing `LiteLLM_ManagedFileTable` and `LiteLLM_ManagedVectorStoreTable`
without providing `created_by`/`updated_by`.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Save and restore litellm.initialized_langfuse_clients around
test_max_langfuse_clients_limit to prevent ordering-dependent failures
in other tests that rely on the counter's value.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The same PyPI-override issue existed in test-litellm.yml, test-mcp.yml,
and .circleci/config.yml. Also adds --no-deps (enterprise has no runtime
deps) to avoid redundant dependency resolution on every forced reinstall.
Addresses greptile review comments on PR #21481.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- test_litellm_pre_call_utils.py: wrap test body in try/finally so
litellm.callbacks is always restored even when an assertion fails,
addressing greptile review comment
- test_langfuse_otel.py: resolve trivial merge conflict in comment
("unpatched" vs "unpatch-ed"), keeping correct spelling
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
test_reload_model_cost_map_admin_access calls the /reload/model_cost_map
HTTP endpoint with get_model_cost_map mocked to return a single-entry
dict. The endpoint handler does a direct module-level assignment
(litellm.model_cost = new_model_cost_map) which persists after the
patch context manager exits, stripping all models except gpt-3.5-turbo
from the in-memory cost map and causing subsequent tests that rely on
models like gemini-1.5-flash, multimodalembedding@001, and gpt-4o to
fail with "model not mapped" errors or zero-cost spend payloads.
Fix: save litellm.model_cost before the test and restore it (along with
invalidating the case-insensitive lookup cache) in a finally block.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The ChatGPT backend API sends non-spec-compliant streaming tool call chunks
where index is always 0 for parallel tool calls and id/name get repeated in
duplicate closing chunks. Add ChatGPTToolCallNormalizer to fix indices and
filter duplicates before they reach the consumer.
Fixes#21482
* fix: guard against None metadata in prometheus metrics
Use get_litellm_metadata_from_kwargs and get_metadata_variable_name_from_kwargs
helpers to properly resolve metadata from both 'metadata' and 'litellm_metadata'
keys, with None safety.
* test: add test for None metadata in prometheus metrics
Update provider matching so github/<model> aliases can resolve capabilities from existing upstream model metadata, including OpenAI and Anthropic entries. Add regression tests for known github aliases and unknown-model fallback behavior.
The adapter was injecting `summary: "detailed"` into the reasoning config
when routing Anthropic thinking requests to OpenAI's Responses API.
Per the OpenAI spec, reasoning.summary is opt-in — it should not be
added unless the user explicitly requests it.
Three test isolation issues fixed:
1. test_mcp_debug.py: Replace deprecated asyncio.get_event_loop().run_until_complete()
with asyncio.run() in TestWrapSendWithDebugHeaders. In Python 3.10+,
get_event_loop() raises RuntimeError when no event loop is set in the
current thread, causing test_injects_headers and test_body_messages_unchanged
to fail in isolation.
2. test_mcp_server_manager.py: After _reload_mcp_manager_module() creates a new
global_mcp_server_manager instance, server.py still holds a stale reference
to the old instance. Tests in test_mcp_server.py that populate the new
manager's registry and then call server.py functions (e.g. _get_tools_from_mcp_servers)
get empty results because server.py reads from the old manager. Fix: update
server.py's module-level reference after each reload.
3. test_litellm_pre_call_utils.py: test_add_litellm_metadata_from_request_headers
sets litellm.callbacks without restoring it afterward. Add cleanup to restore
original callbacks after the test to prevent state leaking to subsequent tests.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Four finally blocks in test_internal_user_endpoints.py and one in
test_ui_sso.py used the pattern:
if original_default_params is not None:
litellm.default_internal_user_params = original_default_params
else:
delattr(litellm, "default_internal_user_params")
Since the attribute is defined in litellm/__init__.py with a default of
None, `getattr(litellm, "default_internal_user_params", None)` returns
None. The else branch then calls delattr(), permanently removing the
attribute from the module for the rest of the process.
Subsequent tests in the same pytest-xdist worker (e.g.
test_add_new_member_* in test_management_helpers_utils.py) then fail
with: AttributeError: module 'litellm' has no attribute
'default_internal_user_params'
Fix: replace all five flawed finally blocks with a simple assignment:
litellm.default_internal_user_params = original_default_params
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
poetry install includes litellm-enterprise from PyPI, then the editable
install step runs. When the same version is already installed, pip may
skip the editable install leaving the PyPI build in place - which may
lack methods added after the latest PyPI release. Adding
--force-reinstall ensures the local editable version always wins.
Fixes enterprise tests failing with AttributeError on methods that exist
locally but not in the cached PyPI-installed package.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
LogRecord objects do not have a .message attribute by default — it is
only populated after the record has been formatted by a Formatter.
The correct way to retrieve the formatted log message is getMessage().
This fixes AttributeError: 'LogRecord' object has no attribute 'message'
in test_cost_calculation_uses_debug_level and
test_batch_cost_calculation_uses_debug_level.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
