Users had to set store_model_in_db in the config YAML and restart the proxy,
causing service downtime. This change allows the value to be written to the
LiteLLM_Config table and read from the database at runtime, with DB values
overriding config file values.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(middleware): replace BaseHTTPMiddleware with pure ASGI middleware
BaseHTTPMiddleware wraps streaming responses with receive_or_disconnect
per chunk, blocking the event loop and causing severe throughput
degradation under concurrent streaming load (53% of CPU in profiling).
Converts PrometheusAuthMiddleware to a pure ASGI middleware using the
__call__(scope, receive, send) protocol.
* fix(streaming): remove expensive debug logging and optimize usage stripping
- Remove print_verbose calls that format chunk/response Pydantic objects,
triggering millions of __repr__ calls (8% of CPU in profiling)
- Guard remaining verbose_logger.debug with isEnabledFor(DEBUG) and use
lazy %s formatting instead of f-strings
- Replace usage stripping round-trip (model_dump + delete + reconstruct)
with a _usage_stripped flag, deferring exclusion to serialization time
* fix(proxy): remove per-chunk debug log and use _usage_stripped flag
- Remove verbose_proxy_logger.debug that formatted every streaming chunk
- Honor _usage_stripped flag from streaming handler to exclude usage
during model_dump_json serialization instead of reconstructing objects
* fix(proxy): remove per-chunk debug log in async_data_generator
Remove verbose_proxy_logger.debug that formatted every streaming chunk,
which triggered expensive Pydantic serialization on the hot path.
* fix indentation and add clarifying comment for usage stripping
* fix: guard calculate_total_usage against None usage in chunks
* fix: store chunk copy to preserve usage for calculate_total_usage
When `default_internal_user_params` was set, `insert_sso_user()` only
preserved SSO-provided roles if `role_mappings` was explicitly configured.
Roles from other valid SSO sources (Microsoft app_roles,
GENERIC_USER_ROLE_ATTRIBUTE, custom SSO handlers) were silently
overwritten with the default "internal_user" role, causing admin users
to be downgraded on first login or after user deletion.
Replace the `role_mappings_configured` gate with `_should_use_role_from_sso_response()`
which validates the role is a recognized LitellmUserRoles value regardless
of origin. Also removes an unnecessary DB round-trip to litellm_ssoconfig
on every new SSO user creation.
Fixes: admin users seeing internal-user UI after SSO login
Fixes: test_get_redirect_url_for_sso flaking due to local env vars
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Save and restore litellm.initialized_langfuse_clients around
test_max_langfuse_clients_limit to prevent ordering-dependent failures
in other tests that rely on the counter's value.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- test_litellm_pre_call_utils.py: wrap test body in try/finally so
litellm.callbacks is always restored even when an assertion fails,
addressing greptile review comment
- test_langfuse_otel.py: resolve trivial merge conflict in comment
("unpatched" vs "unpatch-ed"), keeping correct spelling
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
test_reload_model_cost_map_admin_access calls the /reload/model_cost_map
HTTP endpoint with get_model_cost_map mocked to return a single-entry
dict. The endpoint handler does a direct module-level assignment
(litellm.model_cost = new_model_cost_map) which persists after the
patch context manager exits, stripping all models except gpt-3.5-turbo
from the in-memory cost map and causing subsequent tests that rely on
models like gemini-1.5-flash, multimodalembedding@001, and gpt-4o to
fail with "model not mapped" errors or zero-cost spend payloads.
Fix: save litellm.model_cost before the test and restore it (along with
invalidating the case-insensitive lookup cache) in a finally block.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The ChatGPT backend API sends non-spec-compliant streaming tool call chunks
where index is always 0 for parallel tool calls and id/name get repeated in
duplicate closing chunks. Add ChatGPTToolCallNormalizer to fix indices and
filter duplicates before they reach the consumer.
Fixes#21482
* fix: guard against None metadata in prometheus metrics
Use get_litellm_metadata_from_kwargs and get_metadata_variable_name_from_kwargs
helpers to properly resolve metadata from both 'metadata' and 'litellm_metadata'
keys, with None safety.
* test: add test for None metadata in prometheus metrics
Update provider matching so github/<model> aliases can resolve capabilities from existing upstream model metadata, including OpenAI and Anthropic entries. Add regression tests for known github aliases and unknown-model fallback behavior.
The adapter was injecting `summary: "detailed"` into the reasoning config
when routing Anthropic thinking requests to OpenAI's Responses API.
Per the OpenAI spec, reasoning.summary is opt-in — it should not be
added unless the user explicitly requests it.
Three test isolation issues fixed:
1. test_mcp_debug.py: Replace deprecated asyncio.get_event_loop().run_until_complete()
with asyncio.run() in TestWrapSendWithDebugHeaders. In Python 3.10+,
get_event_loop() raises RuntimeError when no event loop is set in the
current thread, causing test_injects_headers and test_body_messages_unchanged
to fail in isolation.
2. test_mcp_server_manager.py: After _reload_mcp_manager_module() creates a new
global_mcp_server_manager instance, server.py still holds a stale reference
to the old instance. Tests in test_mcp_server.py that populate the new
manager's registry and then call server.py functions (e.g. _get_tools_from_mcp_servers)
get empty results because server.py reads from the old manager. Fix: update
server.py's module-level reference after each reload.
3. test_litellm_pre_call_utils.py: test_add_litellm_metadata_from_request_headers
sets litellm.callbacks without restoring it afterward. Add cleanup to restore
original callbacks after the test to prevent state leaking to subsequent tests.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Four finally blocks in test_internal_user_endpoints.py and one in
test_ui_sso.py used the pattern:
if original_default_params is not None:
litellm.default_internal_user_params = original_default_params
else:
delattr(litellm, "default_internal_user_params")
Since the attribute is defined in litellm/__init__.py with a default of
None, `getattr(litellm, "default_internal_user_params", None)` returns
None. The else branch then calls delattr(), permanently removing the
attribute from the module for the rest of the process.
Subsequent tests in the same pytest-xdist worker (e.g.
test_add_new_member_* in test_management_helpers_utils.py) then fail
with: AttributeError: module 'litellm' has no attribute
'default_internal_user_params'
Fix: replace all five flawed finally blocks with a simple assignment:
litellm.default_internal_user_params = original_default_params
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
LogRecord objects do not have a .message attribute by default — it is
only populated after the record has been formatted by a Formatter.
The correct way to retrieve the formatted log message is getMessage().
This fixes AttributeError: 'LogRecord' object has no attribute 'message'
in test_cost_calculation_uses_debug_level and
test_batch_cost_calculation_uses_debug_level.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The file had two unresolved git merge conflict markers from a merge of
litellm_oss_staging_02_17_2026 into main, causing a SyntaxError when
pytest tried to collect the test module.
Kept the instance-level mocking approach (from litellm_oss_staging) for
test_get_complete_url and test_validate_environment, which is consistent
with the rest of the file and avoids class-reference issues caused by
importlib.reload(litellm) in conftest.py.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The model_prices_and_context_window_backup.json file has 'inference_geo'
fields (e.g. on 'us/claude-sonnet-4-6') for geo-prefixed Anthropic models
used in cost calculation, but the JSON schema validator in test_utils.py
did not include 'inference_geo' as an allowed property.
This caused test_aaamodel_prices_and_context_window_json_is_valid to fail
with: Additional properties are not allowed ('inference_geo' was unexpected)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The test was using setattr() to set module-level attributes (including
proxy_logging_obj = MagicMock()) on the real litellm.proxy.proxy_server
module, but the finally block only had `pass` — no cleanup.
This left proxy_logging_obj as a MagicMock in subsequent tests running
in the same pytest-xdist worker, causing TypeError when log_db_metrics
decorator called asyncio.create_task(proxy_logging_obj.service_logging_obj
.async_service_success_hook(...)) — a MagicMock is not a coroutine.
Fix: save original attribute values before the test and restore them in
the finally block to ensure test isolation.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The implementation correctly preserves tool_call order: existing results first
(call_1), then dummy results for missing ones (call_2). The test was asserting
the reverse order with incorrect comments. Fix the assertions to match the
actual correct behavior.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Three test failures caused by the real langfuse SDK import being triggered
at test time:
1. test_langfuse_prompt_management.py: Both tests create LangfusePromptManagement()
which calls `import langfuse`. Since earlier TestLangfuseUsageDetails tests
remove sys.modules["langfuse"] via patch.dict teardown, the real langfuse
import runs and fails on Python 3.14 (pydantic v1 incompatibility).
Fix: add setup_method/teardown_method to mock sys.modules["langfuse"].
2. test_langfuse.py::test_max_langfuse_clients_limit: Same root cause — creates
LangFuseLogger() without mocking sys.modules["langfuse"].
Fix: wrap test body with patch.dict("sys.modules", {"langfuse": mock}).
3. test_langfuse_otel.py::test_extract_langfuse_metadata_with_header_enrichment:
Replaces sys.modules["litellm.integrations.langfuse.langfuse"] with a stub
without restoring it, causing patch() in later tests to target the stub
instead of the real module.
Fix: use monkeypatch.setitem() which auto-restores after the test.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
test_role_mappings_override_default_internal_user_params was calling
delattr(litellm, 'default_internal_user_params') in its finally block
when the original value was None. This removes the attribute entirely from
the module, causing subsequent tests in the same xdist worker to get
AttributeError when accessing litellm.default_internal_user_params (because
litellm.__getattr__ has no handler for this name).
Fix: always restore the attribute by assignment (litellm.default_internal_user_params = original_default_params)
rather than deleting it.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
