litellm/tests/unit/proxy
Etienne Chabert 372c096388
fix(proxy): evict a deleted key from every worker's auth cache (#47609)
* fix(proxy): evict a deleted key from every worker's auth cache

/key/delete dropped the key object only from the in-memory cache of the worker
that served the call. Auth serves key objects cache-first, so every other
worker, and the Redis copy when enable_redis_auth_cache is on, kept
authenticating the deleted key until user_api_key_cache_ttl expired. /key/update
already evicts and broadcasts through evict_and_broadcast, delete now does the
same for the tokens it deletes

* fix(proxy): evict only the deleted rows and broadcast them in one batch

Scope the cross-worker eviction on /key/delete to the rows the caller was allowed to delete and send it through delete_cache_key_objects, so a caller-supplied name can no longer drop another team's cached entries on every worker

* refactor(proxy): drop the comment above the delete eviction call

* fix(proxy): run the usage cache eviction beside the auth eviction so a hung Redis costs one wait

* test(proxy): wait on the broadcast instead of a sleep and type the peer eviction tests

* test(proxy): wait for the peer to apply the broadcast instead of polling its cache

* test(proxy): cover hashed keys and aliases when deleting a key warmed on a peer

* fix(key_management): evict every requested key on delete, including keys whose row is already gone

A batch delete that names a live key and a key whose row was removed outside the API evicted only the live key, so the gone key kept authenticating from every worker's cache. Every requested key hash is evicted now, while raw non-key cache keys are still left alone

* test(key_management): split the gone-row eviction assertion into one comprehension per worker

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-10-30 10:20:00 -07:00
..
_experimental fix(mcp): load deprecated package aliases on demand (#48615) 2026-10-27 19:46:19 +00:00
a2a
agent_endpoints fix(streaming): keep merged think tags balanced and split SSE lines only on CR/LF (#48755) 2026-10-28 17:00:29 -07:00
analytics_endpoints fix(proxy): scope /global/activity/cache_hits analytics to the caller's own and permitted-team logs (#46210) 2026-10-14 10:01:21 +02:00
anthropic_endpoints fix(count_tokens): count a history that carries a compaction block (#44784) 2026-10-30 00:11:55 -07:00
auth fix(proxy): evict a deleted key from every worker's auth cache (#47609) 2026-10-30 10:20:00 -07:00
batches_endpoints refactor(router): move router callers onto the entry points and invert the proxy imports (#48847) 2026-10-29 17:26:33 +00:00
client refactor(types): replace Any with proven types in 21 files (#48921) 2026-10-29 18:43:24 -07:00
common_utils fix(proxy)!: label shared-handler errors the way the route catch-alls do (#39555) 2026-10-30 06:58:05 -07:00
config_resolvers
container_endpoints
credential_endpoints fix(proxy)!: label shared-handler errors the way the route catch-alls do (#39555) 2026-10-30 06:58:05 -07:00
db fix(spend): keep a NaN or infinite cost from wedging daily spend writes (#48829) 2026-10-30 08:27:35 -07:00
decisions_endpoints feat(decisions): support Ollama decision models (#48871) 2026-10-28 12:35:26 -07:00
discovery_endpoints feat(proxy)!: general_settings.claude_code_marketplace_auth_required puts the Claude Code marketplace behind a LiteLLM key (#48980) 2026-10-29 03:48:35 -07:00
enterprise_billing
example_config_yaml
fine_tuning_endpoints fix(managed_files): send provider file ids when creating a fine-tuning job from managed files (#46552) 2026-10-19 14:40:15 -07:00
google_endpoints fix(google_genai): read one-object and null spellings of repeated fields in countTokens and generateContent (#48771) 2026-10-28 21:46:06 -07:00
guardrails fix(proxy)!: label shared-handler errors the way the route catch-alls do (#39555) 2026-10-30 06:58:05 -07:00
health_endpoints fix(proxy)!: scope /health by wildcard keys and team model allowlists the way auth does (#49155) 2026-10-30 07:36:01 -07:00
hooks fix(proxy)!: log and bill Responses-style bodies sent to /cursor/chat/completions (#30277) 2026-10-30 00:18:15 -07:00
image_endpoints test: migrate SDK and proxy-booting legacy CircleCI tests to their tiers (#45758) 2026-10-11 00:17:21 -07:00
lens fix(proxy): validate JSON object bodies before request handling (#48438) 2026-10-26 16:11:38 -07:00
list_api
logging_endpoints
management fix(ui): take the dashboard's user role and identity from the server (#47025) 2026-10-17 14:39:33 -07:00
management_endpoints fix(proxy): evict a deleted key from every worker's auth cache (#47609) 2026-10-30 10:20:00 -07:00
management_helpers refactor(router): move router callers onto the entry points and invert the proxy imports (#48847) 2026-10-29 17:26:33 +00:00
mcp_server fix(mcp)!: restore upstream-token admission for legacy delegate servers (#48939) 2026-10-29 15:56:29 -07:00
memory
middleware feat(pass-through): add eu/jev decision API pass-through with spend tracking (#46875) 2026-10-29 17:52:43 -07:00
ocr_endpoints fix(proxy)!: return provider-matching 4xx for the LIT-8062 status code follow-ups (#45537) 2026-10-13 11:42:39 -07:00
openai_files_endpoint refactor(router): move router callers onto the entry points and invert the proxy imports (#48847) 2026-10-29 17:26:33 +00:00
pass_through_endpoints fix(passthrough): log the URL model and declared provider on Gemini and Vertex failures (#49075) 2026-10-30 09:40:58 -07:00
policy_engine fix(websearch-interception)!: stream native server-tool blocks to clients (#40967) 2026-10-30 00:06:57 -07:00
prompts
proxy_server fix(proxy)!: stop answering with the literal string "None" as error type and param on the proxy-server, auth, spend, and key health routes (#39540) 2026-10-29 18:20:52 -07:00
public_endpoints feat(lite): offer each model group's reasoning efforts in pi, OpenCode and Codex (#48949) 2026-10-29 18:15:44 -07:00
rag_endpoints feat(vector_stores): add OpenSearch k-NN search as a vector store provider (#47759) 2026-10-29 08:15:05 -07:00
realtime_endpoints
rerank_endpoints
response_api_endpoints fix(proxy)!: log and bill Responses-style bodies sent to /cursor/chat/completions (#30277) 2026-10-30 00:18:15 -07:00
response_polling refactor(router): move router callers onto the entry points and invert the proxy imports (#48847) 2026-10-29 17:26:33 +00:00
roi_calculator fix: cap cold MCP schema workers at usable CPUs and deflake wall-clock unit tests (#45791) 2026-10-15 00:47:59 -07:00
search_endpoints refactor(router): move router callers onto the entry points and invert the proxy imports (#48847) 2026-10-29 17:26:33 +00:00
shutdown fix(proxy): persist tag spend and drain success callbacks within the shutdown budget (#48988) 2026-10-29 09:32:18 -07:00
spend_tracking fix(spend): keep a NaN or infinite cost from wedging daily spend writes (#48829) 2026-10-30 08:27:35 -07:00
telemetry feat(chat): add Moyai UI, model hub, images, and usage metrics (#40557) (#48108) 2026-10-23 23:39:23 +00:00
test_configs test: migrate legacy unit dirs and unrun folders into tests/unit, tests/integration and tests/e2e (#45940) 2026-10-11 06:24:57 -07:00
types_utils refactor(mcp): drop the experimental label from the MCP server and client paths (#48513) 2026-10-27 17:47:50 +00:00
ui_crud_endpoints fix(ui): honor dark and local logos across Connect and Chat (#48532) 2026-10-27 03:00:03 +00:00
utils fix(proxy)!: label shared-handler errors the way the route catch-alls do (#39555) 2026-10-30 06:58:05 -07:00
vector_store_endpoints refactor(router): move router callers onto the entry points and invert the proxy imports (#48847) 2026-10-29 17:26:33 +00:00
vector_store_files_endpoints
video_endpoints refactor(router): move router callers onto the entry points and invert the proxy imports (#48847) 2026-10-29 17:26:33 +00:00
__init__.py
conftest.py feat(pass-through): add eu/jev decision API pass-through with spend tracking (#46875) 2026-10-29 17:52:43 -07:00
router_entry_point_stubs.py feat(lite): offer each model group's reasoning efforts in pi, OpenCode and Codex (#48949) 2026-10-29 18:15:44 -07:00
test__lazy_features.py feat(ui): unify gateway and usage assistance as Aria Agent (#48082) 2026-10-23 20:24:19 +00:00
test__types.py fix(proxy): reject negative RPM and TPM request limits (#48444) 2026-10-26 21:16:15 +00:00
test_admin_mcp.py fix: enforce LiteAdmin MCP route permissions (#48080) 2026-10-30 02:00:04 +00:00
test_aiohttp_cleanup_closed.py
test_aiohttp_session_recovery.py refactor(proxy): expose public names for private proxy helpers (#45170) 2026-10-07 18:59:27 -07:00
test_api_key_masking_in_errors.py
test_aproxy_startup.py
test_audio_speech_prometheus_hooks.py
test_batch_expiry.py
test_batch_metadata_none_fix.py
test_batch_retrieve_bedrock.py
test_batch_x_litellm_model_encoding.py fix(rag)!: use team provider credentials and allow viewer and JWT team access (#45497) 2026-10-17 23:51:48 +00:00
test_blocked_response_usage.py
test_body_snapshot_callback_params.py
test_budget_reservation.py fix(proxy): release concurrency slots when synthetic streams complete (#48628) 2026-10-27 14:47:04 -07:00
test_bug_report_config.py
test_caching_routes.py chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
test_chat_completion_metadata.py refactor(proxy): expose public names for private proxy helpers (#45170) 2026-10-07 18:59:27 -07:00
test_claude_code_marketplace.py
test_collector.py test(unit): keep unix socket paths in unit tests under the macOS AF_UNIX limit (#45040) 2026-10-28 20:23:22 -07:00
test_common_request_processing.py fix(spend): keep a NaN or infinite cost from wedging daily spend writes (#48829) 2026-10-30 08:27:35 -07:00
test_component_allowlists.py fix: enforce LiteAdmin MCP route permissions (#48080) 2026-10-30 02:00:04 +00:00
test_conftest.py
test_cors_config.py
test_credential_slot_registry.py feat(microsoft_365_copilot): add Microsoft 365 Copilot chat provider with OAuth token exchange (#45158) 2026-10-09 15:19:07 -07:00
test_custom_callback_input.py
test_custom_logger_s3_gcs.py
test_custom_proxy.py
test_custom_tokenizer_bug.py fix(count_tokens)!: answer 400 for Vertex partner models without a count API and OpenAI-shaped provider errors on /v1/responses/input_tokens (#47163) 2026-10-17 21:15:38 +00:00
test_db_schema_changes.py perf(ci): isolate and shard integration suites (#44146) 2026-10-10 11:15:19 -07:00
test_deprecated_key_grace_period.py
test_dynamic_mcp_route.py refactor(mcp): drop the experimental label from the MCP server and client paths (#48513) 2026-10-27 17:47:50 +00:00
test_empty_model_list.py refactor(router): move router callers onto the entry points and invert the proxy imports (#48847) 2026-10-29 17:26:33 +00:00
test_enforce_user_param.py
test_fallback_management_endpoints.py fix(router)!: apply content_policy_fallbacks when router settings update at runtime (#45428) 2026-10-30 04:27:11 -07:00
test_fastapi_offline_routes.py
test_filter_models_by_team_access_group.py refactor(router): move router callers onto the entry points and invert the proxy imports (#48847) 2026-10-29 17:26:33 +00:00
test_get_favicon.py
test_get_image.py
test_health_check_functions.py fix(proxy)!: scope /health by wildcard keys and team model allowlists the way auth does (#49155) 2026-10-30 07:36:01 -07:00
test_health_check_max_tokens.py fix(router)!: keep a deployment's own model_id over the request body's (#49045) 2026-10-30 02:40:01 -07:00
test_init_litellm_callbacks.py fix(proxy): build configured logging callbacks at boot so a broken callback fails startup instead of the first request (#46378) 2026-10-14 16:51:55 +00:00
test_langfuse_passthrough_security.py
test_lazy_openapi_snapshot.py
test_litellm_pre_call_utils.py fix(router)!: apply content_policy_fallbacks when router settings update at runtime (#45428) 2026-10-30 04:27:11 -07:00
test_max_budget_env_var.py feat(proxy)!: add suppress_startup_output to turn off startup prints (#47708) 2026-10-29 03:47:04 -07:00
test_mcp_asgi_response.py
test_model_based_routing_files_batches.py
test_model_deprecations_endpoint.py refactor(router): move router callers onto the entry points and invert the proxy imports (#48847) 2026-10-29 17:26:33 +00:00
test_model_dump_with_preserved_fields.py fix(proxy): keep provider refusal, logprobs and system_fingerprint in chat completions (#44788) 2026-10-13 13:17:40 -07:00
test_model_id_header_propagation.py
test_model_info_default_limits.py refactor(router): move router callers onto the entry points and invert the proxy imports (#48847) 2026-10-29 17:26:33 +00:00
test_model_level_guardrails.py refactor(router): move router callers onto the entry points and invert the proxy imports (#48847) 2026-10-29 17:26:33 +00:00
test_model_list_aliases.py fix(proxy): resolve model aliases to their target deployment for listing metadata (#41041) 2026-10-15 00:38:50 -07:00
test_model_list_callback_filter.py refactor(proxy): expose public names for private proxy helpers (#45170) 2026-10-07 18:59:27 -07:00
test_model_list_discoverable.py refactor(proxy): expose public names for private proxy helpers (#45170) 2026-10-07 18:59:27 -07:00
test_model_list_healthy_only.py refactor(router): move router callers onto the entry points and invert the proxy imports (#48847) 2026-10-29 17:26:33 +00:00
test_modify_response_streaming_passthrough.py refactor(proxy): expose public names for private proxy helpers (#45170) 2026-10-07 18:59:27 -07:00
test_moyai_endpoints.py test: fix flaky team delete, refusal sink, loop-blocking, Moyai tamper and model-cost leak tests (#47135) 2026-10-20 18:01:36 +00:00
test_native_compaction.py
test_openai_ws_passthrough_routes.py
test_openapi_schema_validation.py
test_plugin_routes.py chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
test_pointfive_dashboard_config.py
test_pointfive_ui_callback.py
test_pricing_field_strip.py fix(security)!: build proxy-only failure logs from an allowlist and drop caller guardrail info (#48599) 2026-10-27 12:40:59 -07:00
test_prisma_client_backoff_retry.py
test_prisma_engine_watchdog.py
test_prisma_migration.py
test_prometheus_cleanup.py
test_prometheus_metrics_server.py
test_prompt_test_endpoint.py
test_provider_url_destination_guard.py
test_proxy_cli.py fix(proxy): keep the litellm command light so spawned workers answer uvicorn's first health ping (#45039) 2026-10-29 10:25:14 -07:00
test_proxy_config_unit_test.py feat(mcp): add mcp_disabled_capabilities to stop advertising prompts and resources (#46563) 2026-10-15 15:15:40 -07:00
test_proxy_custom_auth.py
test_proxy_logging_hook_detection.py
test_proxy_reject_logging.py fix(logging)!: fire one CustomLogger failure hook per async call outside the async flag list (#47124) 2026-10-20 15:19:14 -07:00
test_proxy_server.py fix: enforce LiteAdmin MCP route permissions (#48080) 2026-10-30 02:00:04 +00:00
test_proxy_server_custom_logger.py test: migrate legacy unit dirs and unrun folders into tests/unit, tests/integration and tests/e2e (#45940) 2026-10-11 06:24:57 -07:00
test_proxy_server_endpoints_and_startup.py fix(router)!: apply content_policy_fallbacks when router settings update at runtime (#45428) 2026-10-30 04:27:11 -07:00
test_proxy_server_exception_mapping.py test: migrate legacy unit dirs and unrun folders into tests/unit, tests/integration and tests/e2e (#45940) 2026-10-11 06:24:57 -07:00
test_proxy_setting_guardrails.py refactor(proxy): expose public names for private proxy helpers (#45170) 2026-10-07 18:59:27 -07:00
test_proxy_token_counter.py fix(context_management)!: run clear_tool_uses in the gateway on the Responses API path (#47861) 2026-10-29 17:42:11 -07:00
test_proxy_types.py fix(mcp): scope gateway OAuth consent to x-mcp-servers and add explicit scope setting (#44957) 2026-10-15 15:58:34 -07:00
test_proxy_utils.py refactor(router): move router callers onto the entry points and invert the proxy imports (#48847) 2026-10-29 17:26:33 +00:00
test_proxy_utils_model_creation_and_error_logging.py refactor(router): move router callers onto the entry points and invert the proxy imports (#48847) 2026-10-29 17:26:33 +00:00
test_pyroscope.py
test_read_model_list.py
test_redis_auth_cache_flag.py fix(sso): share SAML sign-in state over Redis without enable_redis_auth_cache (#46807) 2026-10-16 02:01:04 +00:00
test_reducto_ocr_route.py
test_response_model_sanitization.py fix(proxy): report requested model in streamed Responses response.model (#46662) 2026-10-17 19:52:10 +00:00
test_response_polling_pre_call_checks.py refactor(proxy): expose public names for private proxy helpers (#45170) 2026-10-07 18:59:27 -07:00
test_route_a2a_models.py refactor(router): move router callers onto the entry points and invert the proxy imports (#48847) 2026-10-29 17:26:33 +00:00
test_route_llm_request.py refactor(router): move router callers onto the entry points and invert the proxy imports (#48847) 2026-10-29 17:26:33 +00:00
test_route_priority.py fix(proxy): let configured pass-through endpoints outrank built-in provider prefix routes (#44133) 2026-10-10 12:26:17 -07:00
test_sensitive_route_auth.py
test_server_root_path.py
test_shared_health_check.py
test_spend_log_cleanup.py test: fix flaky spend-log cleanup budget, login lockout, and wedged Redis tests (#47108) 2026-10-29 03:46:32 -07:00
test_swagger_chat_completions.py
test_team_member_update.py refactor(proxy): expose public names for private proxy helpers (#45170) 2026-10-07 18:59:27 -07:00
test_team_org_move.py
test_tools_allowlist_enforcement.py fix(proxy)!: answer 400 for malformed tools on count_tokens instead of crashing (#46583) 2026-10-16 14:57:53 -07:00
test_tracing_endpoints.py refactor(rust-bridge): require the native extension (#48852) 2026-10-28 13:16:56 -07:00
test_ui_path_detection.py
test_unit_test_proxy_hooks.py refactor(proxy): expose public names for private proxy helpers (#45170) 2026-10-07 18:59:27 -07:00
test_update_llm_router_resilience.py refactor(router): move router callers onto the entry points and invert the proxy imports (#48847) 2026-10-29 17:26:33 +00:00
test_update_spend.py fix(proxy): strip caller base_model from spend pricing and reject lone surrogates in multipart metadata (#47639) 2026-10-20 21:14:18 +00:00
test_zero_cost_model_budget_bypass.py refactor(proxy): expose public names for private proxy helpers (#45170) 2026-10-07 18:59:27 -07:00
test_zerobus_dashboard_config.py
vertex_key.json