litellm/tests/test_litellm/proxy
Sameer Kankute a78dc69a09
fix(mcp): alias/display-name tool routing, REST filters, BYOK auth (#32320)
* fix(mcp): resolve tool name prefix via known server prefixes, not string match

When an MCP server's alias differs from its server_name, tool names are
listed with the alias prefix but _execute_tool_calls compared that prefix
against the server_name stored in tool_server_map. The mismatch silently
skipped prefix stripping, forwarding the fully-prefixed tool name upstream
and causing "Unknown tool" failures. Resolve the actual MCPServer object
and strip using its known prefix forms (alias, server_name, server_id)
instead.

* fix(mcp): preserve tool overrides and scope REST tool listing

Return saved tool display/description overrides from the server table API
so the edit UI reloads them, resolve display names before prefix stripping
on tool calls, and honor mcp_server_name and toolset_name filters on the
REST tools list endpoint.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(mcp): inject BYOK credentials on Playground OpenAPI tool calls

Playground and Responses API route MCP execution through call_tool, which
skipped BYOK lookup and never set the OpenAPI auth ContextVar, so upstream
calls went out unauthenticated despite a stored user credential.

Co-authored-by: Cursor <cursoragent@cursor.com>

* test(mcp): cover alias-mismatch prefix stripping and display-name reverse mapping

Regression tests for _execute_tool_calls: an MCP server whose alias differs
from its server_name must still have its tool-name prefix stripped correctly,
and a tool called by its configured display name must resolve back to the
original tool name before dispatch.

* fix(mcp): validate tool display names against Bedrock's tool-name pattern

A display name replaces the tool name sent to the LLM provider, so a value
with spaces or other special characters saves successfully but fails every
subsequent Bedrock tool call. Validate tool_name_to_display_name server-side
(create/update payload) against Bedrock's [a-zA-Z0-9_-]+ constraint, and add
matching inline validation plus a save-blocking guard in the Admin UI's
create and edit MCP server forms.

* style(mcp): fix ruff/prettier formatting on CI

No logic changes; satisfies the format checks flagged on PR #32320.

* fix(mcp): fix CI failures on PR - complexity budget and stale test mock

Extract toolset-scope resolution and query-param normalization out of
list_tool_rest_api into helpers to bring it back under the C901 complexity
budget (was 18, now within the 15 threshold).

Add the missing get_mcp_server_by_name stub to the streaming iterator test's
mock manager; the alias-fallback resolution added for tool-name-prefix
stripping calls it unconditionally when _get_mcp_server_from_tool_name misses.

* test(mcp): cover BYOK OpenAPI auth-header helpers to close codecov patch gap

_format_byok_openapi_auth_header, _openapi_forwarded_extra_headers, and
_resolve_byok_mcp_auth_header were only exercised indirectly via a mocked
call_tool test, leaving their branches (auth-type formatting, header
forwarding/stripping, missing-credential 401) uncovered.

* fix(mcp): resolve BYOK auth before queuing the during-hook task

_resolve_byok_mcp_auth_header can raise a 401 when no credential is stored.
Resolving it after during_hook_task was already queued meant a hook's
side effects (audit logging, rate-limit bookkeeping) could run and record
success for a tool call that then fails on the missing credential.

* fix: correct mcp alias routing regressions

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-07 20:50:21 +05:30
..
_experimental/mcp_server fix(mcp): alias/display-name tool routing, REST filters, BYOK auth (#32320) 2026-07-07 20:50:21 +05:30
a2a feat(a2a): support a2a-sdk 1.x proxy routing for 0.3 and 1.0 agents (#30950) 2026-06-29 09:32:39 +05:30
agent_endpoints feat(a2a): support a2a-sdk 1.x proxy routing for 0.3 and 1.0 agents (#30950) 2026-06-29 09:32:39 +05:30
anthropic_endpoints feat: litellm oss staging (#31935) 2026-07-03 09:27:31 +05:30
auth feat(jwt): fall back to DB team memberships when JWT has no team claims (#31356) 2026-07-06 17:17:10 -07:00
batches_endpoints test(batches): add 1:1 test file scaffold for batches component paths (#30529) 2026-06-29 09:22:58 +05:30
client feat(proxy): add AES-256-GCM at-rest credential encryption with versioned format and re-encryption migration (#31215) 2026-06-29 20:14:22 +02:00
common_utils fix(proxy): stop CacheCodec dropping null fields on cache round-trip (#32207) 2026-07-06 12:53:00 -07:00
db fix(proxy): keep serving reads from the read replica when the primary DB is down at startup (#31951) 2026-07-04 12:45:09 -07:00
discovery_endpoints chore: litellm oss staging160626 (#30527) 2026-06-16 18:23:13 -07:00
experimental/mcp_server
google_endpoints chore: litellm oss staging160626 (#30527) 2026-06-16 18:23:13 -07:00
guardrails fix(model_armor): scan MCP tool calls for pre_mcp_call / during_mcp_call modes (#32296) 2026-07-06 17:09:54 -07:00
health_endpoints fix(proxy): authorize /health/test_connection against loaded deployment's team_id (VERIA-441) (#31767) 2026-07-01 17:05:49 -07:00
hooks fix(dynamic_rate_limiter): inject clock so active-project window is stable within a request (#32299) 2026-07-06 18:12:47 -07:00
image_endpoints fix(proxy): extend response headers hook to streaming, TTS, image gen, and pass-through (#24232) 2026-06-09 22:10:23 +02:00
logging_endpoints feat(proxy): add POST /v1/callbacks/logs to replay logging payloads through callbacks (#31134) 2026-06-24 15:25:10 -07:00
management_endpoints Merge pull request #32075 from BerriAI/litellm_/admiring-sammet-3b1831 2026-07-07 06:32:57 -07:00
management_helpers feat(mcp): add all-proxy-mcpservers sentinel to grant teams every MCP server (#32012) 2026-07-03 13:59:28 -07:00
memory Litellm memory improvements v2 (#26541) 2026-04-25 19:03:43 -07:00
middleware feat(proxy): configurable response headers and login-page hint (#30792) 2026-06-18 18:12:45 -07:00
openai_files_endpoint fix(vertex/files): stream OpenAI->Vertex batch JSONL uploads (#31036) 2026-06-24 13:19:57 -07:00
pass_through_endpoints fix(streaming): stamp completion_start_time on first chunk for /v1/messages and /v1/responses (#32284) 2026-07-06 19:30:31 -07:00
policy_engine fix(policies): reject non-existent team/key/model scope entries on attachment create (#32131) 2026-07-04 11:58:29 -07:00
prompts style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
proxy_server fix(proxy): wire general_settings SSRF allowlist to litellm globals (#32243) 2026-07-07 20:49:55 +05:30
public_endpoints feat(ui): add Amazon Bedrock Mantle to the Add Model provider dropdown (#31034) 2026-06-22 17:31:25 -07:00
rag_endpoints fix: harden /key/update authorization checks (#27878) 2026-05-14 04:16:04 +00:00
realtime_endpoints chore: litellm oss 170626 (#30637) 2026-06-17 21:11:12 -07:00
response_api_endpoints Litellm websocket improvements (#29563) 2026-06-03 11:48:35 -07:00
shutdown feat(proxy): native /health/drain preStop hook for graceful shutdown (#29439) 2026-06-02 16:30:44 -07:00
spend_tracking fix(security): hash Bearer-prefixed API keys in spend logs (#31799) 2026-07-06 13:30:38 -07:00
test_configs
types_utils chore(proxy): also scrub guardrail callbacks / module paths from DB overlay 2026-05-14 01:24:51 +00:00
ui_crud_endpoints fix(mcp): byom visibility, preview UX, and admin settings gating (#31809) 2026-07-02 01:04:22 -07:00
utils fix(proxy): isolate poison spend-log rows so one bad record can't drop the whole batch (#31705) 2026-06-30 12:21:14 -07:00
vector_store_endpoints Title: fix(proxy): resolve vector store file list credentials from team deployments (#29739) 2026-06-06 12:36:05 -07:00
video_endpoints test(videos): add 1:1 test file scaffold for videos component paths (#30631) 2026-06-29 09:12:51 +05:30
__init__.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
conftest.py feat(proxy): native /health/drain preStop hook for graceful shutdown (#29439) 2026-06-02 16:30:44 -07:00
test_aiohttp_cleanup_closed.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_aiohttp_session_recovery.py fix: use AsyncMock for concurrent test consistency 2026-03-18 00:54:23 +00:00
test_api_key_masking_in_errors.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_audio_speech_prometheus_hooks.py fix(proxy): extend response headers hook to streaming, TTS, image gen, and pass-through (#24232) 2026-06-09 22:10:23 +02:00
test_batch_expiry.py fix(proxy): strip LiteLLM policy tracking from OpenAI batch metadata (#28425) 2026-05-26 11:35:42 -07:00
test_batch_metadata_none_fix.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_batch_retrieve_bedrock.py fix(proxy/batches): forward model to retrieve_batch for bedrock 2026-04-29 22:48:03 +02:00
test_batch_x_litellm_model_encoding.py test(batches): move orphan tests into tests/test_litellm for CI coverage (#30510) 2026-06-16 10:20:59 -07:00
test_blocked_response_usage.py feat: litellm oss staging (#31935) 2026-07-03 09:27:31 +05:30
test_budget_reservation.py chore: litellm oss staging (#30745) 2026-06-18 13:55:35 -07:00
test_caching_routes.py Litellm oss staging 040626 (#29671) 2026-06-04 11:07:20 -07:00
test_chat_completion_metadata.py fix: propagate JWT auth metadata to OTEL spans (#19627) 2026-01-23 21:21:23 -08:00
test_common_request_processing.py test(proxy): cover per-key per-model TPM limit triggering gateway fallback 2026-07-06 18:25:49 -07:00
test_component_allowlists.py chore(oss): litellm oss staging 120626 (#30292) 2026-06-12 09:49:25 -07:00
test_cors_config.py refactor: extract _get_cors_config() for testability, fix no-op CORS tests 2026-04-11 22:24:04 +05:30
test_custom_proxy.py
test_dynamic_mcp_route.py Litellm oss staging 040626 (#29671) 2026-06-04 11:07:20 -07:00
test_empty_model_list.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_enforce_user_param.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_fallback_management_endpoints.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_fastapi_offline_routes.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_filter_models_by_team_access_group.py fix: resolve access group names in _filter_models_by_team_id (#25224) 2026-05-01 11:55:36 +05:30
test_health_check_functions.py feat(proxy): skip disable_background_health_check models on GET /health when flag set (#27716) 2026-05-13 09:49:05 -07:00
test_health_check_max_tokens.py fix: skip health check for semantic auto_router deployments (#31668) 2026-06-29 19:47:20 -07:00
test_langfuse_passthrough_security.py chore(callbacks): guard dynamic integration hosts 2026-04-30 14:27:19 -07:00
test_lazy_openapi_snapshot.py [Fix] Tests: Move Misplaced Import in Lazy OpenAPI Snapshot Test 2026-05-01 14:24:53 -07:00
test_litellm_pre_call_utils.py fix(proxy): restore admin key/team callback_vars.turn_off_message_logging override (LIT-3587) (#31905) 2026-07-02 15:01:58 -07:00
test_max_budget_env_var.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_mcp_asgi_response.py fix(mcp): surface upstream 401 for token-forwarding MCP servers (#27847) 2026-05-13 12:03:36 -07:00
test_model_based_routing_files_batches.py test(batches): move orphan tests into tests/test_litellm for CI coverage (#30510) 2026-06-16 10:20:59 -07:00
test_model_dump_with_preserved_fields.py Fix_mapped tests part 2 2026-02-26 12:43:39 +05:30
test_model_id_header_propagation.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_model_info_default_limits.py fix(proxy): align /v1/model/info with router deployments (#30025) 2026-06-10 19:38:21 -07:00
test_model_level_guardrails.py fix(guardrails): run pre_call hook once for model-level guardrails (#30543) 2026-06-16 11:17:03 -07:00
test_model_list_healthy_only.py feat: litellm oss 110626 (#30202) 2026-06-11 22:30:26 -07:00
test_modify_response_streaming_passthrough.py fix(proxy): use e.request_data for logging_obj in ModifyResponseException streaming passthrough (#30800) 2026-06-18 23:29:08 -07:00
test_openapi_schema_validation.py fix(proxy): preserve HTTP operations when injecting WebSocket stubs into OpenAPI schema 2026-05-06 00:28:42 +02:00
test_plugin_routes.py feat: litellm plugin architecture v2 (#30688) 2026-06-20 20:37:22 -07:00
test_pricing_field_strip.py chore: litellm oss staging160626 (#30527) 2026-06-16 18:23:13 -07:00
test_prometheus_cleanup.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_provider_url_destination_guard.py chore(proxy): move URL-valued model/file_id guard from SDK to proxy 2026-05-02 02:17:46 +00:00
test_proxy_cli.py chore: litellm oss staging (#30968) 2026-06-23 07:31:44 -07:00
test_proxy_logging_hook_detection.py perf: reduce per-request and per-chunk overhead across Anthropic streaming hot paths (#28289) 2026-05-23 12:15:59 -07:00
test_proxy_server.py fix(proxy): wire general_settings SSRF allowlist to litellm globals (#32243) 2026-07-07 20:49:55 +05:30
test_proxy_types.py fix(proxy): add default=None to LiteLLM_TeamMembership.litellm_budget_table (#29684) 2026-06-04 12:13:11 -07:00
test_proxy_utils.py fix(proxy): record partial spend on the failure row for interrupted streams (#30788) 2026-06-19 12:03:15 -07:00
test_pyroscope.py Implement normalize_nonempty_secret_str function to trim whitespace from secrets and treat empty values as unset. Update proxy_server to use this function for Grafana credentials. Enhance tests to validate the new normalization behavior. 2026-05-04 18:17:31 +00:00
test_read_model_list.py feat: add minimal rust router + axum ai-gateway calling router.realtime (2/2) (#31135) 2026-06-23 19:16:34 -07:00
test_redis_auth_cache_flag.py refactor(proxy): replace DualCache with UserApiKeyCache for user API key management 2026-04-28 19:15:03 +00:00
test_response_model_sanitization.py perf: eliminate per-request callback scanning on proxy hot path (#27858) 2026-05-14 09:28:31 -07:00
test_route_a2a_models.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_route_llm_request.py fix(realtime): preserve nested transcription model and session-first model priority 2026-07-03 14:21:22 -07:00
test_sensitive_route_auth.py chore(proxy): guard sensitive public endpoints 2026-04-30 11:52:47 -07:00
test_shared_health_check.py feat(proxy): skip disable_background_health_check models on GET /health when flag set (#27716) 2026-05-13 09:49:05 -07:00
test_spend_log_cleanup.py feat(spend_logs): opt-in native Postgres partitioning for SpendLogs retention (#29466) 2026-06-11 11:02:42 -07:00
test_swagger_chat_completions.py fix(proxy): avoid misleading multi-method operation ids 2026-04-30 20:44:14 -07:00
test_team_member_update.py feat(ui): add budget duration to edit team member form (#29717) 2026-06-06 17:24:55 -07:00
test_team_org_move.py fix(team_endpoints): auto-add SSO team members to org on move (proxy admin only) (#26377) 2026-04-24 08:36:25 -07:00
test_tools_allowlist_enforcement.py fix(responses-bridge): custom tool round-trip and allowlist preservation for Codex CLI (#32258) 2026-07-06 17:34:27 -07:00
test_update_llm_router_resilience.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00