mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-17 23:51:30 +00:00
55fe4a7894
* feat(proxy): resolve root_path per request from SERVER_ROOT_PATHS One deployment can encode exactly one client-visible URL path prefix today: SERVER_ROOT_PATH is a scalar stamped onto the app at startup, so a pod fronting several ingress prefixes 404s every prefix but one before any handler runs, and MCP OAuth discovery can emit only one prefix's URLs (RFC 9728 section 3 exact-match fails for the rest). Add an opt-in outermost ASGI middleware that matches the request path against a configured prefix list (SERVER_ROOT_PATHS, comma-separated) on a segment boundary and sets scope["root_path"] for that request only. Everything downstream is stock Starlette: route matching strips root_path so routes stay registered root-relative, and request.base_url re-includes it, so the discovery documents' resource and the 401 challenges' resource_metadata land under the prefix the client actually called — with no discovery-builder changes. LazyFeatureMiddleware now strips the scope root_path (falling back to the cached SERVER_ROOT_PATH scalar) before feature prefix matching, so lazily-registered routers — the MCP OAuth discovery router among them — load under per-request prefixes. Follow-up to the routing discussion on #35226; composes with, but does not depend on, #35576. * fix(proxy): import Sequence from collections.abc (ruff UP035 strict-budget gate) * review(greptile): trim implementation commentary; fixture-own MCP registry state in tests Addresses both P2s from the first Greptile pass: - per_request_root_path_middleware.py (and the related _lazy_features / proxy_server comments) cut down to the constraints the code cannot express, per repo comment guidance - the new discovery tests no longer clear/repopulate the shared MCP registry inline; a fixture snapshots it, hands the test an empty registry, and restores it afterwards so no state leaks between cases * fix(lint): mutable-ok marker on the prefix accumulator (LIT002 type-discipline gate) * fix(proxy): tie 401 challenges and get_custom_url to the per-request root_path The per-request root_path middleware sets scope["root_path"] to the prefix the client actually called, but the OAuth 401 challenges (raise_user_oauth_challenge / raise_token_exchange_challenge) still built their resource_metadata from SERVER_ROOT_PATH. On a pod fronting several prefixes, the challenge advertised a discovery URL under a different prefix than the discovery document served — the two disagreed on where the resource metadata lives, and a strict RFC 9728 client refused the challenge. Route the challenges through a small ContextVar the middleware populates so they read the same effective root_path Starlette resolves the request under. The same accessor fixes get_custom_url: when a request lives under a SERVER_ROOT_PATHS-matched prefix, request.base_url already carries it, so appending the SERVER_ROOT_PATH scalar on top produced e.g. /tenant-a/legacy/sso/callback — a path that does not exist. Reading the per-request prefix instead (and relying on join_paths's tail-dedup) keeps SSO login/callback URLs under one prefix — the one the request actually arrived on. Fallback: outside a request (module-load-time UI URL builders, background tasks) the ContextVar is unset and the accessor reads SERVER_ROOT_PATH, matching get_server_root_path() so scalar-only deployments are byte-identical. * fix(mcp): challenge URL under per-request prefix must route, and mock parity Two follow-ups to the review fix that made the 401 challenge use the per-request root_path: 1. oauth_protected_resource_path must pick the URL structure that actually routes for the mechanism in use: - The scalar SERVER_ROOT_PATH deployment registers the well-known routes with the prefix INSERTED (via well_known_root_suffix at import time), matching RFC 8414 §3. The challenge URL must use the same insertion or a client fetching it 404s. - The per-request SERVER_ROOT_PATHS deployment can't register routes per prefix; PerRequestRootPathMiddleware strips the prefix from scope["path"] and the router matches the un-inserted route. The URL must place the prefix BEFORE .well-known so the strip leaves a matching path. The previous fix used the insertion form for both, which 404'd the discovery fetch on the per-request path — the discovery doc and the challenge would then disagree on where the resource metadata lives, the very failure the review flagged. End-to-end verified: the URL the challenge advertises routes and the doc's `resource` field equals the URL the client originally called (RFC 9728 §3). 2. get_request_root_path now delegates its fallback through get_server_root_path() instead of reading the env directly, so every existing `monkeypatch.setattr("litellm.proxy.utils.get_server_root_path"` test override keeps working. This unstubbed the mock on the /v2/login test that failed on the last CI run. Plus the lint budget: annotate the local accumulator Final, tag the scope["root_path"] rewrite as an intentional ASGI-contract mutation, tag the reused `path`/`root_path` rebinds in LazyFeatureMiddleware, and add reason strings to the two new PLC0415 lazy-import noqas. * test(mcp): pin the reviewer's expected end-state — challenge URL routes, resource matches called URL End-to-end regression test that mounts the discoverable router + the per-request root_path middleware, hits an MCP endpoint that raises raise_user_oauth_challenge, fetches the resource_metadata URL the challenge advertises, and checks the returned document's `resource` equals the URL the client originally called (RFC 9728 §3 exact match). Covers /tenant-a, /tenant-b, and the unprefixed path on the same app so a regression on any prefix — challenge URL 404s, or doc emits a different prefix than the client called — fails at this test rather than in a strict MCP client's discovery. --------- Co-authored-by: gym-cmd <186399764+gym-cmd@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| _experimental/mcp_server | ||
| a2a | ||
| agent_endpoints | ||
| analytics_endpoints | ||
| anthropic_endpoints | ||
| auth | ||
| batches_endpoints | ||
| client | ||
| common_utils | ||
| config_resolvers | ||
| container_endpoints | ||
| credential_endpoints | ||
| db | ||
| discovery_endpoints | ||
| enterprise_billing | ||
| experimental/mcp_server | ||
| fine_tuning_endpoints | ||
| google_endpoints | ||
| guardrails | ||
| health_endpoints | ||
| hooks | ||
| image_endpoints | ||
| list_api | ||
| logging_endpoints | ||
| management_endpoints | ||
| management_helpers | ||
| memory | ||
| middleware | ||
| ocr_endpoints | ||
| openai_files_endpoint | ||
| pass_through_endpoints | ||
| policy_engine | ||
| prompts | ||
| proxy_server | ||
| public_endpoints | ||
| rag_endpoints | ||
| realtime_endpoints | ||
| rerank_endpoints | ||
| response_api_endpoints | ||
| shutdown | ||
| spend_tracking | ||
| test_configs | ||
| types_utils | ||
| ui_crud_endpoints | ||
| utils | ||
| vector_store_endpoints | ||
| vector_store_files_endpoints | ||
| video_endpoints | ||
| __init__.py | ||
| conftest.py | ||
| test__types.py | ||
| test_aiohttp_cleanup_closed.py | ||
| test_aiohttp_session_recovery.py | ||
| test_api_key_masking_in_errors.py | ||
| test_audio_speech_prometheus_hooks.py | ||
| test_batch_expiry.py | ||
| test_batch_metadata_none_fix.py | ||
| test_batch_retrieve_bedrock.py | ||
| test_batch_x_litellm_model_encoding.py | ||
| test_blocked_response_usage.py | ||
| test_budget_reservation.py | ||
| test_caching_routes.py | ||
| test_chat_completion_metadata.py | ||
| test_claude_code_marketplace.py | ||
| test_common_request_processing.py | ||
| test_component_allowlists.py | ||
| test_conftest.py | ||
| test_cors_config.py | ||
| test_custom_proxy.py | ||
| test_dynamic_mcp_route.py | ||
| test_empty_model_list.py | ||
| test_enforce_user_param.py | ||
| test_fallback_management_endpoints.py | ||
| test_fastapi_offline_routes.py | ||
| test_filter_models_by_team_access_group.py | ||
| test_health_check_functions.py | ||
| test_health_check_max_tokens.py | ||
| test_init_litellm_callbacks.py | ||
| test_langfuse_passthrough_security.py | ||
| test_lazy_openapi_snapshot.py | ||
| test_litellm_pre_call_utils.py | ||
| test_max_budget_env_var.py | ||
| test_mcp_asgi_response.py | ||
| test_model_based_routing_files_batches.py | ||
| test_model_deprecations_endpoint.py | ||
| test_model_dump_with_preserved_fields.py | ||
| test_model_id_header_propagation.py | ||
| test_model_info_default_limits.py | ||
| test_model_level_guardrails.py | ||
| test_model_list_healthy_only.py | ||
| test_modify_response_streaming_passthrough.py | ||
| test_openai_ws_passthrough_routes.py | ||
| test_openapi_schema_validation.py | ||
| test_plugin_routes.py | ||
| test_pricing_field_strip.py | ||
| test_prisma_engine_watchdog.py | ||
| test_prisma_migration.py | ||
| test_prometheus_cleanup.py | ||
| test_prometheus_metrics_server.py | ||
| test_provider_url_destination_guard.py | ||
| test_proxy_cli.py | ||
| test_proxy_logging_hook_detection.py | ||
| test_proxy_server.py | ||
| test_proxy_types.py | ||
| test_proxy_utils.py | ||
| test_pyroscope.py | ||
| test_read_model_list.py | ||
| test_redis_auth_cache_flag.py | ||
| test_response_model_sanitization.py | ||
| test_route_a2a_models.py | ||
| test_route_llm_request.py | ||
| test_sensitive_route_auth.py | ||
| test_shared_health_check.py | ||
| test_spend_log_cleanup.py | ||
| test_swagger_chat_completions.py | ||
| test_team_member_update.py | ||
| test_team_org_move.py | ||
| test_tools_allowlist_enforcement.py | ||
| test_update_llm_router_resilience.py | ||