maybe_build_debug_headers used to promote client_ip=None to
INTERNAL_REQUEST before resolving the server name, on the theory that
debug-header generation is passive observability and shouldn't drop
metadata when IP extraction failed. But when x-litellm-mcp-debug:true
is sent on a request whose IP can't be attributed, that bypass leaks
x-mcp-debug-outbound-url / x-mcp-debug-server-auth-type for any
internal-only server named in the request — even though the IP gate
would otherwise hide it.
Pass client_ip straight through. The gate now fails closed for
internal-only servers when IP extraction fails; the debug response
falls back to "(unknown)" / "(none)" instead of leaking real upstream
metadata. Internal callers (admin debug paths) still resolve normally
because their request handlers supply a real internal IP.
Add regression tests for both the leak-prevention path and the
internal-IP resolution path.
The fail-closed gate previously denied access for any server when
client_ip was None — including legitimately-public servers. A real
HTTP request whose IP couldn't be extracted (ASGI middleware nulling
request.client, broken X-Forwarded-* parsing, etc.) would 404 against
a public OAuth endpoint with no actionable diagnostic for operators.
Reorder _is_server_accessible_from_ip to check the public-server
short-circuits (available_on_public_internet and
litellm.public_mcp_servers) before failing closed on None. Non-public
servers still fail closed when client_ip is None; the
mcp_allow_unknown_client_ip opt-out still works as a wider escape
hatch for operators who can't fix IP extraction.
Also remove the None short-circuits in filter_server_ids_by_ip_with_info
and get_filtered_registry so they delegate to the per-server gate,
which now handles None correctly.
Update test_gate_contract to expect public+None → True and add
test_public_server_reachable_when_ip_unknown and
test_no_ip_lets_public_through_blocks_private for the new semantics.
_get_cached_temporary_mcp_server_or_404 (used by
/server/oauth/{server_id}/{authorize,token,register}) used to do:
server = manager.get_mcp_server_by_id(server_id) \
or manager.get_mcp_server_by_name(server_id, client_ip=client_ip)
The id-lookup branch returned servers without applying IP gating, so an
external caller hitting any of those OAuth endpoints with the UUID of
an internal-only server bypassed the IP restriction — the name-lookup
fallback was the only one gated. Apply
_is_server_accessible_from_ip(server, client_ip) to the id-lookup
result before falling through.
Add a regression test that mocks the gate to deny external IPs and
asserts a 404 when an external request hits the UUID of an
internal-only server.
Two compat/safety items flagged by Greptile in PR review:
1. Fail-closed IP gating had no opt-out flag. Operators behind ASGI
middleware or load-balancers where request.client is legitimately
None will get unexpected 403s with no recourse short of code
changes. Add general_settings.mcp_allow_unknown_client_ip (default
false). When true, missing client_ip is promoted to INTERNAL_REQUEST
at the gate. Centralized in _resolve_unknown_client_ip and applied
at the three entry points (_is_server_accessible_from_ip,
filter_server_ids_by_ip_with_info, get_filtered_registry).
2. AsyncHTTPHandler.post() now passes follow_redirects through to
single_connection_post_request on the connection-error retry path.
Without this, a transient RemoteProtocolError followed by a 30x on
reconnect could bypass the SSRF redirect block on the MCP OAuth
/token and /register flows.
The /token and /register OAuth endpoints are reachable without a
LiteLLM API key — they sit mid-OAuth-handshake. When validate_url
blocked an internal destination, the raw SSRFError ("URL targets a
blocked address (10.0.0.50). …") was wrapped verbatim into the
HTTPException detail and returned to the unauthenticated caller,
handing them the resolved internal address of the operator's IdP —
exactly the reconnaissance the SSRF guard is meant to deny.
Log the real reason at WARNING for operators and return a generic
"the destination resolves to a blocked address" detail to the caller.
Add regression tests asserting the resolved IP does not appear in
the HTTPException detail for either role.
Previously _get_allowed_mcp_servers fell back to INTERNAL_REQUEST when
both the explicit client_ip and the auth-context IP were missing,
defeating the new fail-closed gate at the wrapper layer. An external
request with no attributable client IP could reach internal-only MCP
servers via tool/list/resource calls.
Drop the auto-promote: pass None through, which now fails closed in
filter_server_ids_by_ip_with_info. Internal callers must opt in to
INTERNAL_REQUEST explicitly.
Update test_mcp_routing_with_conflicting_alias_and_group_name to mock
_get_client_ip_from_context with a real internal IP — the unit test has
no HTTP request context so it would otherwise be filtered out by the
fail-closed gate.
Update test_mcp_ip_filtering tests to assert the new fail-closed
behavior on missing IP and the INTERNAL_REQUEST escape hatch.
Veria-AI flagged that two more wrappers — get_filtered_registry and
filter_server_ids_by_ip(_with_info) — still treated client_ip=None as
"no filter, return everything." That left request paths that use them
(_resolve_oauth2_server_for_root_endpoints when get_mcp_client_ip
returned None, /public/mcp_hub registry list, server.py's MCP request
preflight) able to auto-select or list internal-only servers when IP
extraction failed.
Both wrappers now follow the same contract as
_is_server_accessible_from_ip and get_mcp_server_by_name: None fails
closed (empty result), INTERNAL_REQUEST bypasses gating, real IPs
apply the existing filter.
All 5 callers already extract client_ip from the request and pass it,
so behaviour for valid external requests is unchanged. The change only
affects the previously-fail-open path where extraction returned None.
Two follow-ups to address Veria-AI findings on the PR.
(1) Disable redirect-following on the OAuth outbound POSTs.
validate_url only inspects the initial URL; httpx clients default to
follow_redirects=True, so a malicious 30x from the validated host could
bounce the proxy to an internal target. Add follow_redirects to the
AsyncHTTPHandler.post wrapper (mirroring the existing get) and pass
follow_redirects=False from both /token and /register flows.
(2) Make get_mcp_server_by_name fail closed on None client_ip. The
wrapper previously translated None to INTERNAL_REQUEST internally to
preserve the "None means internal" convention, but that meant any
external request handler that forgot to pass an IP silently bypassed
gating. Update the wrapper to require an explicit sentinel for
internal callers; update the four external callers in
auth/user_api_key_auth_mcp.py and rest_endpoints.py to pass
INTERNAL_REQUEST (where the lookup is metadata-only) or the real
extracted client_ip.
Adjust the test fixture in test_discoverable_endpoints.py to return
INTERNAL_REQUEST instead of None so OAuth flow tests bypass IP gating
explicitly. Update two stub lambdas in test_rest_endpoints.py to accept
the new client_ip kwarg (CLAUDE.md: keep monkeypatch stubs in sync with
real signatures).
- Cast client_ip to str at the bottom of _is_server_accessible_from_ip so
mypy sees the narrowing the early returns already perform. The runtime
shape is unchanged — INTERNAL_REQUEST and None are both handled in the
early branches above.
- Distinguish "client_ip_unknown" from "ip_filtering" in the
rest_endpoints 403 response. When IP extraction fails on a public
server the previous error told the operator to set
available_on_public_internet=True, which was already set; the new
message points them at use_x_forwarded_for / mcp_trusted_proxy_ranges.
- Preserve the explicit port in the Host header when
litellm.user_url_validation is False. The opt-out path previously
dropped the port, sending a Host that didn't match the connection
target for non-default-port URLs.
Tighten the role parameter on _validate_mcp_oauth_outbound_url to
Literal["token", "registration"] so a future caller typo is caught at
type-check time. Replace the mixed Optional / PEP 604 annotation on
_is_server_accessible_from_ip with a single Union form. Trim the comment
on the get_mcp_server_by_name translation to keep the WHY and drop the
restated WHAT. Fold the four single-axis IP-gating tests into one
parametrize that covers visibility × client_ip kind in 6 cases.
No behavioural change.
The OAuth proxy endpoints /authorize, /token, and /register sit mid-OAuth-
handshake and can't require Depends(user_api_key_auth). They forward to
admin-configured token_url and registration_url. An unauthenticated caller
hitting /token or /register made the proxy POST to whatever URL the admin
configured for the MCP server, which let an internal IdP be probed via the
proxy if the admin had registered one.
Validate the outbound URL through litellm_core_utils.url_utils.validate_url
before each POST in register_client_with_server and exchange_token_with_server.
The helper resolves DNS, blocks RFC1918 / loopback / link-local destinations,
honours the existing user_url_allowed_hosts allowlist, and rewrites the URL
to the validated IP to defeat DNS rebinding. Operators who need to point at
an internal IdP can opt in via the same allowlist that other outbound
URL-validation sites use, or set litellm.user_url_validation = False.
Separately, _is_server_accessible_from_ip used to fail open when client_ip
was None, which let request handlers that couldn't determine a client IP
reach internal-only servers. Lock the contract: None now fails closed and
internal callers must pass the new INTERNAL_REQUEST sentinel to bypass IP
gating. get_mcp_server_by_name keeps its existing "None means internal"
wrapper convention by translating to the sentinel internally, so internal
callers (auth, debug, registry maintenance) continue to work unchanged.
The user-facing callsites in rest_endpoints that previously short-circuited
on client_ip is None now hit the gate and inherit the fail-closed behaviour.
* fix(responses): map chat tool_choice to Responses API when bridging from completions
OpenAI /v1/responses rejects tool_choice.function. Normalize forced-function
choice from chat shape to {type, name} in LiteLLMResponsesTransformationHandler.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(responses): strip tool_choice.function when top-level name is set
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Anthropic rejects tool_use/tool_result when tools is omitted. Always map
and attach the dummy tool in transform_request so CLIs work without
litellm.modify_params.
- Add unit test for transform_request dummy tool with modify_params off
- Adjust parallel function calling integration expectations: Bedrock
Converse still requires modify_params for this path
Co-authored-by: Cursor <cursoragent@cursor.com>
Reject fnmatch wildcards on non-scope claims when the claim string contains
whitespace so malformed iss values cannot match patterns like trusted.*.
Merge every entry when team_id_jwt_field resolves to a list instead of
keeping only the first element.
Co-authored-by: Cursor <cursoragent@cursor.com>
- Extend responses_api_bridge_check when reasoning_effort + summary aliases
(including nested extra_body) without tools
- Merge summary into reasoning_effort for responses bridge; helpers in utils
- Strip summary aliases in GPT-5 chat mapping when not bridged
- Tests for bridge + merge behavior
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(mcp): forward extra_headers for OpenAPI MCP tools
OpenAPI-generated tools only applied static closure headers and BYOK
Authorization via ContextVar. Copy MCPServer.extra_headers from the
incoming MCP request into _request_extra_headers (set in server.py before
local tool dispatch), merge in openapi_to_mcp_generator via a small helper.
OAuth2 M2M: do not forward caller Authorization from raw_headers (same rule
as _prepare_mcp_server_headers for managed MCP).
Adds TestRequestExtraHeaders and clarifies mcp_server_manager registration
comment.
Fixes#26794
Co-authored-by: Cursor <cursoragent@cursor.com>
* refactor(mcp): access has_client_credentials on MCPServer directly
Greptile: getattr default was redundant; property exists on MCPServer and
mcp_server is non-None inside the extra_headers forwarding block.
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
The previous detection treated any model with input_cost_per_image
or output_cost_per_image as image generation. Several chat and
embedding models carry those fields to price multimodal vision input,
not generated images:
- gemini-3.1-pro-preview (mode=chat) has output_cost_per_image=0.00012
alongside input/output token pricing.
- azure/gpt-realtime-* (mode=chat) has input_cost_per_image=5e-6.
- amazon.titan-embed-image-v1 (mode=embedding) has
input_cost_per_image=6e-5.
For these models the image-gen branch fired first and reserved a
fraction of a cent per request, short-circuiting the token-priced
path entirely. Long Gemini chats reserved 1 × $0.00012 instead of
the true token cost.
Gate strictly on mode in {"image_generation", "image_edit"}. All 197
real image_generation entries and all 31 image_edit entries
(Flux Kontext, Stability inpaint/outpaint, etc.) carry the right mode,
so the field-presence fallback was unnecessary.
Adds regression tests for the chat-model-with-image-cost-field case
and for image_edit reservation.
Image-generation routes (dall-e-3, flux, etc.) have no per-token output
cost so they fell through to the no-reservation read-time-only path.
Concurrent image requests against a depleted budget could all pass
common_checks (counter exactly at max_budget passes the strict-`>`
gate) and reach the provider before reconciliation caught up.
Add per-image reservation in _estimate_request_max_cost_for_model:
when the model has a per-image cost field, reserve `n × cost_per_image`
upfront. The atomic counter increment serializes concurrent admissions,
so the second request sees the post-first-reservation counter and
raises BudgetExceededError instead of silently leaking through.
Both `output_cost_per_image` and `input_cost_per_image` are honored —
naming is inconsistent across providers (OpenAI dall-e-3 uses
input_cost_per_image, aiml/dall-e-3 uses output_cost_per_image for
the same per-generated-image price).
Per-pixel pricing (DALL-E 2 size variants) and TTS/STT routes still
fall through to read-time enforcement; those are follow-ups.
- Introduce RoutingPrismaWrapper that transparently routes read operations (find_*, count, group_by, query_raw, query_first) to a reader endpoint while writes remain on the writer, enabling Aurora-style reader/writer endpoint splits
- Add IAMEndpoint dataclass and parse_iam_endpoint_from_url() to capture static connection fields from a reader URL so only the IAM token needs to rotate, avoiding the need for separate DATABASE_HOST_READ_REPLICA/etc. env vars
- Enhance PrismaWrapper with per-instance knobs (db_url_env_var, iam_endpoint, recreate_uses_datasource, log_prefix) so writer and reader wrappers are independent: the reader writes its fresh URL to DATABASE_URL_READ_REPLICA and passes datasource override to Prisma since Prisma only auto-reads DATABASE_URL
- Fix deadlock in PrismaWrapper.__getattr__: when called from inside a running event loop, schedule the token refresh as a background task instead of blocking with run_coroutine_threadsafe + future.result(), which would deadlock the loop thread waiting for a coroutine that needs the loop to run
- Fix botocore crash when DATABASE_PORT is unset by defaulting to "5432" in both proxy_cli.py and PrismaWrapper.get_rds_iam_token(); passing None caused botocore to embed the literal string "None" in the presigned URL
- Implement graceful reader degradation: reader connect/recreate failures are non-fatal; wrapper sets _reader_unavailable=True and silently routes reads to the writer to keep the proxy serving traffic during transient reader outages
- Add PrismaClient.writer_db property so the reconnect smoke-test always validates the writer engine specifically; query_raw on the routing wrapper would route to the reader and not verify the newly-recreated writer
- Expose DATABASE_URL_READ_REPLICA in Helm chart (values.yaml + deployment.yaml) via both plain value and secret key reference, and document the field in docker-compose.yml
- Add 887-line test suite covering routing logic, IAM token refresh paths, reader degradation scenarios, datasource override behavior, and the deadlock regression
Co-authored-by: Yassin Kortam <yassinkortam@g.ucla.edu>
reserve_budget_for_request fell back to reserving the entire remaining
team/key/user headroom whenever a request omitted max_tokens, which
pinned the spend counter at max_budget for the duration of the
in-flight request and false-positive-blocked every concurrent or
back-to-back request until the success callback reconciled. Surfaced
as an integration-test team being budget-blocked at its $2000 cap
while DB spend was $0.144.
Switch the missing-max_tokens path to a fixed default of 16384 output
tokens (mirrors parallel_request_limiter_v3's DEFAULT_MAX_TOKENS_ESTIMATE
precedent), and clamp explicit max_tokens at the model's
max_output_tokens for reservation accounting only. The outbound request
body is unchanged, so providers see whatever the caller actually sent;
only the local integer used to compute reservation cost is bounded.
This also prevents a hostile max_tokens=999999999 from inflating one
request's reservation up to the entire team headroom.
For Opus 4.7 (output $25/M, max_output 128K) on a $2000 budget the
worst-case per-request reservation drops from "everything left" to
$3.20, raising admittable concurrency from 1 to ~625.
