253 commits

Author	SHA1	Message	Date
ryan-crabbe-berri	90b5104475	feat(mcp/auth): additive key access-group grants + opt-in member assignment (#29313) ... * fix(mcp): make key.access_group_ids grants additive over team ceiling A key whose unified access_group_ids grant a private MCP server was having that grant intersected against its team's MCP ceiling, so a key in a team scoped to other servers (or with no own scope) lost the granted server entirely. Resolve access_group_ids once as ungated additive grants and union them on top of the key/team ceiling instead of folding them into the key scope that gets intersected. * test(mcp): align key access-group tests with additive-grant model The previous commit moved key.access_group_ids resolution out of the intersected key ceiling (_get_allowed_mcp_servers_for_key) and into the ungated additive grant path (_get_key_access_group_mcp_server_extras), unioned on top of the team ceiling. Five tests from #28890/#29195 still asserted the old gated / in-key-scope contract and failed: - _get_allowed_mcp_servers_for_key now returns the object_permission ceiling only and never resolves access_group_ids; two tests now assert the group resolver is not called from that path (with and without an object_permission present). - The extras path is ungated, so a group whose assigned_team_ids / assigned_key_ids exclude the caller still contributes its servers. - The end-to-end test asserts the grant surfaces via the extras path rather than the base key path. - Dropped test_key_access_group_ids_empty_returns_no_extras; the empty case is already covered by the extras family's no-groups test. * feat(auth): gate member access-group assignment on keys behind opt-in Non-admin team members could attach access_group_ids to keys they create or update, letting them self-grant resources (MCP servers/models) the team admin never intended. Add an opt-in KEY_ACCESS_GROUP_ASSIGNMENT team-member permission (default-deny) enforced at /key/generate and /key/update; proxy and team admins bypass. Surfaces automatically as a checkbox in the team Member Permissions UI. * fix(auth): gate access-group assignment on /key/regenerate too RegenerateKeyRequest inherits access_group_ids and prepare_key_update_data persists it, so a non-admin key owner could self-grant access groups by regenerating. Apply the same opt-in member gate using the existing key's team. * test(auth): cover member access-group gate and additive MCP grants Add unit tests for enforce_member_can_assign_access_groups (deny without opt-in, allow with opt-in, and proxy-admin / team-admin / non-team-key bypasses) and for _get_key_access_group_mcp_server_extras (no-auth and no-resolved-servers return empty, resolved ids are expanded, errors degrade to no grants).	2026-05-30 17:35:31 -07:00
yuneng-jiang	94a043efb2	refactor(proxy/auth): normalize Bearer prefix in safe-hash helper (#29343) ... * refactor(proxy/auth): normalize Bearer prefix in safe-hash helper UserAPIKeyAuth._safe_hash_litellm_api_key now strips a leading "Bearer "/"bearer " prefix before its existing sk-/JWT classification, so the helper produces the same hashed output regardless of whether the caller stripped the Authorization header prefix or passed the header value through unchanged. * refactor(proxy/auth): make Bearer-prefix strip case-insensitive Per RFC 7235 the HTTP authorization scheme token is case-insensitive. Replace the two-prefix loop with a single case-insensitive check so the helper normalizes "Bearer ", "bearer ", "BEARER ", and any mixed-case variant before classifying the remainder as sk- or JWT. The contract test gains coverage of "BEARER " and "BeArEr ". * test(mcp): align auth-handler test expectations with safe-hash helper The two MCP auth tests asserted that UserAPIKeyAuth(api_key="Bearer ...") retained the raw header bytes on the api_key field. _safe_hash_litellm_api_key now normalizes that input — stripping the Bearer prefix and hashing the resulting sk- key — so the expectations move to the normalized form: the bare token in the parametrize case, and hash_token("sk-...") in the backward-compat assertion. This matches what the real auth flow produces (the builder strips Bearer and the DB stores the hashed token), so the mocks now line up with production rather than with the un-normalized validator output.	2026-05-30 14:04:22 -07:00
Shivam Rawat	2d4c13c00f	fix(mcp): preserve omitted fields on PUT /v1/mcp/server partial updates (#29253) ... * fix(mcp): preserve omitted fields on PUT /v1/mcp/server partial updates Use model_dump(exclude_unset=True) for updates so schema defaults (transport=sse, allow_all_keys=false, etc.) are not written when callers omit them. Serialize JSON fields from the filtered dict and only force is_byok on create. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(mcp): allow explicit alias=None on MCP server partial updates Snapshot caller-provided fields before normalization so omitted alias is not written while an intentional alias=None still clears the stored value. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Cursor <cursoragent@cursor.com>	2026-05-30 12:03:22 -07:00
Shivam Rawat	ace3c65ab3	fix(mcp): preserve source_url in GET /v1/mcp/server list responses (#29249) ... * fix(mcp): preserve source_url in GET /v1/mcp/server list responses The list endpoint builds responses from the in-memory registry, but source_url was dropped during the DB-to-registry roundtrip even though GET /v1/mcp/server/{id} returned it correctly from the database. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(tests/mcp): set source_url on MagicMock table records MagicMock auto-creates source_url as a mock object, which fails MCPServer Pydantic validation after source_url was wired through build_mcp_server_from_table. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Cursor <cursoragent@cursor.com>	2026-05-30 12:01:22 -07:00
Sameer Kankute	c792df64d2	feat(mcp): support stateless and stateful clients via session-id routing (#26857) ... * feat(mcp): support stateless and stateful clients via session-id routing - Add session_manager_stateful (stateless=False) alongside stateless - Route by mcp-session-id: has ID → stateful, initialize (no ID) → stateful, else → stateless - Peek POST body to detect initialize for routing; replay via wrapped receive - Handle stale session IDs for both managers - Add test_mcp_routing_initialize_to_stateful_no_session_to_stateless - Update test_valid_mcp_session_id_is_preserved, test_concurrent_initialize_session_managers Made-with: Cursor * fix(mcp): respect stateful routing and harden initialize detection Ensure streamable MCP requests are dispatched via the computed target session manager, and guard initialize detection against non-object JSON bodies. Update stale-session test patches to target the stateful manager so routing assertions remain correct. Made-with: Cursor * test(mcp): patch stateless/stateful managers in concurrency init test Update concurrent session-manager initialization test to patch session_manager_stateless and session_manager_stateful directly, matching initialize_session_managers() behavior and preventing NameError from undefined mocks. Made-with: Cursor * Fix tests * Fix tests * Fix MCP stateful routing edge cases * Fix stateful MCP auth context refresh * Fix MCP stateful session cleanup * fix(mcp): bind stateful sessions to creator and reject hijacks Stateful mcp-session-id was usable by any authenticated proxy caller. Track the session creator's hashed API key (or user_id) when a new session is issued and reject mismatched callers with 403 before _set_or_update_auth_context overwrites the stored MCPAuthenticatedUser. Also formats nested with-statements in test_mcp_stale_session.py and fixes a pre-existing AsyncMock mismatch in test_stale_mcp_session_id_is_stripped. * fix(mcp): serialize concurrent requests on same stateful session Bugbot's 'Concurrent requests share context' finding: _update_auth_context mutates the single MCPAuthenticatedUser stored per session in place on every request, so two requests sharing one mcp-session-id can overwrite each other's mcp_servers / auth headers / oauth state / client_ip while in-flight callbacks are still reading the same object. Owner-binding alone narrows this to same-principal racing, but the in-place mutation race remains. Add a per-session asyncio.Lock around handle_request so concurrent same-session requests run sequentially. The lock is allocated on demand and torn down with the rest of the session state on DELETE / idle expiry. Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> * fix(mcp): include OAuth2 bearer in stateful session owner fingerprint UserAPIKeyAuth() for OAuth2 passthrough has no api_key/user_id, so every OAuth caller fingerprinted to "anonymous" and could hijack another OAuth caller's mcp-session-id. Hash the upstream Authorization header into the fingerprint as oauth:<sha256>. * fix(mcp): don't hold stateful session lock for streaming GETs The per-session lock wraps handle_request, so a long-lived GET (SSE stream held open for the life of the session) would block every subsequent POST on the same mcp-session-id. Only POST/DELETE mutate the shared MCPAuthenticatedUser, so it's sufficient to serialize those — GETs run lock-free and stream concurrently. * fix(mcp): allow None user_api_key_auth in MCPAuthenticatedUser The set_auth_context / _set_or_update_auth_context / _update_auth_context helpers in server.py all accept Optional[UserAPIKeyAuth] and pass it straight into MCPAuthenticatedUser, but the dataclass-style constructor typed user_api_key_auth as required UserAPIKeyAuth. Mypy flagged this on the stateful-routing branch: server.py:3227: error: Incompatible types in assignment (expression has type "UserAPIKeyAuth | None", variable has type "UserAPIKeyAuth") server.py:3255: error: Argument "user_api_key_auth" to "MCPAuthenticatedUser" has incompatible type "UserAPIKeyAuth | None"; expected "UserAPIKeyAuth" Widen the parameter type to Optional[UserAPIKeyAuth] to match the call sites. Runtime behavior is unchanged. Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> * style: replace with new alias * fix(mcp): fall back to client_ip in stateful session owner fingerprint Addresses Greptile review on PR #26857: when no API key, user_id, or OAuth bearer is available (e.g. unauthenticated/passthrough callers), the owner fingerprint collapsed to a single 'anonymous' value, allowing two unrelated callers to drive each other's stateful MCP sessions. Fold client IP into the fingerprint as a fallback identity signal so distinct anonymous sources do not share an owner identity. Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> * Fix active stateful MCP session cleanup * test(mcp): cancel leaked stateful auth-context cleanup task initialize_session_managers() spawns a real asyncio.create_task running _cleanup_expired_stateful_session_auth_contexts(). The test_concurrent_initialize_session_managers test was saving and restoring the session-manager context-manager globals but did not save, cancel, or restore _stateful_auth_context_cleanup_task. Because pyproject.toml sets asyncio_default_fixture_loop_scope=session, the event loop is shared across tests in the same session, so the leaked task kept running against module-level dicts for the rest of the test run. Save and cancel the task in the finally block so the test fully cleans up after itself. Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> * Fix stateful MCP session fingerprinting * Hash MCP session user owner fingerprints * Fix stale MCP session DELETE cleanup * fix(mcp): harden owner fingerprint hashing for non-str api keys _owner_fingerprint_for assumed api_key/user_id supported .encode(); MagicMock-based tests (and any non-str truthy values) crashed with TypeError before routing. Only hash str/bytes secrets; fall through otherwise so MCP routing and session tests behave correctly. Co-authored-by: Cursor <cursoragent@cursor.com> * Fix MCP stateful cleanup loop resilience * Fix stateful MCP initialize auth capture * fix(mcp): drop orphan per-session lock when auth context absent Defensive cleanup for _stateful_session_locks entries created on sessions that never enter _stateful_session_auth_contexts. The periodic cleanup loop only iterates auth_context_last_seen, so such locks would otherwise live forever. Add a test that reproduces the leak and verifies the request finalizer pops the lock. Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> * chore(mcp): trim verbose comment on lock cleanup Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> * Fix stateful MCP delete failure tracking * fix test * fix(mcp): cap routing-peek body size to bound pre-dispatch memory Authenticated clients that POST without an mcp-session-id forced the proxy to buffer the entire request body before routing, since the peek loop drained every body chunk to decide whether the JSON-RPC method was 'initialize'. Cap the peek at 4 KB (more than enough for an initialize envelope) and let the remainder stream through wrapped_receive into the downstream handler. * test: replace dall-e-3 with gpt-image-1 in health check and router tests (#27813) OpenAI returns 'The model dall-e-3 does not exist' for the test account, breaking test_openai_img_gen_health_check and test_image_generation. Switch to gpt-image-1, matching the existing TestOpenAIGPTImage1 pattern. * fix(tests): drop dall-e-only test classes; route live image tests via gpt-image-1 Second wave of failures from the 2026-05-12 DALL-E shutdown: - tests/image_gen_tests/test_image_edits.py::TestOpenAIImageEditDallE2 and tests/image_gen_tests/test_image_generation.py::TestOpenAIDalle3 are explicitly named for the deprecated models and can't pass; remove. gpt-image-1 coverage already exists in sibling classes. - tests/local_testing/test_router.py image gen tests use dall-e-3 only as a routing example; swap to gpt-image-1. - tests/local_testing/test_custom_callback_input.py image_generation success/failure paths swapped to gpt-image-1. * Fix MCP initialize session active tracking Co-authored-by: Yassin Kortam <yassin@berri.ai> * Fix MCP reinitialize session tracking Co-authored-by: Yassin Kortam <yassin@berri.ai> * Fix MCP reinitialize auth context aliasing Co-authored-by: Yassin Kortam <yassin@berri.ai> * Apply black formatting after merge Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> * Run owner-binding 403 before consuming POST body Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> * Harden MCP routing peek bound and stateful purge race Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> * Remove inadvertently committed Next.js build artifacts Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> * Run owner check before stale MCP session cleanup Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> * fix(mcp): reverse cleanup ordering to terminate transport before clearing owner Reverses _purge_expired_stateful_session_auth_contexts so the transport is popped from server_instances and terminated BEFORE owner/auth tracking is cleared. The previous order left a window where _stateful_session_owners was already empty but server_instances still served the session, so a concurrent request would observe expected_owner is None and bypass the owner-binding check. Addresses Greptile review on PR #26857. Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> * test(mcp): fully reset stateful session tracking in auth-context refresh test Use _remove_stateful_session_tracking in teardown so the test no longer leaks _stateful_session_auth_context_last_seen and _stateful_session_locks between tests, matching the cleanup used by the sibling stateful tests. * fix(mcp): cap concurrent stateful sessions per caller to bound memory --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> Co-authored-by: Sameerlite <sameerlite@users.noreply.github.com> Co-authored-by: yuneng-jiang <yuneng@berri.ai> Co-authored-by: Yassin Kortam <yassin@berri.ai> Co-authored-by: Claude Babysitter <claude@anthropic.com> Co-authored-by: mateo-berri <mateo@berri.ai>	2026-05-29 20:30:36 -07:00
ryan-crabbe-berri	909a5f597a	fix(mcp): resolve key.access_group_ids → MCP servers (ungated) (#29195) ... * fix(mcp): resolve key.access_group_ids → MCP servers (ungated) A teamless virtual key whose unified access_group_ids grant an MCP-granting access group now sees and can call that server instead of getting an empty list / 403. The key path previously read only the legacy object_permission; this folds key.access_group_ids into the key's base scope (ungated), mirroring can_key_call_model's fallback. The gated assigned_*-checked override is unchanged. * fix(mcp): expand name/alias entries in key access-group servers access_mcp_server_ids may hold server names/aliases, not just ids. The new ungated key path now runs them through expand_permission_list at the source, so the early-return and union branches both surface resolved ids — matching the legacy object_permission path and the gated extras path.	2026-05-29 15:07:12 -07:00
ryan-crabbe-berri	ea015332d8	fix(mcp): resolve team.access_group_ids → MCP servers (#28997) ... * fix(mcp): resolve team.access_group_ids → MCP servers A virtual key whose team has an MCP-granting access group attached via /v1/access_group now sees that server through /v1/mcp/server (and can call tools on it) instead of getting an empty list. The runtime already resolves the key's unified access_group_ids; this adds the symmetric resolution on the team side, mirroring the model-side pattern in can_team_access_model — the group being on the team is itself the gate, so no assigned_team_ids re-check is needed. Resolves #27657 * chore(mcp): address greptile review on team access-group resolver Forward already-imported prisma_client / user_api_key_cache / proxy_logging_obj to _get_mcp_server_ids_from_access_groups so it skips its lazy re-import path. Update test docstring + assertions to reflect that the resolver is invoked with [] (and short-circuits without DB access) rather than skipped entirely.	2026-05-27 12:36:50 -07:00
ryan-crabbe-berri	123a9ce487	fix(mcp): extend key access-group union to MCP servers (#28890) ... A key's unified access_group_ids now extend the team's MCP scope instead of being capped by it — mirrors the model-side union from LIT-2404. The group's assigned_team_ids / assigned_key_ids still gate the override, so team members can't pull in MCPs via a foreign team's group. Resolves LIT-3189	2026-05-26 12:55:06 -07:00
Sameer Kankute	d52fbfb458	Litellm oss staging 250526 (#28770) ... * fix(mcp): handle OAuth IdP error responses in /callback (LIT-2750) Per RFC 6749 section 4.1.2.1, when the IdP rejects an OAuth authorization request it redirects back to the client with ?error=...&error_description=... and no code. The MCP /callback handler declared code and state as required query params, so FastAPI rejected such error responses with a 422 before the handler ran -- stranding the MCP client waiting on the loopback. This change: - Makes code and state optional and accepts the RFC-defined error, error_description, and error_uri params. - When state decodes to a trusted client redirect_uri, propagates the error params back to that URI with the client's original (un-wrapped) state preserved, so the client's OAuth library can surface the failure. - When state is missing/undecryptable or the encoded redirect_uri is no longer trusted, renders a 400 HTML page with the (HTML-escaped) error details instead of leaking to an attacker-controlled redirect. - Preserves the existing success path (code + state -> 302 to validated client redirect_uri with original state). Fixes LIT-2750. * test(mcp): regression tests for /callback handling IdP error responses (LIT-2750) Adds a new test module covering the LIT-2750 fix: the MCP OAuth /callback endpoint must accept IdP error responses (e.g. ?error=access_denied) per RFC 6749 section 4.1.2.1 instead of returning a 422 because ``code`` is missing. Coverage: - IdP error with no state -> 400 HTML page surfacing the error. - HTML escaping of user-controlled error / error_description fields. - IdP error with a trusted (loopback) state -> 302 propagating error / error_description / original client state to the client. - IdP error with an untrusted redirect_uri encoded in state -> 400 inline (no open-redirect to attacker-controlled origin). - IdP error with an undecryptable state -> 400 HTML fallback. - Bare GET /callback with no params -> 400 HTML (not Pydantic 422). - Success path (code + state) still 302 to validated client redirect_uri with the original (un-wrapped) state preserved. * refactor(mcp): drop unused _OAUTH_ERROR_PARAMS constant (Greptile P2) The tuple was leftover scaffolding from an earlier draft of the LIT-2750 fix; nothing references it. The explanatory RFC 6749 §4.1.2.1 comment block above the callback handler covers the same intent. * fix(mcp/oauth): preserve empty original_state and clarify missing-param error in /callback Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix(thinking): handle None thinking param in is_thinking_enabled (#28598) Squash-merged by litellm-agent from Terrajlz's PR. * feat(helm): support tpl rendering in podAnnotations (#28609) Squash-merged by litellm-agent from devauxbr's PR. * fix: apply black formatting to base_llm chat transformation Fix CI black --check failure on is_thinking_enabled return formatting. Co-authored-by: Cursor <cursoragent@cursor.com> * merge main (#28836) * fix(proxy): Bedrock Knowledge Base pass-through: preserve SigV4 headers and signed request body (#27526) * Fix Bedrock KB pass-through SigV4 headers and signed body Coerce botocore HeadersDict to a dict for pass-through routes. When forward_headers is true, drop request headers that collide case-insensitively with signed headers so client Bearer auth does not shadow AWS SigV4. Send prepped.body as raw content so the outbound payload matches the signature after logging hooks mutate the parsed dict. Co-authored-by: Cursor <cursoragent@cursor.com> * Simplify pass-through raw body handling Read the SigV4-signed bytes directly from request.state inside pass_through_request instead of threading a custom_raw_body argument through three functions. Helper methods are restored to their original signatures, and the new branch lives in one place at each httpx call site. Co-authored-by: Cursor <cursoragent@cursor.com> * Harden pass-through raw body read from request.state Guard missing request.state (test fixtures) and ignore non-bytes/str values so MagicMock does not trigger the SigV4 raw-body path. Co-authored-by: Cursor <cursoragent@cursor.com> * Test pass_through_request state_raw_body uses httpx content= Cover non-streaming (async_client.request) and streaming (build_request) paths so SigV4 bytes on request.state are not replaced by json= of a hook-mutated dict. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Cursor <cursoragent@cursor.com> * chore(tests): migrate Bedrock CI to AWS account 941277531214 (#28728) * chore(tests): migrate Bedrock CI from AWS account 888602223428 to 941277531214 The original account (888602223428) was put under a security restriction by AWS after a root access key leaked in a PR comment. While that account works its way through the AWS Support unlock process, Bedrock-touching CI tests have been migrated to a fresh account (941277531214). Changes: - Replace 26 hardcoded references to 888602223428 with 941277531214 across 8 files (provisioned-model ARNs, imported-model ARNs, AgentCore runtime ARNs, batch execution role ARN, and example proxy config). - The provisioned-model and imported-model ARNs are referenced only from mocked unit tests — no AWS resources to recreate. - The batch execution IAM role has been recreated in the new account with the same name and equivalent permissions. - The two AgentCore runtimes (hosted_agent_r9jvp-3ySZuRHjLC, hosted_agent_13sf6-cALnp38iZD) are being recreated in the new account under the same names — see tools/agentcore-deploy/ in a follow-up. CircleCI env vars AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY / AWS_REGION_NAME were updated separately via the CircleCI API to point at the new account. Smoke-tested locally against the new account: aws bedrock-runtime converse --region us-west-2 \ --model-id us.anthropic.claude-sonnet-4-5-20250929-v1:0 \ --messages '[{"role":"user","content":[{"text":"ping"}]}]' → 200, model returned 'pong' Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * chore(tests): refresh AgentCore ARN suffixes to match newly-deployed runtimes The first migration commit replaced just the account ID, but AgentCore auto-assigns a random 10-char suffix to every runtime on creation — we can't reuse the original suffixes (`3ySZuRHjLC`, `cALnp38iZD`) in the new account. Updated the AgentCore-runtime ARNs in the three files that reference real runtime IDs (not the mock-based unit-test ARNs). Deployed runtimes: arn:aws:bedrock-agentcore:us-west-2:941277531214:runtime/hosted_agent_r9jvp-Rq79QFC2fp arn:aws:bedrock-agentcore:us-west-2:941277531214:runtime/hosted_agent_13sf6-4046UzHSwy Both runtimes are status=READY and pass a smoke invoke: $ aws bedrock-agentcore invoke-agent-runtime --agent-runtime-arn ... --payload '{"prompt":"ping"}' → 200, {"result": "echo: ping"} The agent is a minimal echo (see /tmp/agentcore_deploy/agent.py for the deploy artifacts). Tests that only verify the SDK wiring will pass; if any test asserts on agent output content, swap the echo for the real agent. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * chore(tests): point Bedrock batch tests at new-account S3 bucket The account migration (888602223428 -> 941277531214) was a flat account-ID swap, which only rewrites ARNs that embed the account number. S3 bucket names carry no account ID, so the live Bedrock batch tests still uploaded to `litellm-proxy` — a bucket that lives in the old account. S3 names are globally unique, and the old account still holds that name, so it can't be recreated in the new account. Rename to `litellm-proxy-941277531214` (account-ID suffix guarantees global uniqueness). The bucket must be created in 941277531214 and the batch execution role granted s3:GetObject/PutObject/ListBucket on it before this job is run in CI. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(tests): point live S3 logging test at new-account bucket Same account-ID-free blind spot as the batch bucket: `load-testing-oct` lives in the old account and its name can't be reused globally. The `logging_testing` CI job is wired into the workflow and runs test_basic_s3_logging, which uploads to this bucket with the CI env creds, then lists and deletes objects — a live dependency. Rename to `load-testing-oct-941277531214`. The bucket must exist in the new account with the CI IAM principal granted s3:PutObject/GetObject/ListBucket/DeleteObject before this job runs. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(tests): repoint Bedrock guardrail IDs to new-account guardrails The migration left guardrail IDs untouched (no account ID in them), so all live guardrail tests failed with "guardrail identifier or version does not exist" against 941277531214. Recreated both guardrails in the new account and updated the hardcoded IDs: - wf0hkdb5x07f -> zgkmukebruil (PII mask: PHONE + CREDIT_DEBIT_CARD, with explicit inputAction=ANONYMIZE so masking applies to INPUT, which is the source litellm's moderation hook sends) - ff6ujrregl1q -> 4w3d1di3snt5 (blocks "coffee"; blocked message set to the exact string the tests assert on) Updated test_bedrock_guardrails.py, otel_test_config.yaml, and the guardrailConfig in test_bedrock_completion.py. Verified locally: the 5 previously-failing guardrail tests now pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(bedrock): migrate legacy models to current inference profiles The new CI account (941277531214) cannot invoke legacy Bedrock models (AWS gates them: "marked by provider as Legacy... not actively using in the last 30 days"). Migrated the live-call tests: - anthropic.claude-3-sonnet-20240229 -> us.anthropic.claude-sonnet-4-5-20250929-v1:0 - anthropic.claude-3-haiku-20240307 -> us.anthropic.claude-haiku-4-5-20251001-v1:0 Current Claude models on Bedrock require the us. inference-profile prefix (bare on-demand ids are rejected). cohere.command-r-plus has no working replacement (all Cohere is legacy- gated in the new account): swapped to claude-haiku-4-5 in provider- agnostic param lists. amazon.titan-image-generator skipped (no working replacement). Mocked/transformation/cost tests that reference the legacy strings are intentionally left unchanged. Verified live against the new account. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(bedrock): repoint SageMaker + Knowledge Base to new-account resources These referenced account-scoped resources by hardcoded id that only existed in the old account, so the migration's account-ID swap missed them. Recreated in 941277531214 and repointed: - SageMaker endpoint jumpstart-dft-hf-textgeneration1-mp-20240815-185614 -> litellm-ci-textgen (gpt2 on a TGI container, ml.g5.xlarge) - Bedrock Knowledge Base T37J8R4WTM -> LCYXFBR2TU (OpenSearch Serverless vector store + titan-embed-text-v2, seeded with a LiteLLM doc) Verified live: test_sagemaker.py (12 passed) and test_bedrock_knowledgebase_hook.py (12 passed). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(reasoning_effort_grid): skip bedrock claude-opus-4-7 cells (not entitled on 941277531214) claude-opus-4-7 is listed in the new Bedrock CI account's foundation models but invoke is denied (AccessDeniedException: "not available for this account"). Bedrock access to the flagship Opus requires an AWS Sales request, not the self-serve model-access toggle, so it can't be enabled inline with the rest of the account migration. Add an optional `skip_reason` to ModelEntry and set it on the bedrock-claude-opus-4-7 entry; the grid test honors it via pytest.skip. Cell count (231) and route coverage are unchanged, so the structural asserts still pass. Restore coverage by deleting the one skip_reason line once access is granted. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(bedrock): swap/skip legacy-gated models unavailable on new CI account The migrated AWS account (941277531214) cannot access several models that the old account could, so the remaining red CI jobs were hitting real Bedrock "Access denied / Legacy" and "account not authorized" errors: - image_gen: skip both Nova Canvas test classes (amazon.nova-canvas-v1:0 is legacy-gated), matching the existing titan skip. - batches: skip test_async_file_and_batch (Bedrock batch inference is not authorized on the new account; requires an AWS support case). - litellm_overhead: swap legacy claude-3-5-haiku for the active us.anthropic.claude-haiku-4-5 inference profile. - test_completion_claude_3_function_call: swap legacy claude-3-sonnet for the active us.anthropic.claude-sonnet-4-5 inference profile. https://claude.ai/code/session_01Y7zgHYu9GX29YRwV4yiWAa * test(bedrock): fix remaining e2e legacy-model + batch failures on new CI account - e2e_openai_endpoints: skip test_bedrock_batches_api (Bedrock batch inference is not authorized on account 941277531214) and migrate the missed s3_bucket_name in oai_misc_config.yaml to litellm-proxy-941277531214. - build_and_test: swap legacy bedrock claude-3-sonnet for the active us.anthropic.claude-sonnet-4-5 inference profile in the proxy structured output e2e test. https://claude.ai/code/session_01Y7zgHYu9GX29YRwV4yiWAa * test(bedrock): make opus-4-7 + batch cells fail loudly and mock image-gen (#28791) Replace the silent skips added for the new CI account with noisier behavior: - reasoning-effort grid: opus-4-7 cells now fail (when AWS creds are present) instead of skipping, so the missing entitlement stays visible in CI; they still skip when AWS creds are absent (local dev) - Bedrock batch inference tests: drop the skip so they run and fail until batch access is granted - Titan + Nova Canvas image-gen tests: mock the Bedrock HTTP call so the transform + cost-tracking path stays under test without live model access https://claude.ai/code/session_01MT7SWDnXUjv6e6EPG7BDjT Co-authored-by: Claude <noreply@anthropic.com> * test(bedrock): use pytest.xfail for known-failing opus-4-7 cells Replace pytest.fail with pytest.xfail when a model has a fail_reason, so known-broken cells stay visible as XFAIL without keeping CI red. Co-authored-by: Yassin Kortam <yassin@berri.ai> --------- Co-authored-by: Mateo <mateo@Mateos-MacBook-Pro.local> Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix(otel): export SERVER span on management-endpoint success without http_request (#28794) Co-authored-by: Yassin Kortam <yassinkortam@Yassins-MacBook-Pro.local> * chore(ci): merge dev branch (#28801) * chore(proxy): route path-dependent call sites through get_request_route Replace direct ``request.url.path`` reads in auth, ACL, routing, and audit-log decisions with ``get_request_route(request)`` — the helper already added in ``auth/auth_utils.py`` that returns the ASGI ``scope["path"]`` with ``root_path`` stripped. Starlette reconstructs ``url.path`` from the Host header; ``scope["path"]`` is uvicorn's parse of the request line and matches what FastAPI dispatches on, so it's the authoritative route for any decision that should agree with the actual handler. Sites: - _experimental/mcp_server/auth/user_api_key_auth_mcp.py - management_endpoints/mcp_management_endpoints.py - vector_store_endpoints/utils.py - pass_through_endpoints/pass_through_endpoints.py - auth/route_checks.py - litellm_pre_call_utils.py - spend_tracking/spend_management_endpoints.py - common_utils/http_parsing_utils.py - management_helpers/utils.py - health_endpoints/_health_endpoints.py Adds regression tests in tests/proxy_unit_tests/test_proxy_routes.py that construct a Request with scope["path"] set to a benign route and the Host header crafted so url.path would resolve differently; each site's decision is asserted against scope["path"]. * chore(proxy): make get_request_route imports lazy at call sites Move the ``from litellm.proxy.auth.auth_utils import get_request_route`` imports added in the prior commit back to the function bodies that use them. The module-level form participates in a long-standing import cycle through ``auth_utils -> _types -> ...`` and was flagged by CodeQL on the PR; the lazy form matches the pattern the proxy already uses for ``user_api_key_auth`` and related helpers elsewhere in these files. Also drop the ``RouteChecks._is_assistants_api_request`` delegation in ``_get_metadata_variable_name`` introduced in the prior commit — the delegation pulled ``RouteChecks`` into the same cycle, and the call site reuses the resolved route for its other branches, so inlining the substring check is both cycle-free and avoids a redundant second ``get_request_route`` call. Comment in test_proxy_routes.py acknowledges that the two MCP table entries exercise ``get_request_route`` directly rather than the full production handler (which needs ASGI scope + MCP state to invoke). --------- Co-authored-by: shin-berri <shin-laptop@berri.ai> Co-authored-by: user <70670632+stuxf@users.noreply.github.com> * chore(ci): merge dev branch (#28657) * feat(dashboard): navbar hierarchy + Agent Platform notifications (#27543) * feat(dashboard): refine navbar zones and Agent Platform notice Restructure the admin navbar for production users: clear product vs community vs personal columns with vertical dividers, icon-only Slack/GitHub in a shared chip, and Docs/Blog typography aligned on an 8px rhythm. Add a notifications bell with popover linking to the LiteLLM Agent Platform repo and optional mark-as-read persistence. Promote the account control with initials avatar, single-line display name, and navDisplayName mapping for placeholder user ids (e.g. default_user_id). Co-authored-by: Cursor <cursoragent@cursor.com> * fix(dashboard): address PR review — AntD buttons, public page guard, dedupe regex - Replace raw <button> with AntD Button in BlogDropdown, NotificationsBell, UserDropdown, and test mock - Guard NotificationsBell + container behind !isPublicPage to avoid rendering on public pages - Remove redundant equality checks in navDisplayName (regex already covers them) - Remove unused `lower` variable after simplification Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: yuneng-jiang <yuneng@berri.ai> * fix(dashboard): drop dead useHealthReadiness import in navbar The module was removed in #27896 (replaced by useHealthReadinessDetails), but the import survived the rebase. The symbol is unused — only useHealthReadinessDetails is consumed in the file. Removing the dead import unblocks the UI TypeScript build. * fix(dashboard): align CommunityEngagementButtons test with icon-only aria-labels The component was refactored to an icon-only chip with aria-label='LiteLLM on GitHub' (squash #27543), but the test still asserted /star us on github/i. Update the query to match the rendered accessible name. * refactor(dashboard): drop unused props from NavbarProps The navbar refactor moved user identity + dark-mode state to internal hooks (useAuthorized, useWorker), but the NavbarProps interface still declared userID, userEmail, userRole, premiumUser, isDarkMode, and toggleDarkMode as required, forcing every caller to thread them through. Drop them from the interface and all four call sites (page.tsx, (dashboard)/layout.tsx, public_model_hub.tsx, navbar.test.tsx). Also shrinks the destructure in layout.tsx so the now-unused locals stop being pulled out of useAuthorized(). * refactor(dashboard): use useSyncExternalStore for NotificationsBell dismiss flag Reads/writes of the litellmHideAgentPlatformBanner key were done directly inside NotificationsBell via a useEffect + useState pair. Every other localStorage-backed flag in the dashboard (Disable ShowPrompts, DisableBouncingIcon, DisableShowNewBadge, DisableUsageIndicator, DisableBlogPosts) is wrapped in a useSyncExternalStore hook over localStorageUtils so all mounted components stay in sync. Extract useHideAgentPlatformBanner to follow the same shape, swap NotificationsBell to consume it, and add a regression test that two sibling bells stay in sync without a remount when one is dismissed. * refactor: mask credential fields in proxy settings GET responses (#28682) * refactor: mask credential fields in proxy settings GET responses Brings SSO settings, cache settings, and the email/Slack alerting view in /get/config/callbacks in line with the HashiCorp Vault config-override pattern, so persisted credentials are not transported back to the UI in plaintext. * refactor: harden short-value masking and hoist alerting var constant Closes two review observations: - mask_sensitive_keys now replaces short values (below the visible prefix+suffix length) with an all-mask string instead of returning them unchanged, so a 1-7 character credential is no longer round-tripped verbatim. - _ALERTING_SENSITIVE_VARS is moved out of get_config() to a module-level constant, matching the analogous _SSO_SENSITIVE_FIELDS and _CACHE_SENSITIVE_FIELDS in the SSO and cache endpoint files. --------- Co-authored-by: Krrish Dholakia <krrish+github@berri.ai> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * fix(ui): show 2-decimal precision for max_budget on key overview (#28809) The Key Info Overview tab's Spend card truncated sub-dollar budgets to "$0" because formatNumberWithCommas defaults to 0 decimals. The Settings tab passes 2; align the overview so a $0.10 budget renders as "$0.10". Resolves LIT-2845 * feat(proxy): allow `llm_api_routes` virtual keys to list MCP servers (#28442) * feat(proxy): allow llm_api_routes virtual keys to list MCP servers Add a new `mcp_discovery_routes` group (GET /v1/mcp/server and GET /v1/mcp/server/{server_id}) and include it in `llm_api_routes` so that virtual keys configured with `allowed_routes=["llm_api_routes"]` can discover the MCP servers they have access to. Previously these calls failed with 'Virtual key is not allowed to call this route. Only allowed to call routes: [llm_api_routes]'. The GET handlers already sanitize the response for restricted virtual keys via `_sanitize_mcp_server_list_for_virtual_key`, stripping credential-bearing fields (url, headers, env). Write methods (POST/PUT/DELETE) on the same paths remain gated by the existing handler-level admin role checks. The new discovery list is intentionally kept OUT of `mcp_inference_routes`, so `is_llm_api_route()` still returns False for these paths — this preserves the existing contract that DISABLE_LLM_API_ENDPOINTS must not block the Admin UI from listing MCP servers. Co-authored-by: ryan-crabbe-berri <ryan-crabbe-berri@users.noreply.github.com> * refactor(proxy): make MCP discovery carve-out method-aware Replace the `mcp_discovery_routes` group in `llm_api_routes` with a method-aware special case inside `is_virtual_key_allowed_to_call_route`. Virtual keys with allowed_routes=["llm_api_routes"] are now permitted to call only GET /v1/mcp/server and GET /v1/mcp/server/{server_id} — non-GET methods and multi-segment admin sub-paths fall through to the existing 403. This keeps the general llm_api_routes list free of management paths and avoids accidentally exposing POST/PUT/DELETE writes through the route-check layer. --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: ryan-crabbe-berri <ryan-crabbe-berri@users.noreply.github.com> * chore(ci): merge dev branch (#28807) * chore(proxy): route path-dependent call sites through get_request_route Replace direct ``request.url.path`` reads in auth, ACL, routing, and audit-log decisions with ``get_request_route(request)`` — the helper already added in ``auth/auth_utils.py`` that returns the ASGI ``scope["path"]`` with ``root_path`` stripped. Starlette reconstructs ``url.path`` from the Host header; ``scope["path"]`` is uvicorn's parse of the request line and matches what FastAPI dispatches on, so it's the authoritative route for any decision that should agree with the actual handler. Sites: - _experimental/mcp_server/auth/user_api_key_auth_mcp.py - management_endpoints/mcp_management_endpoints.py - vector_store_endpoints/utils.py - pass_through_endpoints/pass_through_endpoints.py - auth/route_checks.py - litellm_pre_call_utils.py - spend_tracking/spend_management_endpoints.py - common_utils/http_parsing_utils.py - management_helpers/utils.py - health_endpoints/_health_endpoints.py Adds regression tests in tests/proxy_unit_tests/test_proxy_routes.py that construct a Request with scope["path"] set to a benign route and the Host header crafted so url.path would resolve differently; each site's decision is asserted against scope["path"]. * chore(proxy): make get_request_route imports lazy at call sites Move the ``from litellm.proxy.auth.auth_utils import get_request_route`` imports added in the prior commit back to the function bodies that use them. The module-level form participates in a long-standing import cycle through ``auth_utils -> _types -> ...`` and was flagged by CodeQL on the PR; the lazy form matches the pattern the proxy already uses for ``user_api_key_auth`` and related helpers elsewhere in these files. Also drop the ``RouteChecks._is_assistants_api_request`` delegation in ``_get_metadata_variable_name`` introduced in the prior commit — the delegation pulled ``RouteChecks`` into the same cycle, and the call site reuses the resolved route for its other branches, so inlining the substring check is both cycle-free and avoids a redundant second ``get_request_route`` call. Comment in test_proxy_routes.py acknowledges that the two MCP table entries exercise ``get_request_route`` directly rather than the full production handler (which needs ASGI scope + MCP state to invoke). --------- Co-authored-by: shin-berri <shin-laptop@berri.ai> Co-authored-by: user <70670632+stuxf@users.noreply.github.com> * fix(team): keep team_alias cache in sync on _cache_team_object writes (#28737) * fix(team): keep team_alias cache in sync on _cache_team_object writes _cache_team_object wrote only to the team_id:<id> cache key, but the JWT auth path that uses team_alias_jwt_field reads from a separate team_alias:<alias> key (get_team_object_by_alias caches under both keys on miss, but reads only the alias-keyed one). After any team-mutation endpoint (team_model_add, team_model_delete, update_team, the two access-group writes) the team_id cache was refreshed but the team_alias cache stayed stale until TTL — JWT callers using team_alias_jwt_field kept seeing the pre-mutation team for the full cache window. Mirror the write under the alias key inside _cache_team_object so every existing caller stays in sync without further changes. Skip the alias write when team_alias is None/empty so we don't collide across alias-less teams. Surfaced testing the LIT-3244 cherry-pick on patch/1.86.0: the LIT-3244 fix correctly invalidated the team_id cache but the customer's JWT used team_alias_jwt_field, so they kept hitting the stale alias-keyed entry. * fix(team): delete (not overwrite) team_alias cache on _cache_team_object The prior shape of this PR wrote both team_id:<id> AND team_alias:<alias> from _cache_team_object. team_alias is NOT unique in the schema (no @unique on LiteLLM_TeamTable.team_alias), and get_team_object_by_alias enforces uniqueness on its own DB-fetch path (len(teams) > 1 raises). Writing the alias-keyed cache from the generic refresh path bypassed that check: a team admin renaming their team to collide with another team's alias could silently overwrite the cached team for JWT-by-alias auth, swapping the resolved team under that alias for the cache window. Switch the alias-keyed operation from a write to a delete (mirroring the dual-cache delete pattern in _delete_cache_key_object). After every team write, the next JWT-by-alias reader cache-misses and falls through to get_team_object_by_alias, which (a) re-fetches the fresh team from DB, closing the LIT-3244 staleness gap that motivated this PR, and (b) enforces alias uniqueness before populating either cache key. team_id:<id> writes are unchanged — team_id is the table PK and is guaranteed unique. Surfaced in veria-ai review on #28739. * fix(managed-files): anchor model_id regex so it doesn't match llm_output_file_model_id extract_model_id_from_unified_id used `re.search(r"model_id,([^;]+)", ...)` which substring-matches the `model_id,` inside the file-ID encoding's `llm_output_file_model_id,<deployment_uuid>` field. parse_unified_id then fed that deployment UUID back into the auth path as a model candidate via _extract_models_from_managed_resource_id, and every team-BYOK file attach 403'd with: team not allowed to access model. This team can only access models=['openai/*']. Tried to access <deployment-uuid> The team's models list correctly contains the public name (`openai/*`) that target_model_names matches, but the bogus UUID candidate fails the wildcard check first. Anchor the regex to a field boundary (`(?:^|;)model_id,`) so it matches the legitimate top-level `model_id,<value>` field on vector_store unified IDs and skips substring matches inside other fields. File-IDs (which have no top-level `model_id` field) now return None and contribute no spurious UUID candidate. Surfaced reproducing LIT-3244 on patch/1.86.0 with the customer's exact flow: team with openai/* BYOK deployment, JWT-scoped user, POST /v1/vector_stores/{id}/files attaching a file uploaded with target_model_names=openai/gpt-4o. * fix(proxy): hydrate wildcard discovery credentials (#28284) (#28822) * fix(proxy): hydrate wildcard discovery credentials * fix(proxy): constrain wildcard credential hydration Co-authored-by: Dibyo Mukherjee <dibyo@adobe.com> * ci: add daily oss-agent-shin branch creation workflow (#28829) Creates litellm_oss_agent_shin_MM_DD_YYYY from main every day at 00:00 UTC. Lets us retarget oss-agent-shin fork PRs onto a canonical branch so CircleCI runs with secrets, without granting the agent write access. Co-authored-by: shin-berri <shin-laptop@berri.ai> Co-authored-by: yuneng-jiang <yuneng@berri.ai> Co-authored-by: Ishaan Jaffer <ishaanjaffer0324@gmail.com> * test(proxy): add harness for proxy_server.py behavior-pinning (#28827) * test(proxy): add harness for proxy_server.py behavior-pinning Creates tests/test_litellm/proxy/proxy_server/ with: - conftest.py: 11 shared fixtures (app, client, mock_prisma, auth_as, mock_router with parametrized response builders, normalize, etc.) - _coverage_check.py: per-PR coverage gate (line + branch) against a baseline, self-selects target by inspecting which placeholder files have been filled - _pin_check.py: AST-based gate that verifies every pin-list item has >=1 happy + >=1 error test with a real assertion (no status-only) - test_harness_smoke.py: 19 smoke tests covering every fixture + both scripts end-to-end - 26 placeholder test files (one docstring each) reserved for follow-up PRs per the directory ownership in the Notion plan - .coverage_baseline pinned at 0% so future PRs measure deltas against new-tests-only and aren't entangled with the broader scattered test suite Adds a dedicated proxy-server job to test-unit-proxy-endpoints.yml so this directory's runtime + coverage are tracked independently. Plan: https://www.notion.so/36c43b8acdab81ee845fd5365128a2fc * ci(proxy-endpoints): allow workflow_dispatch Lets the workflow be triggered manually on a branch via `gh workflow run`, which is needed for the verify-first flow on workflow changes before opening a PR. * test(proxy): address review feedback on proxy_server harness - conftest.py: anchor sys.path insert to __file__ (Path(__file__).resolve().parents[4]) instead of CWD-relative os.path.abspath("../../../../") which resolved to the wrong directory when pytest is launched from the repo root. - _coverage_check.py: actually read .coverage_baseline and use it as the floor (line_min = max(target, baseline)). Closes the gap between the PR description's "delta semantics" and what the script was doing. With baseline=0.0 today this is a no-op; future PRs that update the baseline cause regressions (test deletions etc.) to trip the gate even if the static PR target is still met. - _pin_check.py: drop unreachable startswith("_") guard (test_*.py glob never yields underscore-prefixed names) and read each test file once instead of twice. * feat(openai): apply regional-processing cost uplift for EU/US data residency (#28626) * feat(openai): apply regional-processing cost uplift for EU/US data residency OpenAI charges a 10% uplift on the latest GPT models when requests are served from a regionalized hostname (eu./us.api.openai.com). Infer the region from `api_base`, expose it on `kwargs["litellm_params"]["data_residency"]`, and multiply the computed cost by a per-model `regional_processing_uplift_multiplier_<region>` field. https://claude.ai/code/session_012ebH44s7ohYxjoix5CXzTW * test: allow regional_processing_uplift_multiplier_{eu,us} in model_prices schema * fix(cost): tighten data_residency inference and restore model_cost in tests - Only infer OpenAI data_residency when custom_llm_provider == "openai"; drop the implicit None fallback so non-OpenAI callers can't accidentally pick up a regional tag from a stray OpenAI hostname. - _local_model_cost_map fixture now snapshots and restores litellm.model_cost and LITELLM_LOCAL_MODEL_COST_MAP so tests don't leak state across the session. * refactor(openai): move data_residency helper under llms/openai * fix: thread data_residency through realtime stream cost calculation Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix(cost): thread data_residency through batch_cost_calculator Apply the OpenAI regional-processing uplift multiplier to retrieve_batch cost paths so Batch API requests served via eu./us.api.openai.com are priced at the same uplifted token rates as completions/transcriptions. * refactor(openai): encapsulate provider check inside infer_openai_data_residency Move the custom_llm_provider == "openai" guard from get_litellm_params into the helper itself so the core utility no longer carries provider-specific dispatch logic. Callers pass through the provider unconditionally; the helper returns None for any non-OpenAI provider. * fix(responses): thread data_residency through Responses logging params The Responses API paths build their logging litellm_params dict after provider resolution but did not include data_residency, so cost calc saw None even when the effective api_base was a regional OpenAI host. --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Yassin Kortam <yassin@berri.ai> --------- Co-authored-by: milan-berri <milan@berri.ai> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Mateo Wang <277851410+mateo-berri@users.noreply.github.com> Co-authored-by: Mateo <mateo@Mateos-MacBook-Pro.local> Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> Co-authored-by: Yassin Kortam <yassin@berri.ai> Co-authored-by: Yassin Kortam <yassinkortam@Yassins-MacBook-Pro.local> Co-authored-by: yuneng-jiang <yuneng@berri.ai> Co-authored-by: shin-berri <shin-laptop@berri.ai> Co-authored-by: user <70670632+stuxf@users.noreply.github.com> Co-authored-by: Krrish Dholakia <krrish+github@berri.ai> Co-authored-by: ryan-crabbe-berri <ryan@berri.ai> Co-authored-by: ryan-crabbe-berri <ryan-crabbe-berri@users.noreply.github.com> Co-authored-by: Dibyo Mukherjee <dibyo@adobe.com> Co-authored-by: ishaan-berri <155045088+ishaan-berri@users.noreply.github.com> Co-authored-by: Ishaan Jaffer <ishaanjaffer0324@gmail.com> * fix: preserve OTEL response payload and remove duplicate constant - _emit_management_endpoint_otel_span now passes result as response on success - remove duplicate _CREDENTIAL_LITELLM_PARAM_FIELDS assignment in model_checks Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix: address bug detection findings - pass_through_endpoints: use request.method instead of hardcoded POST in streaming SigV4-signed request path for consistency with the non-streaming branch - llm_cost_calc/utils: hoist DataResidency value set to a module-level frozenset to avoid rebuilding it on every cost calculation - example_config_yaml/oai_misc_config: replace real-looking AWS account ID with placeholder 123456789012 in example bucket and role ARN Co-authored-by: Yassin Kortam <yassin@berri.ai> * chore(github_copilot): refresh model catalog from upstream /models API (#28055) Aligns the github_copilot catalog with values returned by Copilot's public /models endpoint (capabilities.limits + capabilities.supports + model.supported_endpoints). - Adds 10 new model entries: claude-opus-4.7, claude-sonnet-4.6, gemini-3-flash-preview, gemini-3.1-pro-preview, gpt-4-0125-preview, gpt-5.2-codex, gpt-5.4, gpt-5.4-mini, gpt-5.5, oswe-vscode-prime. - Updates max_input_tokens for existing entries to reflect each model's true context window (e.g. gpt-4o-mini 64000 -> 128000, gpt-5-mini 128000 -> 264000, gpt-5.3-codex 128000 -> 400000, claude-haiku-4.5 128000 -> 200000). - Adds supports_reasoning, supports_response_schema, supports_function_calling, supports_parallel_function_calling, supports_vision based on capabilities.supports. - Declares supported_endpoints for entries missing it (e.g. gpt-3.5-turbo, gpt-4o, embeddings). - For responses-only models (gpt-5.2-codex, gpt-5.4, gpt-5.4-mini, gpt-5.5), sets mode to 'responses'. - gpt-41-copilot.mode changes from 'completion' to 'chat' because Copilot reports capabilities.type = 'chat'. Revertible on request. Pricing fields and other manually-curated values are preserved. * feat(datadog): emit litellm.overhead.latency as a standalone Datadog metric (#28831) Adds a new `litellm.overhead.latency` gauge metric to `DatadogMetricsLogger` (the `/api/v2/series` path). The value is sourced from `hidden_params["litellm_overhead_time_ms"]` already computed in `ResponseMetadata` and exposed in `StandardLoggingPayload`. Matches the Prometheus integration which exposes the same value via `litellm_overhead_latency_metric`. Emitted in seconds (ms ÷ 1000) for consistency with the other latency series. Co-authored-by: shin-berri <shin-laptop@berri.ai> Co-authored-by: yuneng-jiang <yuneng@berri.ai> Co-authored-by: Shin <shin@litellm.ai> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: ishaan-berri <155045088+ishaan-berri@users.noreply.github.com> * feat(arize): route Phoenix traces via per-project TracerProviders (#28876) Use LRU-cached TracerProviders with project-scoped OTEL Resources so team/key metadata routes traces correctly. On the proxy, project selection is limited to server-controlled user_api_key_auth_metadata; client metadata fields stay banned. * fix(arize_phoenix): skip _emit_semantic_logs on failure path Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix(arize_phoenix): skip raw request logging and metrics on failure path Restores pre-refactor behavior: _handle_failure no longer emits raw-request sub-spans or records OTEL metrics, matching the original _handle_failure that did not call these helpers. Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix(security): close two medium telemetry trust-boundary issues Issue 1 (arize_phoenix.py — caller-controlled telemetry routing): - _is_proxy_request no longer detects proxy mode by checking user_api_key_auth_metadata in request metadata. That field is user-supplied, so an authenticated caller could fake proxy-mode detection and have _project_from_metadata_dict read their own dict for project selection, routing telemetry to arbitrary Arize/Phoenix projects. Proxy mode is now determined solely by the server-set proxy_server_request field in litellm_params. - auth_utils.py adds user_api_key_auth_metadata to the banned request body params list so the proxy rejects any attempt to supply the field at the HTTP layer. The field is server-reserved: it is written exclusively by add_user_api_key_auth_to_request_metadata from the authenticated key's database record after the ban check runs. Issue 2 (management_helpers/utils.py — API key in OTEL span): - _emit_management_endpoint_otel_span stripped plaintext credential fields (key, token, api_key, secret, …) from the response dict before passing it to the OTEL success hook. dict(result) on a Pydantic GenerateKeyResponse includes the freshly-generated key field, which would previously be written as a span attribute to every configured OTEL collector/backend. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: shin-berri <shin-laptop@berri.ai> Co-authored-by: yuneng-jiang <yuneng@berri.ai> Co-authored-by: oss-agent-shin <ext-agent-shin@berri.ai> Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Yassin Kortam <yassin@berri.ai> Co-authored-by: Terrajlz <info@jouleselectrictech.com> Co-authored-by: Bruno Devaux <devaux.br@gmail.com> Co-authored-by: milan-berri <milan@berri.ai> Co-authored-by: Mateo Wang <277851410+mateo-berri@users.noreply.github.com> Co-authored-by: Mateo <mateo@Mateos-MacBook-Pro.local> Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> Co-authored-by: Yassin Kortam <yassinkortam@Yassins-MacBook-Pro.local> Co-authored-by: user <70670632+stuxf@users.noreply.github.com> Co-authored-by: Krrish Dholakia <krrish+github@berri.ai> Co-authored-by: ryan-crabbe-berri <ryan@berri.ai> Co-authored-by: ryan-crabbe-berri <ryan-crabbe-berri@users.noreply.github.com> Co-authored-by: Dibyo Mukherjee <dibyo@adobe.com> Co-authored-by: ishaan-berri <155045088+ishaan-berri@users.noreply.github.com> Co-authored-by: Ishaan Jaffer <ishaanjaffer0324@gmail.com> Co-authored-by: rinto <54238243+ririnto@users.noreply.github.com> Co-authored-by: Shin <shin@litellm.ai> Co-authored-by: mubashir1osmani <mubashir.osmani777@gmail.com>	2026-05-26 11:57:39 -07:00
milan-berri	7270f723de	fix(mcp): forward upstream initialize instructions on cold gateway init (#28231) ... Prefetch upstream InitializeResult.instructions before merging gateway initialize options when YAML/DB do not set instructions, so clients receive upstream server text on the first MCP initialize without list_tools. Co-authored-by: Cursor <cursoragent@cursor.com>	2026-05-22 16:00:47 -07:00
Sameer Kankute	ef36e89638	feat(mcp): Add tool call and tool list support via UI for Oauth mcps (#28454) ... * feat(mcp): cache OAuth token client-side so Tools tab loads without re-auth After a user creates an OAuth MCP server and completes the authorization flow, the resulting access token is now stored in sessionStorage keyed by server_id. The MCP Tools tab reads this cached token and includes it as an MCP auth header when listing and invoking tools, so the user never sees an empty tool list. When the session ends (tab close / new browser) an Authorize button re-triggers the flow without leaving the Tools screen. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * fix(ui/mcp): surface listMCPTools 401 errors so auth gate reappears listMCPTools previously swallowed all errors (including HTTP 401) by returning a synthetic { tools: [], error: 'network_error', ... } payload. That made the useQuery retry-on-401 guard and mcpToolsError dead code, so expired OAuth tokens never re-triggered the auth gate. - Throw an enhanced Error with .status attached on non-2xx responses (still preserves the legacy shape for true network failures so the caller can render a generic message without crashing). - Clear the cached OAuth session token when the tools query fails with 401, mirroring callMCPTool's onError handler so the Authorize button is shown again. - Surface mcpToolsError in the existing error banner. Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix(mcp-tools): stable onSuccess + reuse parsed flow state - Pass stable setOauthToken setter directly as onSuccess to avoid recreating useToolsOAuthFlow's resumeOAuthFlow on every render. - Reuse the already-parsed FLOW_STATE_KEY value (peeked) instead of re-reading and re-parsing sessionStorage in resumeOAuthFlow. Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix(ui/mcp): restore listMCPTools never-throws contract The previous fix made listMCPTools throw on HTTP errors while still returning a synthetic object on network errors. This inconsistent contract broke existing callers (MCPToolPermissions, MCPAppsPanel, MCPConnectPicker) which inspect result.error / result.message and expect the function to never throw. - Return a normalized { tools: [], error, message, status, ... } object on HTTP errors (instead of throwing) so all callers see a consistent shape and the user-visible error text from result.message is preserved. - Convert the returned error object into a thrown Error inside the one caller that needs it — the useQuery in mcp_tools.tsx — so the 401 retry/onError handlers still trigger and clear the cached OAuth token. Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix greptile * fix(mcp): align OAuth header alias lookup with dashboard sanitization Backend auth header resolution now matches x-mcp-{alias} keys produced by the dashboard sanitizer, and the Tools tab re-syncs OAuth tokens when serverId changes. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(mcp): widen auth header lookup types for list_tools Accept legacy str | dict server auth maps and annotate list_tools server_auth_header as Union[str, dict] for mypy. Co-authored-by: Cursor <cursoragent@cursor.com> * refactor(ui): extract shared buildCallbackUrl/clearStorage for MCP OAuth hooks Hoist the duplicate buildCallbackUrl and clearStorage helpers out of useToolsOAuthFlow and useUserMcpOAuthFlow into a new shared module src/hooks/mcpOAuthUtils.ts so the two hooks cannot drift if the URL construction or storage cleanup logic needs to change. Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix(ui): don't gate M2M OAuth MCP servers behind interactive authorize M2M (client_credentials) OAuth servers share auth_type="oauth2" with interactive PKCE servers, but the backend fetches their token internally and they typically lack a user authorization endpoint. Gating tool listing on them rendered an Authorize button that would fail or redirect incorrectly. Detect M2M via the presence of token_url (matching the existing heuristic in mcp_server_edit.tsx) and skip the auth gate. Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix(ui/mcp): return error shape when listMCPTools JSON parse fails Restore the never-throws contract when response.json() fails on a 2xx body so callers do not receive null and crash on result.tools. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Yassin Kortam <yassin@berri.ai>	2026-05-22 09:04:04 -07:00
Sameer Kankute	7a93cceb9f	Add error_description and hint for oauth flows (#28471) ... * Add error_description and hint for oauth flows * Fix tests * fix(mcp-oauth): improve redirect_uri errors without leaking internal config Use NoReturn on _oauth_invalid_request, structured errors for BYOK loopback validation, and refactor validate_trusted_redirect_uri to satisfy PLR0915. Keep PROXY_BASE_URL and raw proxy_base_url in server logs only, not in the HTTP 400 body returned to unauthenticated callers. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(mcp-oauth): stop leaking internal proxy origin in redirect_uri 400 body The trusted-redirect-uri rejection helper included the proxy's resolved scheme/host/port (e.g. http://litellm-internal:4000) in both the error_description and as a top-level proxy_origin field. Since the OAuth /authorize endpoint is unauthenticated, any caller could probe with a crafted redirect_uri and enumerate the internal network topology behind a reverse proxy. Keep full diagnostic detail in the server-side warning log (including the computed proxy base) but omit proxy-side values from the HTTP 400 body. Also drop the duplicated origin computation in _raise_trusted_redirect_uri_rejected now that those values are no longer needed by the response. Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix(mcp-oauth): remove dead userinfo check in redirect_uri validation The first check combined missing netloc with userinfo presence, making the second userinfo-only check unreachable. Split into two distinct checks so each error message reflects the actual failure mode. Co-authored-by: Yassin Kortam <yassin@berri.ai> --------- Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Yassin Kortam <yassin@berri.ai>	2026-05-22 09:00:39 -07:00
Sameer Kankute	718c4637a8	feat(mcp): allow native MCP OAuth support for cursor (#28327) ... * feat(mcp): allow native MCP OAuth redirect URIs (cursor://) Discoverable OAuth /authorize rejected cursor:// callbacks because validate_trusted_redirect_uri only accepted http/https. Add an allowlisted native path with a built-in Cursor default and optional MCP_TRUSTED_NATIVE_REDIRECT_URIS env for other clients. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(mcp): address Greptile native redirect URI review Lowercase paths in normalizer so env allowlist entries match case- insensitively. Tighten wildcard prefix matching to reject sibling paths (e.g. callback-2) unless the prefix ends with /. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(mcp): reject query params on native OAuth redirect URIs Greptile: normalization stripped query strings before allowlist compare, so cursor://.../callback?injected=... could pass validation. Reject any native redirect_uri with a query component (same as fragments). Co-authored-by: Cursor <cursoragent@cursor.com> * fix(model_cost_map): add mistral/ministral-8b-2512 entry Mistral rotated the 'mistral/mistral-tiny' alias to return 'ministral-8b-2512' as the response model, which is not in the cost map. This caused test_completion_mistral_api and test_completion_mistral_api_modified_input to fail in completion_cost lookup. Add the entry mirroring the existing openrouter/mistralai/ministral-8b-2512 pricing. * fix(mcp): lowercase default native redirect URIs Make _parse_trusted_native_redirect_uris apply the same lowercasing to built-in defaults as it does to env-var entries. * fix(tests): backfill local model_cost into remote-fetched map litellm.model_cost is loaded at import time from the URL pinned to main, so pricing entries that exist only in this branch (e.g. mistral/ministral-8b-2512, freshly added because Mistral now returns this id from mistral-tiny) are absent at test time and completion_cost lookups raise. Backfill the in-tree backup so cassette-driven cost calculations resolve against the entries that ship with the branch under test. Fixes the local_testing_part1 failures on test_completion_mistral_api and test_completion_mistral_api_modified_input. --------- Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> Co-authored-by: Claude <claude@anthropic.com>	2026-05-20 15:28:44 -07:00
Sameer Kankute	68efe6970c	fix(mcp): JWT on tools/list and REST tools/call server resolution (#28227) ... * fix(mcp): JWT on tools/list, REST server_id resolution, tool_server_mismatch Sign outbound MCP JWTs for list_mcp_tools and inject headers on the tools/list path. Resolve server_id on /mcp-rest/tools/call and return 403 tool_server_mismatch when the tool does not belong to the requested server. Default missing arguments to {}. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(mcp): restrict list JWTs to mcp:tools/list and default REST arguments to {} - List-only JWTs (call_type=list_mcp_tools) no longer carry the broad mcp:tools/call scope. _build_scope() now emits only mcp:tools/list when no tool name is provided, mirroring the existing least-privilege rule that tool-call JWTs omit mcp:tools/list. - REST /tools/call now defaults a missing 'arguments' field to {} so execute_mcp_tool() and downstream **arguments / .keys() calls don't receive None and crash with TypeError/AttributeError. Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix(mcp): validate tool/server in call_tool; skip JWT signer when not configured or static auth present Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix(mcp): align tests and mypy with user_api_key_auth on tools/list Update mocks for the new _get_tools_from_server parameter, mock server registry in REST access-denied test, and narrow static_headers for mypy. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(test): accept user_api_key_auth in get_tools_from_mcp_servers mock The side_effect for the all-servers case did not accept the new kwarg, so tools/list returned an empty list. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(mcp): fail fast for unknown tools when server mapping exists Server-name fallback in call_tool must not open an upstream session when the tool is absent from a populated mapping. Update the HTTP transport test to register a known tool before asserting not-found behavior. Co-authored-by: Cursor <cursoragent@cursor.com> * fix mypy * Fix mypy * fix(mcp): preserve tools/call scope on missing tool name; pass user_api_key_auth in list_tools Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix(mcp): match alias/server_name in _resolve_mcp_server_for_tool_call The registry lookup in _resolve_mcp_server_for_tool_call previously only compared candidate.name against the provided server_name, but tool name prefixes can be derived from a server's alias or server_name (see get_server_prefix). When the tool→server mapping is empty/stale (cold start, dynamic tools), the lookup would fail for alias-configured servers even though get_mcp_server_by_name (used by the REST path) matches alias, server_name, and name. Match the same priority of identifiers in both the registry pass and the unprefixed fallback so the MCP protocol call_tool path is consistent with the REST path. Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix(mcp): reuse proxy_logging DualCache in inject_mcp_jwt_headers_for_upstream Instead of allocating a fresh DualCache() on every tools/list invocation, prefer the shared proxy_logging_obj.internal_usage_cache.dual_cache when available. The cache argument is currently unused by MCPJWTSigner, but sharing the proxy's cache avoids per-call allocation overhead and matches the cache identity used elsewhere in the proxy hook plumbing — so any future per-request state stored in cache will survive across list calls. Co-authored-by: Claude <noreply@anthropic.com> * fix(mcp): return 403 ip_filtering for IP-restricted servers in tools/call name lookup Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix(test): accept user_api_key_auth kwarg in list_tools mocks The proxy-infra job was failing on four TestMCPServerManager tests because the mock_get_tools_from_server stubs did not accept the new user_api_key_auth keyword argument that list_tools now forwards to _get_tools_from_server. Add the kwarg to each stub so list_tools can call through cleanly. Co-authored-by: Claude <claude@anthropic.com> * fix(mcp): skip JWT injection when per-user mcp_auth_header is set MCPClient._get_auth_headers() applies extra_headers AFTER writing Authorization from auth_value, so an injected JWT silently overwrites the user's per-server OAuth token. Guard the JWT signer with 'not mcp_auth_header' so per-user OAuth (and any dict-form per-user auth) takes precedence, mirroring the existing static_headers guard. Adds a regression test that the signer's inject helper is not called when mcp_auth_header is supplied. * fix(mcp): skip JWT injection when extra_headers already has Authorization When a server uses per-user OAuth tokens, the resolved token is passed into _get_tools_from_server via extra_headers. The JWT injection guard only checked mcp_auth_header and the server's static headers, so the signer would silently overwrite the user's OAuth Authorization header. Add a check for an existing Authorization entry in extra_headers so caller-supplied per-user OAuth tokens take precedence over JWT signing. Co-authored-by: Yassin Kortam <yassin@berri.ai> * test(mcp): cover JWT signer + tool-call resolution branches Adds unit tests for the new MCPServerManager helpers (_resolve_mcp_server_for_tool_call, _resolve_oauth2_headers_for_tool_call) and the new MCPJWTSigner paths (_build_scope call_type branches and inject_mcp_jwt_headers_for_upstream). Brings patch coverage above the auto target without changing behavior. Co-authored-by: Claude <claude@anthropic.com> * fix(mcp): retry tool-server lookup with prefixed name in REST mismatch check When the REST /mcp-rest/tools/call path sends a raw tool name plus requested_server_id, _get_mcp_server_from_tool_name(name) can return None if the mapping only stores the prefixed form. That bypassed the tool_server_mismatch 403 guard and let the call fall through to trusting requested_server. Retry the lookup with every known prefix of the requested server so the mismatch check fires whenever the tool is actually registered. Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix(mcp): always reject unknown tools in server-name fallback Defense-in-depth: _resolve_mcp_server_for_tool_call previously skipped the unknown-tool check whenever the per-server mapping had no entries yet (cold start, OAuth2 lazy listing, or upstream listing failure), allowing arbitrary tool names to reach upstream servers. Tighten the check so the server-name fallback always rejects tool names not present in the mapping. Callers must call list_tools first (standard MCP flow) before tools/call can resolve. Removes the now-unused _mapping_has_tools_for_server helper and adds an explicit empty-mapping rejection test alongside the existing populated-mapping rejection test. Co-authored-by: Sameer Kankute <sameer@berri.ai> --------- Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Yassin Kortam <yassin@berri.ai> Co-authored-by: Claude <claude@anthropic.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Claude (greptile subagent) <claude-greptile-bot@anthropic.com>	2026-05-20 13:31:44 -07:00
Mateo Wang	1b0ae3af83	fix(mcp-oauth): PROXY_BASE_URL escape hatch + diagnostic logging for {"detail":"invalid_request"} (#28086) ... * fix(mcp-oauth): add PROXY_BASE_URL escape hatch + diagnostic logging for invalid_request Customers hitting "{"detail":"invalid_request"}" on the MCP /authorize endpoint had no way to recover when their ingress mangles X-Forwarded-* headers (the same-origin check in validate_trusted_redirect_uri compares the browser-supplied redirect_uri against get_request_base_url, which is reconstructed from those headers). Two contained changes: 1. get_request_base_url now honours PROXY_BASE_URL as the canonical public origin when set, bypassing the X-Forwarded-* trust gate entirely. Operators who know their public URL can set it once instead of debugging ingress header rewrites. 2. The rejection path in validate_trusted_redirect_uri emits a WARN log carrying the redirect_uri, computed proxy base, and the X-Forwarded-* / Host headers seen. A bare 400 was undiagnosable; this turns it into a one-line root-cause. * test(mcp-oauth): capture warnings from correct logger ("LiteLLM") Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix(mcp-oauth): reject malformed PROXY_BASE_URL with one-shot diagnostic A scheme-less PROXY_BASE_URL (e.g. "litellm.example.com" instead of "https://litellm.example.com") would sail through urlparse with empty scheme + netloc, silently breaking every same-origin compare in validate_trusted_redirect_uri and leaving the operator staring at the same opaque 400 the env var was meant to fix. Validate it once at read time: only honour values that parse as http(s) URLs with a non-empty netloc; otherwise log a one-shot WARN naming the bad value and fall through to the request-derived origin so the proxy still serves traffic. * fix(mcp/oauth): normalize PROXY_BASE_URL to strip query/fragment Match the X-Forwarded-* path's normalization so a configured PROXY_BASE_URL containing a query string or fragment does not break downstream f-string concatenation like f"{base_url}/callback". Co-authored-by: Yassin Kortam <yassin@berri.ai> * refactor(mcp-oauth): drop non-essential comments from PROXY_BASE_URL changes Strip narrative comments and verbose docstrings added in this PR; the code is intuitive enough on its own and the log messages already carry their own diagnostic context. Pre-existing comments are left untouched. --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Yassin Kortam <yassin@berri.ai>	2026-05-16 17:48:03 -07:00
Sameer Kankute	106b2f2da8	Merge pull request #27977 from BerriAI/litellm_mcp_internal_delegate_pkce ... fix(mcp): delegate PKCE bypass for internal MCP servers	2026-05-15 22:57:48 +05:30
Sameer Kankute	4e2b2d9d1f	fix(mcp): expose delegate_auth_to_upstream in MCP server list rows (#27936) ... _build_mcp_server_table omitted delegate_auth_to_upstream, so GET /v1/mcp/server always returned the default false while the registry kept the DB value. Co-authored-by: Cursor <cursoragent@cursor.com>	2026-05-15 04:32:14 -07:00
Sameer Kankute	d855e56333	chore(mcp): warn on internal + upstream PKCE delegate ... Log verbose_logger.warning when loading oauth2 interactive servers with available_on_public_internet=false and delegate_auth_to_upstream=true (config + DB). Dashboard Alert for the same combo. CLAUDE note for operators. Tests for log and M2M skip.	2026-05-15 10:05:35 +05:30
Sameer Kankute	5aabfccf57	fix(mcp): allow delegate PKCE bypass for internal MCP servers ... Remove available_on_public_internet gating from delegate-auth-to-upstream paths so oauth2 + delegate_auth_to_upstream interactive servers behave the same when marked internal. Keeps M2M exclusion. Updates tests.	2026-05-15 08:32:09 +05:30
Dennis Henry	9b6ab55c5f	fix: allow for allowlisted redirect URIs (#27761) ... * fix: allow for allowlisted redirect URIs * github comment addressing * Update litellm/proxy/_experimental/mcp_server/oauth_utils.py Co-authored-by: veria-ai[bot] <224490171+veria-ai[bot]@users.noreply.github.com> * harden oauth wildcard further * test: cover wildcard entry with dot-leading suffix rejection --------- Co-authored-by: veria-ai[bot] <224490171+veria-ai[bot]@users.noreply.github.com>	2026-05-14 11:19:30 -07:00
Sameer Kankute	18f77ff7bc	feat(mcp): add delegate_auth_to_upstream flag for PKCE passthrough (#27834) ... * feat(mcp): add delegate_auth_to_upstream flag for PKCE passthrough Adds an opt-in per-server flag that lets clients (e.g. VS Code) complete PKCE directly with an upstream OAuth2 MCP server, instead of LiteLLM double-gating with its own API-key/SSO check. Only honored when auth_type=oauth2 and the operator explicitly sets the flag; mixed-target or non-oauth2 requests fail closed. - Adds the field to Pydantic models, Prisma schema, and a migration - New MCPRequestHandler._target_servers_delegate_auth_to_upstream gate that runs only when no x-litellm-api-key is present, so authenticated users still get user_id resolution + stored-credential lookup - Anonymous callers now see delegate servers in get_allowed_mcp_servers (scoped to delegate servers only; the upstream still enforces auth) - mcp_management_endpoints: allow anonymous /authorize and /token for delegate servers so VS Code can complete PKCE without a LiteLLM session - UI toggle (shown only for oauth2) + payload/view wiring - Tests covering: oauth2 on/off, non-oauth2 with flag, mixed targets, no resolvable target, explicit key precedence, and 401 emission Co-authored-by: Cursor <cursoragent@cursor.com> * Enforce oauth2 for delegated MCP auth bypass Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix(mcp): close secondary Authorization bypass for delegate servers The delegate-auth bypass gated only on the primary `x-litellm-api-key` header, so a LiteLLM key sent via `Authorization: Bearer sk-...` (the secondary header) was silently dropped — skipping spend tracking and rate limiting. Gate on the resolved litellm_api_key (which considers both headers) so the bypass fires only when neither is present. Also update the existing "Authorization header present" test to reflect that an upstream OAuth token now flows through the existing oauth2 fallback (LiteLLM auth attempt → fail → anonymous), not via the delegate branch. Co-authored-by: Cursor <cursoragent@cursor.com> * Avoid duplicate MCP OAuth credential lookup Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix(mcp): block delegate bypass for M2M and internal-only servers Two security issues flagged in code review: 1. High – client_credentials (M2M) servers must not be delegatable: LiteLLM auto-fetches the upstream token using stored credentials, so allowing anonymous bypass would let any external caller invoke tools authenticated as LiteLLM's service account. Fix: check `server.has_client_credentials` in `_target_servers_delegate_auth_to_upstream`, the anonymous allow-list in `get_allowed_mcp_servers`, and `_mcp_oauth_user_api_key_auth`. 2. Medium – internal-only servers exposed to public internet: The anonymous delegate allow-list was not filtering by `available_on_public_internet`, so external callers with an upstream OAuth token could invoke tools on servers marked internal-only. Fix: add `available_on_public_internet` guard to the anonymous delegate server list in `get_allowed_mcp_servers`. Tests added for both cases. Co-authored-by: Cursor <cursoragent@cursor.com> * Require public MCP delegate auth servers Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix(mcp): align delegate auth path parsing with downstream routing `_extract_target_server_names_from_path` used a naive segments-based split while `server.py::_get_mcp_servers_in_path` uses a regex that allows server names with one embedded slash and comma-separated lists. With the old parser, a request to `/mcp/<delegated>/<garbage>` was parsed as targeting `<delegated>` by the auth gate (bypassing LiteLLM auth) while the routing layer parsed it as `<delegated>/<garbage>` — when that name did not resolve, the request fell back to the anonymous allow-list, which can include `allow_all_keys` servers that normally require a LiteLLM key. Replace the parser with the same regex logic as `_get_mcp_servers_in_path` so auth gating sees the exact target name(s) downstream routing sees. Add regression tests covering parser parity and the specific extra-path-segment bypass attempt. https://claude.ai/code/session_01SjyPmwfmrq8fveFgw9iHW9 * fix(mcp): close header/path TOCTOU in MCP delegate auth gate `_target_servers_delegate_auth_to_upstream` and `_target_servers_use_oauth2` trusted the `x-mcp-servers` header when present, but `server.py::extract_mcp_auth_context` overrides that header with the path-derived list for `/mcp/...` routes. An attacker could set `x-mcp-servers: <delegated>` while pointing the URL path at a non-delegate server, flipping the auth gate without changing the target downstream routing actually uses. Extract a shared `_resolve_target_server_names` helper that mirrors the downstream override (path-derived names for `/mcp/...` routes, header value otherwise). Add regression tests covering the TOCTOU attempt and the helper's path-vs-header precedence. https://claude.ai/code/session_01SjyPmwfmrq8fveFgw9iHW9 * Fix delegated MCP OAuth test mock Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix(mcp): drop unreachable /{server}/mcp branch in auth path parser `_extract_target_server_names_from_path` also matched the ``/{server_name}/mcp`` form, but the downstream parser ``_get_mcp_servers_in_path`` only handles ``/mcp/...`` — and ``dynamic_mcp_route`` in ``proxy_server`` rewrites ``/{name}/mcp`` to ``/mcp/{name}`` on the scope before the MCP handler runs. Parsing the un-rewritten form on the auth side was therefore unreachable in production, and contradicted the docstring's claim of mirroring the downstream parser — exactly the kind of mismatch that risks a future header/path TOCTOU if any new entry point skips the rewrite. Drop the branch; the canonical ``/mcp/...`` path matches both parsers. Update the regression test to assert the new behavior. https://claude.ai/code/session_01SjyPmwfmrq8fveFgw9iHW9 * Fix MCP path auth target resolution Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix(mcp): require auth for refresh_token grants on delegate-auth servers `_mcp_oauth_user_api_key_auth` gates the unauthenticated PKCE flow for ``delegate_auth_to_upstream`` servers, but the bypass applied to BOTH ``/authorize`` and ``/token`` regardless of grant type. ``mcp_token`` accepts ``grant_type=refresh_token`` as well as ``authorization_code``, and ``exchange_token_with_server`` attaches the server's stored ``client_secret`` to whatever is forwarded upstream. An unauthenticated caller holding a refresh token issued to that OAuth client could mint fresh upstream access tokens through LiteLLM. Limit the anonymous bypass on ``/token`` to ``grant_type=authorization_code`` (the only grant PKCE actually protects via ``code_verifier``); fall through to normal LiteLLM auth for ``refresh_token`` and any other grant. ``/authorize`` continues to allow anonymous PKCE redirects. https://claude.ai/code/session_01SjyPmwfmrq8fveFgw9iHW9 * fix(ui): clear delegate_auth_to_upstream when switching off oauth2 The ``delegate_auth_to_upstream`` form field is rendered inside an ``isOAuth2 && (...)`` conditional, so the Form.Item unmounts when the user changes ``auth_type`` away from ``oauth2``. The follow-up ``form.setFieldValue("delegate_auth_to_upstream", false)`` runs after the field has already deregistered, so ``onFinish`` receives ``undefined`` and the fallback ``?? mcpServer.delegate_auth_to_upstream`` preserved the old ``true``. The flag then persisted in the database for a non-oauth2 server and silently re-activated if ``auth_type`` was later switched back to ``oauth2``. In the edit payload, force the flag to ``false`` whenever ``auth_type !== oauth2``; only trust the form value (and the existing DB fallback) when the server is actually oauth2. Backend defense-in-depth already ignores the flag for non-oauth2 servers, but the DB state should stay clean too. https://claude.ai/code/session_01SjyPmwfmrq8fveFgw9iHW9 * Fix MCP delegate auth reset on edit Co-authored-by: Yassin Kortam <yassin@berri.ai> --------- Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Yassin Kortam <yassin@berri.ai> Co-authored-by: Claude <claude@anthropic.com>	2026-05-13 12:06:13 -07:00
Sameer Kankute	466f06df6d	fix(mcp): surface upstream 401 for token-forwarding MCP servers (#27847) ... * fix(mcp): surface upstream 401 for token-forwarding MCP servers For MCP servers configured with extra_headers: [Authorization], the gateway forwards the client token directly to the upstream. When that token is rejected (expired or invalid) the upstream returns 401, but the MCP SDK starts the SSE stream with 200 OK before calling handlers, so the 401 can't be returned mid-stream. Fix: add a pre-flight httpx probe in handle_streamable_http_mcp — before the SDK opens the session — so the gateway can still return HTTP 401 with WWW-Authenticate: Bearer authorization_uri=<gateway-discovery-url> when the upstream rejects the token. The probe fails-open (returns 200) on network errors so a transient hiccup does not block valid requests. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(mcp): parallelize pre-flight auth probes and use HEAD to avoid side effects - Extract forwarded_auth outside the pass-through server loop (was called N times for the same scope value) - Gather all upstream auth probes concurrently with asyncio.gather instead of sequentially; eliminates N×5 s worst-case latency - Switch probe from POST+initialize JSON-RPC body to HEAD request; HEAD carries the Authorization header so the upstream rejects invalid tokens with 401 but never allocates a session or writes an audit entry Co-authored-by: Cursor <cursoragent@cursor.com> * fix(mcp): use get_async_httpx_client in _probe_upstream_auth Replaces bare httpx.AsyncClient with the project-standard get_async_httpx_client(httpxSpecialProvider.MCP) to satisfy the ensure_async_clients_test code coverage check and avoid the +500 ms per-request overhead of creating a new client on every probe call. Co-authored-by: Cursor <cursoragent@cursor.com> * refactor(mcp): extract pre-flight probe into _check_passthrough_upstream_auth Moves the parallel upstream auth probe logic out of handle_streamable_http_mcp into a dedicated helper to satisfy Ruff PLR0915 (Too many statements > 50). Co-authored-by: Cursor <cursoragent@cursor.com> * fix(mcp): gate pre-flight probes on authorized server set to prevent bypass _check_passthrough_upstream_auth was resolving user-supplied server names directly before authorization ran, letting any permitted LiteLLM key trigger an upstream HEAD probe to a server it was not allowed to use. Changes: - Call _get_allowed_mcp_servers inside the helper so only servers the caller's key is authorized for are probed. - Move the call site to after toolset scoping so the auth context is fully resolved before the probe list is built. - Thread user_api_key_auth into the helper signature (replaces the raw mcp_servers name list). Co-authored-by: Cursor <cursoragent@cursor.com> * Add async HTTP HEAD support Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix(mcp): use Scope type annotation in _get_forwarded_auth_from_scope Co-authored-by: Cursor <cursoragent@cursor.com> * Fix MCP upstream auth probe method Co-authored-by: Yassin Kortam <yassin@berri.ai> * Remove unused AsyncHTTPHandler head method Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix(mcp): exclude has_client_credentials servers from pre-flight auth probe _prepare_mcp_server_headers skips caller Authorization when the server uses OAuth client-credentials (M2M), but the pre-flight probe was still selecting those servers and forwarding the caller's raw token in the HEAD request. Exclude servers with has_client_credentials from the probe list to match the actual downstream header-preparation logic. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(mcp): propagate upstream 403 as 403, not 401 with WWW-Authenticate Per RFC 9110, 401 means "go get new credentials." Mapping an upstream 403 to a gateway 401 causes OAuth clients to restart the authorization flow, obtain a fresh token with identical scopes, hit 403 again, and loop indefinitely. 401 from upstream → gateway 401 + WWW-Authenticate (re-authorize) 403 from upstream → gateway 403 (no WWW-Authenticate hint) Co-authored-by: Cursor <cursoragent@cursor.com> * fix(mcp): skip auth probe when Authorization may be the LiteLLM proxy key The pre-flight upstream probe must not forward the caller's Authorization header when it could itself be the LiteLLM proxy API key. Restrict the probe to requests that supply x-litellm-api-key explicitly — only then is the Authorization header unambiguously the upstream OAuth token the caller wants forwarded. * Fix MCP ASGI HTTPException propagation Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix(mcp): use public AsyncHTTPHandler.post() in auth probe Use AsyncHTTPHandler.post() and catch httpx.HTTPStatusError explicitly so the 401/403 we want to surface is not silently swallowed by the broad fail-open except Exception block. Avoids reaching into the handler's private client attribute, which would silently regress to fail-open if AsyncHTTPHandler is ever refactored. * Fix MCP auth probe tests Co-authored-by: Yassin Kortam <yassin@berri.ai> * test(mcp): add coverage for httpx.HTTPStatusError path in auth probe AsyncHTTPHandler.post() calls raise_for_status() internally, so a real upstream 401/403 lands as httpx.HTTPStatusError. Add a test that exercises that specific exception path so a regression that swallows the error in the broad fail-open except Exception would be caught. --------- Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Yassin Kortam <yassin@berri.ai> Co-authored-by: claude-bot <claude-bot@anthropic.com>	2026-05-13 12:03:36 -07:00
Sameer Kankute	aa9e7b9808	feat: litellm shin agent oss staging 05 10 2026 (#27631) ... * fix: invalidate cached tag object on tag budget reset (#27481) (#27572) Squash-merged by litellm-agent from oss-agent-shin's PR. * chore(mcp): tighten stdio server registration paths (#27570) Squash-merged by litellm-agent from stuxf's PR. * fix(proxy): clear MCP OpenAPI mappings on server eviction; widen budget cache invalidation Evict OpenAPI tools from global_mcp_tool_registry and strip tool_name_to_mcp_server_name_mapping entries when a server leaves the runtime registry (remove_server and approval-status eviction). Invalidate user_api_key_cache for keys, orgs, and team members on budget-tier spend resets alongside tags. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(mcp): align update_server eviction with remove_server name fallback Document budget-reset test assertion flip (cross-pod cache staleness). Greptile: eviction now pops by server_id then server_name like remove_server; test docstring explains assert_not_awaited -> assert_any_await change. Co-authored-by: Cursor <cursoragent@cursor.com> * Fix org budget cache invalidation --------- Co-authored-by: oss-agent-shin <ext-agent-shin@berri.ai> Co-authored-by: stuxf <70670632+stuxf@users.noreply.github.com> Co-authored-by: Cursor <cursoragent@cursor.com>	2026-05-11 20:31:43 -07:00
Sameer Kankute	9ed99037d2	Merge pull request #27422 from BerriAI/shin_agent_oss_staging_05_07_2026 ... [litellm-agent] Staging → litellm_internal_staging (5/7/2026)	2026-05-11 11:58:05 +05:30
milan-berri	9380940ced	fix(mcp): forward extra_headers for OpenAPI MCP tools (#27383) ... * fix(mcp): forward extra_headers for OpenAPI MCP tools OpenAPI-generated tools only applied static closure headers and BYOK Authorization via ContextVar. Copy MCPServer.extra_headers from the incoming MCP request into _request_extra_headers (set in server.py before local tool dispatch), merge in openapi_to_mcp_generator via a small helper. OAuth2 M2M: do not forward caller Authorization from raw_headers (same rule as _prepare_mcp_server_headers for managed MCP). Adds TestRequestExtraHeaders and clarifies mcp_server_manager registration comment. Fixes #26794 Co-authored-by: Cursor <cursoragent@cursor.com> * refactor(mcp): access has_client_credentials on MCPServer directly Greptile: getattr default was redundant; property exists on MCPServer and mcp_server is non-None inside the extra_headers forwarding block. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>	2026-05-09 15:10:54 -04:00
ishaan-berri	e4c14862fc	feat(mcp): add OBO MCP Auth (#27421) ... * feat(mcp): add oauth2 token exchange auth Co-authored-by: ishaan-berri <ishaan-berri@users.noreply.github.com> * fix(mcp): cache token exchange fallback Co-authored-by: ishaan-berri <ishaan-berri@users.noreply.github.com> --------- Co-authored-by: oss-agent-shin <279349115+oss-agent-shin@users.noreply.github.com> Co-authored-by: ishaan-berri <ishaan-berri@users.noreply.github.com>	2026-05-07 15:35:21 -07:00
oss-pr-review-agent-shin[bot]	158b0c28c0	[litellm-agent] Staging → litellm_internal_staging (5/7/2026) (#27375) ... Squash-merged by litellm-agent from oss-pr-review-agent-shin[bot]'s PR.	2026-05-07 21:29:47 +00:00
ishaan-berri	bd1a05aed9	Fix MCP DB reload partial failures (#27314) ... * Fix MCP database reload partial failures Co-authored-by: ishaan-berri <ishaan-berri@users.noreply.github.com> * Avoid staged MCP registry exposure Co-authored-by: ishaan-berri <ishaan-berri@users.noreply.github.com> --------- Co-authored-by: oss-agent-shin <279349115+oss-agent-shin@users.noreply.github.com> Co-authored-by: ishaan-berri <ishaan-berri@users.noreply.github.com>	2026-05-06 15:18:18 -07:00
Krrish Dholakia	454ce5073f	fix(anthropic, mcp): sanitize tool names to match Anthropic's [a-zA-Z0-9_-]{1,128} pattern (#26788) ... * fix(anthropic, mcp): sanitize tool names to match Anthropic's `^[a-zA-Z0-9_-]{1,128}$` Tool names with characters like `/` or `.` (commonly produced by the OpenAPI -> MCP generator from `operationId`s such as `actions/download-job-logs-for-workflow-run`) caused Anthropic to reject requests with `tools.N.custom.name: String should match pattern '^[a-zA-Z0-9_-]{1,128}$'`. Two layers of fix: 1. Anthropic transformation: build a per-request forward map (original -> sanitized, disambiguated by suffix on collisions) and a reverse map (only for names actually rewritten). Forward map is applied to tool defs, `tool_choice`, and historical assistant tool_calls in messages. Reverse map is threaded through both the non-streaming and streaming response paths so callers continue to see their original tool names in `tool_use` blocks. 2. OpenAPI -> MCP generator: sanitize `operationId` (and the method+path fallback) at registration time so generated MCP tools are valid for any strict-name provider, not just Anthropic. The dashboard preview endpoint applies the same sanitization for parity. Includes unit tests covering: collision disambiguation between `foo_bar` and `foo/bar` in the same request, reverse-map only firing for actually-rewritten names, message rewrite for historical tool_calls, streaming chunk_parser reverse-mapping, and sanitization of OpenAPI operationIds plus the preview endpoint output. Made-with: Cursor * fix(anthropic): build tool-name maps in transform_request, not optional_params The previous patch stashed the per-request forward and reverse tool-name maps under ``optional_params["_anthropic_tool_name_forward_map"]`` and ``optional_params["_anthropic_tool_name_map"]``. ``optional_params`` is the dict that becomes the JSON body via ``data = {**optional_params}``, so those internal keys leaked over the wire and Anthropic 400'd with: _anthropic_tool_name_forward_map: Extra inputs are not permitted Worse, this meant *every* request whose tool list contained any name with an invalid character (the exact case the patch was meant to fix) regressed into a confusing meta-error pointing at LiteLLM's internal map instead of the offending tool. Fix: move all tool-name sanitization into ``transform_request``, which is the single chokepoint already shared by ``AnthropicConfig``, ``AmazonAnthropicConfig`` (Bedrock invoke), ``VertexAIAnthropicConfig``, and ``AzureAnthropicConfig`` (all call ``super().transform_request`` / ``AnthropicConfig.transform_request(self, ...)``). New static helper ``_sanitize_tool_names_in_request`` walks the already-Anthropic-shaped ``optional_params["tools"]`` (only ``type=="custom"`` entries -- hosted tool names are reserved by Anthropic and must not be touched), builds the per-request forward/reverse maps, and applies the forward map in place to ``tools[*].name`` and ``tool_choice.name``. The reverse map is stashed exclusively on ``litellm_params`` (which is never serialized to a provider) under ``_anthropic_tool_name_map`` for the response paths to consume. Side effect of this restructure: ``map_openai_params`` is now a pure OpenAI->Anthropic param translator with no side-channel state, which matches its contract everywhere else in the codebase. Tests: replaced the now-incorrect "stashes maps in optional_params" tests with regressions that assert no underscore-prefixed keys appear in either ``optional_params`` after ``map_openai_params`` or in the final ``transform_request`` body. Added end-to-end coverage for: sanitization in ``transform_request``, ``tool_choice`` rewriting, historical ``tool_calls`` rewriting in messages, and hosted-tool passthrough. Made-with: Cursor * fix(anthropic): always sanitize empty text content blocks Anthropic 400s on `{"role": "user", "content": ""}` with: "messages: text content blocks must be non-empty" LiteLLM already had `_sanitize_empty_text_content` to rewrite empty text to a placeholder, but it was gated behind `litellm.modify_params=True`. With that flag off (default), empty content from upstream agent frameworks (e.g. pydantic-ai) flowed straight through and tripped the Anthropic validator. Fix: - Always run `_sanitize_empty_text_content` at the top of `anthropic_messages_pt`, independent of `modify_params`. There is no way to "pass through" an empty text block, so this is non-optional. The richer tool-call sanitizations (Cases A/B/D, which actually mutate conversation structure) remain gated on `modify_params`. - Extend `_sanitize_empty_text_content` to also handle list-of-blocks content (`[{"type": "text", "text": ""}]`), not just string content. Adds 3 regression tests covering string content, list-of-blocks content, and the no-op case (non-empty messages with modify_params off). Made-with: Cursor * fix(anthropic): drop dead tool-name forward-map params, fix mypy + caller-mutation - remove unused `name_forward_map` param from `_map_tool_choice`, `_map_tool_helper`, `_map_tools` and the `_apply_anthropic_tool_name_forward` helper. Production sanitization runs in `_sanitize_tool_names_in_request` at `transform_request`; these params were never threaded through. - handler.py: use `ANTHROPIC_TOOL_NAME_REVERSE_MAP_KEY` constant instead of the hardcoded `"_anthropic_tool_name_map"` string. - fix mypy `"object" has no attribute "__iter__"` in `_rewrite_tool_names_in_messages` by guarding `tool_calls` with `isinstance(..., list)`. - `_sanitize_tool_names_in_request`: build a new tools list with copy-on- change entries (and copy `tool_choice` on rewrite) so a caller reusing the same tool list/dicts across requests doesn't see its inputs permanently rewritten. - doc-comment `_build_request_tool_name_maps` clarifying it operates on OpenAI-format tools (vs `_sanitize_tool_names_in_request` which runs on Anthropic-format tools post-`_map_tools`). - tests: drop 3 tests pinning the now-removed param paths; add coverage for tool_calls + None function_call rewrite and caller-dict immutability. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(mcp): inherit stored credentials in test/tools/list for edit flow When editing an existing MCP server, the Tool Configuration preview calls POST /mcp-rest/test/tools/list with server_id but no credentials (management API redacts them). The endpoint now calls _inherit_credentials_from_existing_server() so stored bearer tokens and OAuth2 M2M credentials are loaded from global_mcp_server_manager automatically — tools load without re-entering credentials. New servers (no server_id) and requests with explicit credentials are unaffected (function is a no-op in both cases). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(mcp): show all tools in edit panel, not just allowed tools Edit flow was passing externalTools (from GET /tools/list, filtered by allowed_tools) to MCPToolConfiguration, disabling the internal hook. Remove the external props so the internal hook fires via POST /test/tools/list, which returns all tools unfiltered. Combined with the credential inheritance fix, tools load automatically without re-entering credentials and all tools are visible for re-configuration. existingAllowedTools still pre-checks previously allowed tools. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Fix order-dependent collision in _build_anthropic_tool_name_maps Use a two-pass approach: first pre-register all already-valid tool names in the 'used' set, then sanitize/disambiguate names that need rewriting. This ensures valid names always have priority regardless of input order, preventing duplicate tool names on the wire when e.g. 'foo/bar' appears before 'foo_bar' in the tool list. Add regression test for the reversed ordering case. * Fix OpenAPI tool name collision: disambiguate sanitized names with numeric suffixes sanitize_openapi_tool_name replaces all invalid chars with '_', but when two operationIds differ only by sanitized characters (e.g. 'foo/list' and 'foo.list' both become 'foo_list'), the second registration silently overwrites the first in the tool registry. Add collision disambiguation in register_tools_from_openapi that appends _2, _3, ... suffixes when a sanitized name is already taken, mirroring the existing logic in _build_anthropic_tool_name_maps. * Fix preview endpoint missing collision disambiguation for tool names Add used_names tracking and _2/_3 suffix disambiguation to _preview_openapi_tools, matching the logic in register_tools_from_openapi. Without this, two operationIds that sanitize to the same string (e.g. 'foo/list' and 'foo.list' both becoming 'foo_list') would show duplicate names in the preview while registration would disambiguate them. * Align preview HTTP method order with register_tools_from_openapi The preview endpoint and register_tools_from_openapi both use order-dependent collision disambiguation (_2, _3 suffixes). When the iteration order differs, two operations on the same path with sanitized names that collide get different suffixes in preview vs registration, so the dashboard shows names that don't match what actually got registered. Also adds a regression test that fails on the swapped order. Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> * Skip duplicate originals in _build_anthropic_tool_name_maps If the same invalid tool name appeared twice in original_names (e.g. ['foo/bar', 'foo/bar']), the second occurrence overwrote the forward map entry with a freshly-suffixed name (foo_bar_2), leaving foo_bar orphaned in 'used' with no reverse mapping. _sanitize_tool_names_in_request then rewrote both tool entries to foo_bar_2, and Anthropic 400'd on duplicate tool names. Skip the rewrite if forward already has the original mapped. Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>	2026-05-06 00:00:36 +00:00
Sameer Kankute	f576eb3228	Merge pull request #26960 from BerriAI/litellm_org_mcp_permissions ... feat(mcp): enforce org-level MCP server and toolset permissions	2026-05-02 11:25:55 +05:30
yuneng-jiang	c154b0df24	Merge pull request #27016 from stuxf/fix/mcp-openapi-tool-auth-bypass ... fix(mcp): run pre_call_tool_check on OpenAPI/local-registry path (VERIA-7)	2026-05-01 17:38:40 -07:00
user	8ee599aa7d	fix(mcp): use canonical proxy_logging_obj, deny when MCP server is unresolvable ... Greptile flagged two follow-ups on the OpenAPI/local-registry pre-call check: 1. **P1 runtime crash via None proxy_logging_obj.** `kwargs.get("proxy_logging_obj")` is `None` on the MCP entry path, and `pre_call_tool_check` calls `proxy_logging_obj._create_mcp_request_object_from_kwargs` unconditionally after the security checks, which would have crashed every legitimate call with `AttributeError`. Source the logging object from `litellm.proxy.proxy_server` the same way `_handle_managed_mcp_tool` already does. 2. **P2 authorization-bypass window when mcp_server is None.** Previously the new check was guarded by `if mcp_server is not None`, so any local tool whose registry entry had no resolvable server (a startup-race window before `_initialize_tool_name_to_mcp_server_name_mapping` completes, or an orphaned registry entry) ran without the security check. Tools registered via openapi_to_mcp_generator are always tied to a server, so a missing one is a configuration/timing fault — fail the call with 503 instead of dispatching unguarded. Tests: existing two pass with an added assertion that `proxy_logging_obj` is non-None at the call site, plus a new test that covers the 503 deny branch when the tool→server mapping is missing. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-01 22:28:46 +00:00
yuneng-jiang	8363fe00e1	Merge pull request #26841 from stuxf/fix/mcp-xff-trust-gate ... chore(mcp): require trusted-proxy gate before honouring X-Forwarded-* on OAuth discovery	2026-05-01 15:08:40 -07:00
user	5daf0168a8	fix(mcp): run pre_call_tool_check on OpenAPI/local-registry path (VERIA-7) ... `execute_mcp_tool` dispatches in two ways: managed MCP servers go through `_handle_managed_mcp_tool`, which calls `MCPServerManager.pre_call_tool_check` to enforce allowed/banned tool lists, key/team `object_permission` tool grants, and parameter validation. OpenAPI-backed tools, however, were resolved via `global_mcp_tool_registry` and dispatched directly to `_handle_local_mcp_tool` — entirely skipping `pre_call_tool_check`. A caller could invoke any registered OpenAPI tool regardless of their key/team permissions, including administrative or destructive operations on the upstream API. Run `pre_call_tool_check` before the local-registry dispatch whenever the resolved server is set (the same condition used to surface server context to the managed path). Honor any guardrail-modified arguments the hook returns. Errors raised by the hook propagate up before `_handle_local_mcp_tool` runs. Tests cover both directions: the pre-call hook fires when the local tool resolves alongside a server, and a hook-raised HTTPException prevents the local handler from being invoked. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-01 22:02:47 +00:00
Sameer Kankute	8300657af9	fix(mcp): preserve oauth2 m2m auth for tools routes (#26871) ... * Fix tool/list M2M creds issue * Fix tool call creds issue * Fix greptile review * Fix lint * Fix lint * Fix lint * Fix lint	2026-05-01 10:26:10 -07:00
Cursor Agent	fca21a979c	Fix org MCP permission ceiling escalation	2026-05-01 14:54:29 +00:00
Sameer Kankute	b540a71e47	feat(mcp): enforce org-level MCP server and toolset permissions ... Apply organization object_permission as a ceiling on allowed MCP servers and tool permissions, consistent with vector store org checks. Includes unit tests for org ceiling, intersection, and tool filtering. Made-with: Cursor	2026-05-01 10:10:38 +05:30
yuneng-jiang	256e05e474	Merge pull request #26849 from stuxf/fix/mcp-oauth-discovery-ssrf ... chore(mcp): SSRF guard on OAuth metadata discovery follow-up fetches	2026-04-30 13:44:16 -07:00
yuneng-jiang	174c770b07	Merge pull request #26836 from stuxf/fix/byok-credential-encryption ... chore(mcp): encrypt user-scoped MCP credentials at rest	2026-04-30 13:42:57 -07:00
user	51a3e90451	fix(mcp): reuse safe URL fetch for OAuth discovery	2026-04-30 12:42:52 -07:00
yuneng-jiang	3c060364fb	Merge pull request #26840 from stuxf/codex/mcp-oauth-root-visibility ... chore(mcp): tighten OAuth root endpoint resolution	2026-04-30 11:59:03 -07:00
user	1bb04cdc72	fix(mcp): close redirect bypass + empty-getaddrinfo gap on SSRF guard ... Three follow-ups to the OAuth-discovery SSRF guard: 1. Greptile P1 (redirect bypass): the validated origin could return a 3xx whose ``Location`` points at an internal address, and httpx would follow without re-checking the new target. Pass ``follow_redirects=False`` to both gated httpx GETs. Spec-compliant OAuth/OIDC metadata endpoints serve the JSON directly, so this doesn't affect legitimate providers. 2. Greptile P2 (empty getaddrinfo): POSIX doesn't strictly forbid an empty success-list from ``getaddrinfo``. Add an explicit ``if not infos: return False`` so the guard fails closed instead of falling through to ``return True``. 3. Mypy: ``info[4][0]`` is typed ``str | int``; narrow at the boundary with an ``isinstance`` check (fail-closed if non-str). Adds two regression tests verifying ``follow_redirects=False`` is passed at both gated fetch sites, and one verifying the empty-list case rejects the URL.	2026-04-30 02:59:50 +00:00
user	df62dd8768	chore(mcp): SSRF guard on OAuth metadata discovery follow-up fetches ... The OAuth discovery code in mcp_server_manager followed two attacker-influenceable URLs without validation: the ``resource_metadata`` URL parsed out of a ``WWW-Authenticate`` challenge, and the ``authorization_servers[0]`` field of the PRM JSON returned by the resource server. A malicious MCP server could point those at a cloud-instance-metadata service, an internal admin panel, or a loopback debug endpoint and the proxy would issue a blind GET on its behalf. Add ``_is_safe_metadata_url(url, server_url)`` and gate both follow- up fetch sites on it. A URL is allowed when: - it shares scheme + host + port with ``server_url`` (well-known endpoints constructed from the admin's URL, and PRM published at the resource server itself per RFC 9728 §3.3), or - it resolves to publicly-routable IPs only (covers federated authorization servers — Azure Entra, Google, Okta, GitHub — hosted cross-origin from the resource server). URLs that resolve to private / loopback / link-local / cloud-metadata addresses, or that don't resolve at all, are rejected. ``http`` and ``https`` are the only schemes accepted. The IP block list is provided by the existing ``_is_blocked_ip`` helper from ``litellm_core_utils.url_utils`` so the policy stays consistent with the rest of the proxy. The guard does not protect against active DNS rebinding between this resolution and the subsequent httpx GET — the same-authority pin remains the primary mitigation; the IP check is defence in depth. The surface only triggers on config load / add-server, not per request, so the synchronous ``getaddrinfo`` is acceptable. Threads ``server_url`` through ``_fetch_oauth_metadata_from_resource``, ``_fetch_authorization_server_metadata``, and ``_fetch_single_authorization_server_metadata``. Existing tests for those helpers updated for the new signature; new ``TestOAuthDiscoverySSRFGuard`` covers same-authority allow, private-IP rejection across IPv4 and IPv6, multi-A-record dual- stack rejection, unresolvable hosts, non-http schemes, and end-to-end "no network call when guard denies".	2026-04-30 02:43:30 +00:00
yuneng-jiang	2e561bd04e	Merge pull request #26463 from stuxf/fix/mcp-routing-auth ... fix(mcp): tighten public-route detection and OAuth2 fallback gating	2026-04-29 19:31:00 -07:00
user	e0b32eb1cf	fix(mcp): re-encrypt user credentials during master-key rotation ... Greptile P1: this PR encrypts LiteLLM_MCPUserCredentials rows under the salt key, but the /key/regenerate rotation endpoint had no corresponding step for that table. Rotating the master key would leave every BYOK and OAuth2 user credential permanently unreadable. Adds rotate_mcp_user_credentials_master_key, mirroring the existing rotate_mcp_server_credentials_master_key pattern: read each row with the current key (via _decode_user_credential, which also handles unmigrated legacy plaintext rows), re-encrypt under the new master key, write back. One bad row is logged and skipped instead of aborting the whole rotation. Wired into key_management_endpoints.py as step 4b, alongside the existing server-credentials rotation, with the same try/except shape so a transient DB error on this table doesn't kill the whole regenerate-key flow. Tests cover: round-trip through rotation under a new key, automatic re-encryption of legacy plaintext rows (rotation also acts as a migration trigger), and a corrupt row not aborting the rotation.	2026-04-30 01:58:26 +00:00
user	c3fcafb1bf	fix(mcp): warn once when use_x_forwarded_for is on but no trusted ranges ... Greptile P1: deployments that today have ``use_x_forwarded_for: true`` but never configured ``mcp_trusted_proxy_ranges`` would silently see their MCP OAuth discovery URLs revert to the proxy's literal bind address after this change, with no log line explaining why. Emit a one-shot WARNING the first time the gate denies for that specific reason, telling the operator exactly which setting to add. The warning is module-scoped (not per-request) so the proxy log stays quiet after the first hit.	2026-04-30 01:46:42 +00:00
user	8af0544ef0	chore(mcp): tighten OAuth root endpoint resolution	2026-04-29 18:36:17 -07:00
user	463781d8cb	chore(mcp): gate X-Forwarded-* trust on get_request_base_url ... get_request_base_url unconditionally honoured X-Forwarded-Proto / Host / Port to build OAuth issuer / redirect_uri / authorization_endpoint values for the MCP discovery endpoints. In a deployment where the proxy is reachable from a caller that can send those headers (direct internet exposure, or a reverse proxy that does not strip them), an attacker could poison the OAuth metadata and steer MCP clients at an attacker-controlled host. Apply the same trusted-proxy gate the codebase already uses for get_mcp_client_ip: only honour the headers when use_x_forwarded_for is enabled in proxy settings AND the direct connection IP falls inside mcp_trusted_proxy_ranges. When that's not configured, fall back to the request's literal base_url, so an untrusted caller cannot poison the discovery metadata. The existing X-Forwarded-* parsing test cases now opt into a trust_xff fixture (the parsing logic itself is unchanged). Adds a matrix for the new gate covering: XFF disabled, XFF enabled with no ranges, caller outside ranges, caller inside ranges, and the loopback-dev-deployment case.	2026-04-30 01:30:41 +00:00
user	c76c300392	fix(mcp): address Greptile P2s on credential encoding helpers ... Three minor fixes from Greptile review: 1. _decode_user_credential now also catches TypeError so a null credential_b64 value returns None instead of propagating, matching the documented "returns None when neither path yields a valid string" contract. 2. The OAuth2 BYOK guard error no longer claims the existing row is a BYOK credential — after a salt-key rotation, an OAuth2 row can fail to decrypt and reach the same guard. Reword to "could not be verified as an OAuth2 token", which is accurate for both cases. 3. Drop the no-op sys.path.insert in the new test file (other tests in the directory don't need it; pytest picks up the package via the installed editable wheel). Adds a regression test for the None-input case.	2026-04-30 00:44:37 +00:00
user	f3000bda36	chore(mcp): encrypt user-scoped credentials at rest ... LiteLLM_MCPUserCredentials.credential_b64 stored both BYOK API keys and OAuth2 access tokens as plain urlsafe-base64 of the raw value. Any DB read could recover the upstream-provider key. Run all writes through encrypt_value_helper (nacl SecretBox, the same helper used for the server-level credentials column) and read back via a small dual-path helper that tries decryption first, then falls back to plain base64 so existing rows keep working until they get rewritten. Folds the three near-identical "decode -> json.loads -> check type == oauth2" sites into _decode_oauth_payload, which simplifies the BYOK guard inside store_user_oauth_credential.	2026-04-30 00:27:15 +00:00