Commit graph

588 commits

Author SHA1 Message Date
Aanchal Khandelwal
7a72b1062d
Merge pull request #29528 from aanchal22/litellm_byok-alias-merge
fix(proxy): atomic merge for team model aliases and team.models on BYOK create

(cherry picked from commit ac7c2dc0d7)
2026-06-15 18:32:16 -07:00
tin-berri
5204ab5639
fix(mcp): allow team access-group grants in OAuth authorize/token access check (#30041)
* fix(mcp): honor team access-group grants in OAuth authorize/token access check

* test(mcp): mock build_effective_auth_contexts in non-admin authorize tests for isolation

(cherry picked from commit 5b7063d194)
2026-06-15 18:21:45 -07:00
Mateo Wang
4ec4ab99d0
feat(mcp): per-server env vars with global + per-user scopes (#28917) 2026-06-05 20:15:11 -07:00
yuneng-jiang
53cf3d8416
fix(proxy): drop deleted team BYOK model name from team.models (#29820)
Deleting a team-scoped BYOK model left its public name in team.models, so /models
with a team key kept listing the now-deleted "ghost" model. delete_model stripped
team.models using only litellm_modeltable alias lookups, but models added via
/model/new with a team_id never create an alias row; their public name lives only
in team.models and model_info.team_public_model_name, so it was never removed. The
team cache was also left stale because the delete path skipped _refresh_cached_team.

The cleanup now keys off team_public_model_name (falling back to alias keys), runs
after the deployment row is deleted, and strips a public name only when no remaining
team deployment still backs it, so a load-balanced replica is not revoked and
concurrent deletes cannot leave a ghost. The updated team row is refreshed in cache
so /models reflects the change immediately
2026-06-05 18:35:50 -07:00
Sameer Kankute
89f177b7b6
fix(galileo): use ingest traces API and standard logging payload (#29651)
* fix(galileo): use ingest traces API and standard logging payload

Switch hosted Galileo logging to /ingest/traces with nested trace/span payloads, read metrics from standard_logging_object, and include cost and total tokens on trace metrics.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(galileo): route username/password auth to v2 traces ingest

Hosted Galileo no longer serves /observe/ingest; JWT login should post the same trace payload to /v2/projects/{project_id}/traces.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(galileo): address Greptile review on logging and timestamps

Use debug-level logs for per-request Galileo callback messages and fall back to start_time/end_time when standard_logging_object omits startTime/endTime.

Co-authored-by: Cursor <cursoragent@cursor.com>

* feat(galileo): add Galileo to proxy UI callback configuration

Expose Galileo in the admin callback selector and config APIs so credentials can be configured through the dashboard instead of YAML only.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(galileo): align response type logging with Langfuse

Mirror Langfuse input/output handling for rerank, speech, transcription,
realtime, pass-through, and other response types so Galileo ingest no longer
skips supported call types.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(galileo): redact trace payload in debug logs and format with black

Avoid logging prompts and model responses in flush debug output while
keeping structural metadata for troubleshooting.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(galileo): stop logging full trace payload in debug output

Log only flush URL and trace count so prompts and model responses are not
written to application logs when debug logging is enabled.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Fix Galileo token totals and prompt messages

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-05 09:03:17 -07:00
michelligabriele
3f79222350
fix(proxy): persist oauth2_flow on MCP server registration (#29690) 2026-06-05 18:52:52 +05:30
yuneng-jiang
56aa55b991
fix(proxy): stop team BYOK model name corruption on model edit (#29731)
* fix(proxy): stop team model name corruption on edit (#28382) (#29001)

Team-scoped ("Team-BYOK") models store an internal routing key
model_name_{team_id}_{uuid} in the model_name column and the user-facing
name in model_info.team_public_model_name. The internal name leaked into
/v1, /v2, and /model/info responses; the dashboard bound its edit form to
it, so any non-rename save (e.g. a TPM tweak) PATCHed the internal name
back. The update path then treated it as a rename, overwriting
team_public_model_name and rewriting the team's models[] ACL with the
mangled string -- breaking team key calls with team_model_access_denied.

Two-layer fix:

- Read path (root cause): add _translate_model_name_for_response and apply
  it in model_info_v2 and _get_proxy_model_info so /v1, /v2, and
  /model/info surface the public name for team-scoped rows. The DB column
  and router index keep the internal name as the routing key; this is a
  presentation-layer swap on a shallow copy (never mutates input).

- Write path (defense in depth): harden _get_public_model_name so a value
  matching the internal shape, or a no-op against the current DB column,
  is never treated as a rename -- for both the top-level model_name and an
  explicit model_info.team_public_model_name.

Tests: regression for the reported scenario, full branch coverage of
_get_public_model_name, two internal-shape guard cases, an end-to-end
PATCH through _update_team_model_in_db (asserts the team ACL is untouched),
and four response-translation cases. 60 passed (model management),
181 passed (proxy server).

* fix(ui): key Agent Builder agent selection on model_info.id (#29729)

* fix(ui): key Agent Builder agent selection on model_info.id

Once team-scoped BYOK models can share a public name (the backend now
returns the public name on /model/info instead of the internal routing
key), selecting agents by model_name collides. Key selection, create,
update and delete on the stable model_info.id instead, falling back to
model_name only for config-defined agents that have no id.

* fix(ui): add name-match fallback to post-create agent selection

If the just-created agent's id is not yet present in the re-fetched
list, try matching by name before falling back to the first agent.
Addresses greptile review on #29729.

---------

Co-authored-by: tushar8408 <32977767+tushar8408@users.noreply.github.com>
2026-06-04 20:40:40 -07:00
yuneng-jiang
e9417603a3
fix(key_generate): scope session-token team-key budget exemption to caller-supplied team_id (#29641)
Some checks failed
Unit Tests: Proxy DB Operations / assert-shard-coverage (push) Has been cancelled
Unit Tests: Security / security (push) Has been cancelled
Unit Tests: Proxy DB Operations / auth-checks (push) Has been cancelled
Unit Tests: Proxy DB Operations / budgets (push) Has been cancelled
Unit Tests: Proxy DB Operations / custom-logging (push) Has been cancelled
Unit Tests: Proxy DB Operations / db-and-spend (push) Has been cancelled
Unit Tests: Proxy DB Operations / endpoints-and-responses (push) Has been cancelled
Unit Tests: Proxy DB Operations / guardrails-hooks (push) Has been cancelled
Unit Tests: Proxy DB Operations / jwt-and-keys (push) Has been cancelled
Unit Tests: Proxy DB Operations / key-generation (push) Has been cancelled
Unit Tests: Proxy DB Operations / logging-misc (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-runtime (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-server-core (push) Has been cancelled
Unit Tests: Proxy DB Operations / schema-migration (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-utils (push) Has been cancelled
#29612 exempts UI/CLI session tokens from the key budget ceiling when they
create a team key, keyed on data.team_id. That value is read after the
default_key_generate_params loop can populate team_id, so on deployments that
set default_key_generate_params.team_id a request the caller did not scope to a
team is treated as a team key and skips the ceiling. Capture _requested_team_id
before defaults run and key the exemption off it, mirroring how
_requested_max_budget is already captured. Requests the caller did not scope to a
team keep the ceiling.
2026-06-03 19:11:53 -07:00
yuneng-jiang
97ba7e1a30
fix(key_generate): exempt UI/CLI session tokens from the budget ceiling for team keys (#29612)
Non-admin users creating a team key through the UI were rejected with
"max_budget cannot exceed the caller's own max_budget (0.25)". The request is
authenticated by a UI/CLI session token whose max_budget is the per-session chat
spend cap (max_ui_session_budget, default $0.25), and the delegated-authority
budget ceiling (GHSA-q775-qw9r-2r4g) treated that cap as a delegation limit.

Skip the ceiling only when a session token creates a team key (data.team_id set);
that key's spend is bounded by the team budget at request time. Personal keys and
every other non-admin caller keep the ceiling, so a session token cannot mint an
arbitrary-budget personal key.
2026-06-03 14:07:59 -07:00
milan-berri
b11833c737
fix(key_generate): allow team members to create keys on org-scoped teams (#29310)
* fix(key_generate): allow team members to create keys on org-scoped teams

When a virtual key is created for a team, enterprise logic inherits the
team's organization_id onto the key (add_team_organization_id). Since the
VERIA-55 org-IDOR fix, /key/generate then required the caller to be an
explicit LiteLLM_OrganizationMembership member of that org, returning
403 "Caller is not a member of organization_id=<uuid>". Admins normally
only add users to teams (not orgs), so self-serve key creation regressed
for any user on an org-scoped team (regression since v1.84.0-rc.1).

Skip the org-membership check when organization_id was inherited from the
key's team (organization_id == team_table.organization_id). Team-level
authorization already gates this path, so team membership is sufficient.
The membership check still runs when a caller assigns an organization_id
that did not come from the key's team, preserving the IDOR protection.

Adds regression tests covering both the team-inherited (allowed) and
foreign-org (still blocked) cases.

Co-authored-by: Cursor <cursoragent@cursor.com>

* test(key_generate): cover mismatched team org IDOR path on generate

Add test_generate_key_foreign_org_with_mismatched_team_still_enforces_membership
for the case where a team is present but request organization_id differs from
team_table.organization_id. Enterprise inheritance is no-op'd in the test so
the guard is exercised directly; membership validation must still run.

Addresses Greptile review on #29310.

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-03 09:55:45 -07:00
ryan-crabbe-berri
0a767ed14f
fix(auth): let internal users view search tools (#29542)
* fix(auth): let internal users view search tools

Internal users could not see search tools in the UI even when an admin
created them, while vector stores were visible. The Search Tools page
rendered but its list calls 403'd because the read routes were not in
internal_user_routes. Grant internal users read-only access to the
listing and provider routes; create/update/delete stay admin-only.

Resolves LIT-3150

* fix(auth): scope /search_tools/list to caller-allowed tools

Adding the read routes to internal_user_routes let any internal user
call /search_tools/list, which returned every configured tool's id,
provider, api_base, and metadata regardless of the caller's
object_permission.search_tools allowlist. The api_key was masked, so
this was metadata disclosure rather than a credential leak, but it
ignored the key/team scoping that /search already enforces.

Filter the listing through the same can_key_call_search_tool /
can_team_call_search_tool checks (exposed as a boolean
can_user_view_search_tool), so non-admin callers only see tools they
may invoke; admins still see all. Mirrors how /vector_store/list
scopes results.

* fix(types): resolve mypy errors in list_search_tools

The config and DB build loops reused one loop variable, so mypy pinned
it to the config element type (SearchToolTypedDict); its .get() calls
returned object and the DB element (SearchTool) failed the reuse
assignment. Give each loop its own variable so each gets its real type,
and coerce the config tool's SearchToolInfoTypedDict to a plain dict to
match SearchToolInfoResponse.search_tool_info.
2026-06-03 01:36:14 +00:00
Sameer Kankute
4a81ec4982
feat(proxy): add per-MCP-server RPM rate limiting for keys and teams (#29482)
* feat(proxy): add per-MCP-server RPM rate limiting for keys and teams

Adds mcp_rpm_limit, a dict keyed by MCP server name (alias if set, else the
configured name) that caps requests per minute per server for a key or team.
The v3 rate limiter builds a per-server descriptor only when a limit is
configured for the server being called, so other servers stay uncapped and no
TPM reservation is engaged. Server identity is surfaced into the request data
via mcp_rate_limit_server_name so the limiter can resolve it.

* fix(proxy): gate MCP rpm descriptors on call_mcp_tool; document mcp_rpm_limit param

Only honor mcp_server_name when the call is an actual MCP tool call. Without
this, a normal LLM request could inject mcp_server_name in its body to consume
a target server's MCP quota and 429 legitimate tool calls. Also adds the
mcp_rpm_limit parameter docstring to update_key, new_user, and user_update so
the API docs validator passes.

* Fix MCP rate limit quota handling

* Delete scripts/test_mcp_rpm_limit.sh

* docs(proxy): clarify mcp_rpm_limit is enforced for keys and teams, not per user

* fix(proxy): accept mcp_rpm_limit in generate_key_helper_fn

NewUserRequest and GenerateKeyRequest inherit mcp_rpm_limit from
GenerateRequestBase, so /user/new and /key/generate forwarded the field
to generate_key_helper_fn, which did not accept it and returned a 500
("unexpected keyword argument 'mcp_rpm_limit'"). Accept the param and
store it in metadata, matching model_rpm_limit/model_tpm_limit, so the
limit is persisted where get_key_mcp_rpm_limit reads it.

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-06-02 12:52:10 -07:00
Mateo Wang
6d6eda8101
[internal copy of #28008] Support MCP OAuth passthrough and issuer-scoped JWT auth (#28356)
* fix(proxy): point /metrics 401 at the opt-out flag

Operators upgrading past 35bbca60b0 (which made /metrics auth
default-on) see "Malformed API Key passed in. Ensure Key has 'Bearer '
prefix." with no hint that
litellm_settings.require_auth_for_metrics_endpoint: false restores the
previous unauthenticated behavior. Append that discovery hint to the
existing 401 body so a Prometheus scraper that breaks after upgrade
has a clear migration path. No behavior change.

* fix(proxy): bound budget reservation per request instead of pinning to remaining headroom

reserve_budget_for_request fell back to reserving the entire remaining
team/key/user headroom whenever a request omitted max_tokens, which
pinned the spend counter at max_budget for the duration of the
in-flight request and false-positive-blocked every concurrent or
back-to-back request until the success callback reconciled. Surfaced
as an integration-test team being budget-blocked at its $2000 cap
while DB spend was $0.144.

Switch the missing-max_tokens path to a fixed default of 16384 output
tokens (mirrors parallel_request_limiter_v3's DEFAULT_MAX_TOKENS_ESTIMATE
precedent), and clamp explicit max_tokens at the model's
max_output_tokens for reservation accounting only. The outbound request
body is unchanged, so providers see whatever the caller actually sent;
only the local integer used to compute reservation cost is bounded.
This also prevents a hostile max_tokens=999999999 from inflating one
request's reservation up to the entire team headroom.

For Opus 4.7 (output $25/M, max_output 128K) on a $2000 budget the
worst-case per-request reservation drops from "everything left" to
$3.20, raising admittable concurrency from 1 to ~625.

* fix(proxy): reserve per-image cost for image-generation requests

Image-generation routes (dall-e-3, flux, etc.) have no per-token output
cost so they fell through to the no-reservation read-time-only path.
Concurrent image requests against a depleted budget could all pass
common_checks (counter exactly at max_budget passes the strict-`>`
gate) and reach the provider before reconciliation caught up.

Add per-image reservation in _estimate_request_max_cost_for_model:
when the model has a per-image cost field, reserve `n × cost_per_image`
upfront. The atomic counter increment serializes concurrent admissions,
so the second request sees the post-first-reservation counter and
raises BudgetExceededError instead of silently leaking through.

Both `output_cost_per_image` and `input_cost_per_image` are honored —
naming is inconsistent across providers (OpenAI dall-e-3 uses
input_cost_per_image, aiml/dall-e-3 uses output_cost_per_image for
the same per-generated-image price).

Per-pixel pricing (DALL-E 2 size variants) and TTS/STT routes still
fall through to read-time enforcement; those are follow-ups.

* fix(proxy): gate image-gen reservation strictly on model mode

The previous detection treated any model with input_cost_per_image
or output_cost_per_image as image generation. Several chat and
embedding models carry those fields to price multimodal vision input,
not generated images:

- gemini-3.1-pro-preview (mode=chat) has output_cost_per_image=0.00012
  alongside input/output token pricing.
- azure/gpt-realtime-* (mode=chat) has input_cost_per_image=5e-6.
- amazon.titan-embed-image-v1 (mode=embedding) has
  input_cost_per_image=6e-5.

For these models the image-gen branch fired first and reserved a
fraction of a cent per request, short-circuiting the token-priced
path entirely. Long Gemini chats reserved 1 × $0.00012 instead of
the true token cost.

Gate strictly on mode in {"image_generation", "image_edit"}. All 197
real image_generation entries and all 31 image_edit entries
(Flux Kontext, Stability inpaint/outpaint, etc.) carry the right mode,
so the field-presence fallback was unnecessary.

Adds regression tests for the chat-model-with-image-cost-field case
and for image_edit reservation.

* build(packaging): relax core runtime pins to ranges

Backport of #27241 onto litellm_1.84.0rc2.

The 12 entries in `[project.dependencies]` were exact `==` pins, a side
effect of the Poetry -> uv migration. This forces every downstream
package that lists litellm as a dependency to downgrade common runtime
libraries (openai, pydantic, aiohttp, click, jsonschema, ...) to the
exact versions we ship.

Switch to lower-bounded ranges with upper bounds where the upstream
package is pre-1.0 or has a known breaking-major-version policy.
Reproducibility for our Docker proxy and CI continues to come from
`uv.lock`, which is regenerated here as a metadata-only diff.

Conflict resolution vs upstream merge:
- The upstream merge commit also surfaced unrelated context entries
  (nvidia-riva-client, soundfile/stt-nvidia-riva extra) that exist in
  staging but not in rc2. Those are not part of #27241's intent and
  were dropped from the resolution; the rc2 uv.lock keeps its existing
  entry set, only the 12 specifier strings changed.
- `uv lock --check` passes (392 packages resolved, no drift).

* build(packaging): raise jinja2 floor to 3.1.6

Our `uv.lock` already resolves jinja2 to 3.1.6, so Docker / CI installs
get that version. The `pyproject.toml` floor was lagging at 3.1.0,
which means downstream consumers using `--resolution=lowest-direct` or
older constraint files can land on 3.1.0-3.1.5 instead of the version
we actually test against.

Aligns the declared floor with the resolved version so external
installers see the same baseline our test matrix exercises.

`uv lock` diff is metadata-only (no resolved-version drift).

* fix(mcp): forward extra_headers for OpenAPI MCP tools

OpenAPI-generated tools only applied static closure headers and BYOK
Authorization via ContextVar. Copy MCPServer.extra_headers from the
incoming MCP request into _request_extra_headers (set in server.py before
local tool dispatch), merge in openapi_to_mcp_generator via a small helper.

OAuth2 M2M: do not forward caller Authorization from raw_headers (same rule
as _prepare_mcp_server_headers for managed MCP).

Adds TestRequestExtraHeaders and clarifies mcp_server_manager registration
comment.

Fixes #26794

Co-authored-by: Cursor <cursoragent@cursor.com>

* refactor(mcp): access has_client_credentials on MCPServer directly

Greptile: getattr default was redundant; property exists on MCPServer and
mcp_server is non-None inside the extra_headers forwarding block.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(mcp): static headers win over forwarded headers in OpenAPI MCP

Match the existing MCP invariant in merge_mcp_headers and the managed MCP
path: operator-configured static headers always override caller-forwarded
headers on name conflict, with case-insensitive comparison so different
casing cannot bypass the precedence. _request_auth_header (BYOK) still
overrides Authorization last.

Addresses Veria review on PR #27383.

Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>

* fix(proxy): always merge caller-supplied tags into request metadata

Caller-supplied tags (`x-litellm-tags` header, body `tags`, `metadata.tags`)
were silently dropped unless the key/team had
`metadata.allow_client_tags: true` set. Restore the documented behavior:
tags from the request always flow into `metadata.tags` and union with any
admin-configured static tags from key/team/project metadata.

Removes the `allow_client_tags` opt-in flag from the pre-call pipeline.
The flag was only ever read here; it has no schema or endpoint footprint,
so leftover values in existing key metadata are inert.

Test cleanup mirrors the simplification: drop the three tests that
verified the strip-when-not-opted-in path, drop the `allow_client_tags`
fixture lines from the merge/union tests.

* docs(proxy): refresh stale comments referencing removed tag strip

The tag-strip block was removed in the parent commit but two surrounding
comments still referenced "tags without opt-in" and "runs AFTER the
strip". Update them to describe the remaining user_api_key_* and
_pipeline_managed_guardrails strip that the snapshot/merge ordering
actually protects against.

* chore: reject bare str at file-input sinks to prevent local-file read (#27762)

Cherry-pick of #27762 onto litellm_1.84.0rc2.

* chore: reject bare str at file-input sinks to prevent local-file read (#27667)
* fix: use os.PathLike in ocr sink and check truthy reasoningSummary for bridge
  - ocr/main.py: widen Path check to os.PathLike for consistency with other sinks
  - main.py: bridge condition checks truthiness of reasoning_summary, not just None
* fix: remove unused pathlib.Path import in ocr/main.py

Co-authored-by: yuneng-jiang <yuneng@berri.ai>
Co-authored-by: ryan-crabbe-berri <ryan@berri.ai>
Co-authored-by: stuxf <70670632+stuxf@users.noreply.github.com>

* Strip SERVER_ROOT_PATH before lazy-feature prefix match

LazyFeatureMiddleware compared the raw scope path against registered
prefixes (e.g. /policies), so requests under a server root path like
/api/v1/policies/... never matched, the feature never loaded, and the
endpoint returned 404. Strip the configured root path before matching,
normalizing trailing slashes and enforcing a component boundary so
/api does not falsely match /apiv2.

* Cache normalized SERVER_ROOT_PATH at middleware init

SERVER_ROOT_PATH is a process-startup env var. Read it once in
__init__ instead of calling get_server_root_path() + rstrip on every
request that arrives before all lazy features have loaded.

* chore(proxy): backport /key/regenerate ownership-rebind + premium-gate guards (#27793)

Backport of #27793 onto litellm_1.84.0rc2.

A non-admin caller could rebind their own key's user_id via /key/regenerate.
_execute_virtual_key_regeneration had org/team guards but no user_id guard,
and prepare_key_update_data did not strip the field — it survived
model_dump(exclude_unset=True) into the Prisma update. On the next request,
_return_user_api_key_auth_obj resolved the rebound user_id against
litellm_usertable and returned PROXY_ADMIN whenever the target row's
user_role was admin.

/key/update had the equivalent guard inline at _validate_update_key_data;
extract it to a shared helper _validate_caller_can_change_key_ownership and
call from both /key/update and _execute_virtual_key_regeneration.

Also tighten the premium gate that allowed the master-key rotation branch to
skip the enterprise check. The previous predicate was a field-presence test,
not an identity check. Verify the caller actually holds the master key via
_is_master_key before allowing the non-premium path.

Block explicit-null user_id and empty-string user_id as removal attempts;
both 403-reject for non-admin callers.

* fix(proxy): expose db status on public /health/readiness

Backport of #27866 onto litellm_1.84.0rc2.

External readiness probes consumed the legacy detailed payload's `db`
field to drive alerting and pod-rotation decisions. Stripping the body
to {"status": "healthy"} broke those probes silently — the HTTP code
still flipped to 503, but probes checking body.db == "connected"
treated the response as healthy.

Add `db` back to the unauthenticated payload. The rest of the diagnostic
fields (litellm_version, callbacks, cache, log_level) stay behind
/health/readiness/details so the recon-leak gate from #26912 holds.
Values match the legacy contract: "connected", "disconnected",
"Not connected". The 503-on-DB-disconnect behavior from LIT-2607 is
preserved.

* fix(ui): fetch version + debug flag from /health/readiness/details

The proxy moved `litellm_version`, `is_detailed_debug`, and other
diagnostic fields off the public `/health/readiness` payload behind
an auth-gated `/health/readiness/details` endpoint. The navbar
version tag and the detailed-debug-mode banner stopped working
because they were still reading those fields from the unauthed
response, which no longer contains them.

Replace `useHealthReadiness` with a `useHealthReadinessDetails`
hook that takes an `accessToken` argument and sends a Bearer header
to the auth-gated endpoint. The hook stays disabled while
`accessToken` is falsy, so the navbar can keep rendering on the
public model hub (where the token is null) without triggering an
auth redirect or a 401-loop.

* fix(ui): disable retries on readiness/details + cover token forwarding

Two small follow-ups on the readiness/details migration:

- Set `retry: false` on the query. The payload feeds a passive
  navbar tag and a debug banner; a 401 from an expired token
  shouldn't fan out into three retries against the proxy.
- Add navbar specs that assert the `accessToken` prop is forwarded
  into the hook (matches the DebugWarningBanner spec). Without
  this, the navbar could silently regress to passing `undefined`
  and the existing tests wouldn't catch it.

* chore: update Next.js build artifacts (2026-05-14 03:52 UTC, node v20.20.2)

* Merge pull request #27898 from stuxf/chore/banned-params-extra-body-cover

chore(proxy): cover extra_body + azure_ad_token in banned-params check

(cherry picked from commit a6a9d8edf0)

* Merge pull request #27801 from stuxf/chore/get-instance-fn-runtime-s3-gate

chore(proxy): refuse remote-URL instance-fn loads outside config-file path

(cherry picked from commit e3e5209f51)

* fix: block client-side pricing injection via request body

Authenticated clients could supply CustomPricingLiteLLMParams fields
(input_cost_per_token, output_cost_per_token, etc.) in the request body.
These were forwarded to register_model() in main.py, permanently mutating
the shared global litellm.model_cost dict for all users on the instance.

Adds all CustomPricingLiteLLMParams fields to _BANNED_REQUEST_BODY_PARAMS
so is_request_body_safe() rejects them before they reach completion().
New pricing fields added to CustomPricingLiteLLMParams are auto-covered.

Admin opt-in via allow_client_side_credentials or
configurable_clientside_auth_params still works as before.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

* fix: block SSRF fields in RAG ingest vector_store config

aws_sts_endpoint, aws_web_identity_token, and aws_bedrock_runtime_endpoint
in ingest_options.vector_store were passed directly to the Bedrock ingestion
class, which reads them into boto3 STS client construction. Any authenticated
caller could redirect AssumeRole calls to an attacker-controlled server,
leaking the proxy's instance profile credentials.

Calls is_request_body_safe() on ingest_options["vector_store"] before
forwarding to litellm.aingest(). Same banned-params list and admin opt-in
escape hatch (allow_client_side_credentials) as the /chat/completions path.
ValueError from the safety check is caught and re-raised as HTTP 400.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

* fix: harden /key/update authorization checks (#27878)

* fix: patch Host-header auth bypass in get_request_route

Starlette reconstructs request.url from the Host header. A malformed
Host like `localhost/?x=1` causes Starlette to build the full URL as
`http://localhost/?x=1/health`, which url-parses to path="/". Since "/"
is in LiteLLMRoutes.public_routes, all protected routes became reachable
without authentication.

Fix: read scope["path"] (set by uvicorn from the HTTP request line,
not derivable from headers) instead of request.url.path. Sub-path
deployments are handled via scope["app_root_path"] / scope["root_path"],
mirroring Starlette's own base_url construction logic.

Affected variants confirmed fixed:
  Host: localhost/?x=1
  Host: localhost:4000/?x=1
  Host: localhost/#test
  Host: localhost:4000/#test

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

* style: reduce comments in route fix

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

* fix: block credential fields in RAG ingest vector_store options

Credential fields (vertex_credentials, aws_access_key_id, api_key, etc.)
in ingest_options.vector_store are now rejected at the API boundary with
a 400 error. Credentials must be configured server-side.

Previously any authenticated user could supply a vertex_credentials dict
with type=external_account pointing credential_source.file at an
arbitrary path (e.g. /proc/1/environ) and token_url at an
attacker-controlled server. google-auth's identity_pool.Credentials
refresh() would read the file and POST its contents to the attacker.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

* fix: block /key/update self-escalation by assigned users

Non-admin users who were assigned a key (created_by != caller) could
update any non-budget field — models, rpm_limit, guardrails, etc. —
without admin authorization, allowing privilege self-escalation.

Gate: only the key creator (created_by == caller) may edit their own
key without admin check; budget changes always require admin regardless
of creator status. All other callers must pass _check_key_admin_access.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

* fix: block user-controlled api_base in RAG ingest vector_store options

A user-supplied api_base in ingest_options.vector_store caused the server
to forward its configured provider credentials (Gemini, OpenAI) to an
attacker-controlled endpoint via SSRF.

Add api_base to the blocked credential params set alongside api_key and
the existing credential fields.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

* fix: restrict /utils/transform_request to PROXY_ADMIN and apply body safety check

Any authenticated internal_user could POST arbitrary provider config
(aws_sts_endpoint, api_base, etc.) to /utils/transform_request and have
the server forward its credentials to an attacker-controlled endpoint.

- Gate the endpoint on PROXY_ADMIN role (403 for all other roles)
- Call is_request_body_safe() to reject banned params even for admins
- Convert ValueError from safety check to HTTP 400

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

* fix: apply banned-param check to /utils/transform_request

Without is_request_body_safe(), any authenticated user could pass
aws_sts_endpoint, api_base, or aws_web_identity_token to
/utils/transform_request and have the server forward its configured
provider credentials to an attacker-controlled endpoint during SDK
credential resolution.

Applies the same banned-param blocklist already used by LLM endpoints.
Endpoint remains accessible to all authenticated users.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

* fix: block SSRF via api_base in /prompts/test dotprompt YAML frontmatter

Any frontmatter key not in ["model","input","output"] flowed into
optional_params and was merged into the LLM call data dict, bypassing
is_request_body_safe. An attacker with any bearer key could set
api_base in YAML to redirect the outbound LLM request — including the
provider API key — to an attacker-controlled host.

Fix: call is_request_body_safe on the constructed data dict after
optional_params are merged, before invoking ProxyBaseLLMRequestProcessing.
ValueError from the banned-param check is surfaced as HTTP 400.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

* Update litellm/proxy/rag_endpoints/endpoints.py

Co-authored-by: veria-ai[bot] <224490171+veria-ai[bot]@users.noreply.github.com>

* fix: coerce nested config strings before banned-param check

_NESTED_CONFIG_KEYS descent used isinstance(nested, dict) which silently
skipped litellm_embedding_config when delivered as a JSON string via
multipart/form-data. Banned params (api_base, aws_sts_endpoint, etc.)
nested inside the stringified value were invisible to is_request_body_safe.

_NESTED_METADATA_KEYS already used _coerce_metadata_to_dict which parses
JSON strings before checking. Apply the same coercion to _NESTED_CONFIG_KEYS.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

* fix: replace substring match with prefix match in is_llm_api_route

mapped_pass_through_routes used `_llm_passthrough_route in route` (substring)
so any admin-only path whose URL contained a provider name (openai, anthropic,
azure, bedrock, etc.) was misclassified as an LLM API route and bypassed the
admin gate in non_proxy_admin_allowed_routes_check.

Confirmed live: non-admin key could GET /credentials/by_name/openai (read
masked provider API key) and DELETE /credentials/openai (delete credential).

Fix: use exact match or startswith(prefix + "/") — the same pattern used
everywhere else in RouteChecks — so only routes that actually start with a
passthrough prefix are allowed through.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

* fix: stabilize PR #27878 test failures

- key_management_endpoints: extend can_skip_admin_check to team keys so
  team members with /key/update permission can update non-budget fields.
  can_team_member_execute_key_management_endpoint already validates team
  membership + permission and raises if unauthorized; reaching the admin
  check on a team key means the caller was authorized.

- test: set created_by on mock key in
  test_update_key_non_budget_fields_allowed_for_internal_user so
  caller_is_creator resolves correctly (MagicMock default ≠ user_id).

- auth_utils.get_request_route: guard against non-dict request.scope
  (e.g. MagicMock in unit tests) to prevent a MagicMock leaking into
  UserAPIKeyAuth.request_route and failing Pydantic validation.

- ci: assign test_multipart_bypass_repro.py to the proxy-runtime shard
  in test-unit-proxy-db.yml to satisfy the shard-coverage check.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

* fix(lint): add explicit str() cast in get_request_route for MyPy

scope.get() returns Any|None which MyPy cannot coerce to str implicitly.
Wrap both scope.get() calls in str() to satisfy the type checker.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

* fix: guard bare-/ root_path strip + make total_spend migration idempotent

auth_utils.get_request_route: when Starlette sets scope["app_root_path"]
to "/" (e.g. behind some middleware), the old stripping logic would
remove the leading slash from every path ("/team/new" → "team/new"),
breaking route matching and causing auth to misclassify protected routes.
Skip stripping when root_path is bare "/".

migration: add IF NOT EXISTS to total_spend ALTER TABLE so the migration
is safe to replay when a prior partial run already created the column.
Without this guard, prisma migrate deploy fails on CI DBs that were
partially migrated, causing all subsequent DB operations (including
/team/new) to 500.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

* fix: require creator still owns key for personal-key bypass in /key/update

caller_is_creator now requires both created_by == caller AND user_id ==
caller. Previously checking only created_by let a demoted admin who
originally created a key for another user continue editing non-budget
fields on it after reassignment, bypassing _check_key_admin_access.

Adds regression test: creator whose key was reassigned is blocked (403).

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

* fix: extract auth checks to fix PLR0915 + broaden max_budget assertion

internal_user_endpoints._update_single_user_helper exceeded 50 statements
(PLR0915). Extract authorization checks into _check_user_update_authz helper
to bring statement count under the limit.

test_validate_max_budget: assert "negative" (substring of both the local
"cannot be negative" and the CI "non-negative finite number" messages) so
the test is stable regardless of which exact wording the function uses.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: veria-ai[bot] <224490171+veria-ai[bot]@users.noreply.github.com>

* bump: version 0.4.71 → 0.4.72

* uv lock

* feat(mcp): support OAuth passthrough discovery

* fix(mcp): support OAuth browser auth

* fix(mcp): refine upstream OAuth metadata fallback

* feat(proxy): support issuer-scoped JWT auth

* fix(mcp): validate oauth callback redirect sink

* feat(proxy): support issuer-scoped JWT auth

* test(mcp): align trusted proxy fixtures

* style(mcp): satisfy black formatting

* chore(ui): bump next to 16.2.6

* fix(mcp): address oauth passthrough review findings

* test(mcp): split oauth passthrough regressions

* fix(interactions): align openapi response fields

* security: prevent forwarding litellm api keys to upstream mcp servers

- Strip Authorization header from extra_headers for pass-through servers
- Pass-through servers (auth_type=None with extra_headers: [Authorization])
  must not receive the user's LiteLLM API key
- Only OAuth2 M2M and pass-through servers skip Authorization header
- Other headers (x-request-id, x-trace-id) are still forwarded normally
- Fixes credential leakage / authentication bypass in MCP pass-through mode

* fix(interactions): remove steps field not in google openapi spec

The steps field was added but is not present in the current Google
Interactions OpenAPI specification. Revert to using only the fields
that are actually defined in the spec.

* fix(mcp): forward Authorization in pass-through when x-litellm-api-key is admission

Commit 3753970cc9 widened the Authorization strip to cover all
is_oauth_passthrough servers — protecting against the LiteLLM admission
key leaking upstream when the caller used Authorization for admission,
but also silently stripping legitimate upstream OAuth bearers when the
caller used x-litellm-api-key for admission.

That broke transparent OAuth pass-through (EAI-506 V5/V6): standards-
compliant MCP clients (OpenCode, Claude Code, mcp-inspector) complete
PKCE against the upstream IdP and send the resulting token as plain
Authorization: Bearer per the MCP spec — with the wider strip in place,
that token never reaches the upstream and tools/list returns empty.

Narrow the strip: skip Authorization for pass-through servers only when
the caller did NOT supply x-litellm-api-key. When x-litellm-api-key is
present, admission is unambiguous and Authorization is free to carry
the upstream OAuth bearer.

The original security guarantee is preserved — a client that sends only
Authorization (no x-litellm-api-key) still has it stripped, so the
LiteLLM key cannot leak upstream via that path.

Tests:
- new: forwards Authorization when x-litellm-api-key is present
- new: still strips Authorization when only Authorization is present
- existing pass-through + M2M tests unchanged

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(interactions): align status enum with openapi spec

* fix(mcp,jwt): address greptile review concerns

- Cache _get_agent_object_permission via user_api_key_cache (sentinel for
  no-permission rows) so MCP requests from agent keys don't hit the DB on
  every tool-list / tool-call.
- Re-raise HTTPException in handle_sse_mcp so 401 + WWW-Authenticate
  challenges (and other HTTP errors) propagate to SSE clients instead of
  being swallowed as 500.
- Normalise booleans in _validate_token_response so admin rules written as
  JSON-style "true" / "false" match upstream responses that return
  Python True / False.
- Treat configured JWT issuer claim mappings as advisory: when a mapped
  field is absent or empty, leave the normalised claim unset instead of
  raising, matching the global litellm_jwtauth path.

Co-authored-by: Claude <noreply@anthropic.com>

* test: replace dall-e-3 with gpt-image-1 in health check and router tests (#27813)

OpenAI returns 'The model dall-e-3 does not exist' for the test account,
breaking test_openai_img_gen_health_check and test_image_generation.
Switch to gpt-image-1, matching the existing TestOpenAIGPTImage1 pattern.

(cherry picked from commit aee58db880)

* fix(tests): drop dall-e-only test classes; route live image tests via gpt-image-1

Second wave of failures from the 2026-05-12 DALL-E shutdown:
- tests/image_gen_tests/test_image_edits.py::TestOpenAIImageEditDallE2
  and tests/image_gen_tests/test_image_generation.py::TestOpenAIDalle3
  are explicitly named for the deprecated models and can't pass; remove.
  gpt-image-1 coverage already exists in sibling classes.
- tests/local_testing/test_router.py image gen tests use dall-e-3 only
  as a routing example; swap to gpt-image-1.
- tests/local_testing/test_custom_callback_input.py image_generation
  success/failure paths swapped to gpt-image-1.

(cherry picked from commit 945b10ded4)

* test(fireworks): replace deprecated llama-v3p3-70b-instruct model

Fireworks removed llama-v3p3-70b-instruct from serverless, so every
live test using it now fails with NotFoundError ("Model not found,
inaccessible, and/or not deployed").

Swap the 6 references (3 files) to the currently-served
accounts/fireworks/models/deepseek-v3p1 — the canonical model in
Fireworks' current docs examples and present in LiteLLM's cost map.
test_get_model_params_fireworks_ai is a pure pricing-heuristic test
(no network) asserting the >16b branch, so it uses llama-v3p1-70b-
instruct instead to keep the "fireworks-ai-above-16b" assertion and
branch coverage intact.

(cherry picked from commit 39a1d438f2)

* test(fireworks): mock remaining live smoke tests

test_completion_fireworks_ai and test_completion_cost_fireworks_ai
made real Fireworks calls and broke whenever Fireworks rotated its
serverless catalog (no externally-verifiable model list exists).
They also asserted nothing — just printed.

Mock the HTTP post and assert real behavior instead: the request is
built with the right model/messages and the OpenAI-compatible
response parses back; the cost path yields a non-zero cost against
the local cost map. No network, no model dependency, stronger than
the old smoke checks.

(cherry picked from commit b5db7ed37d)

* fix(tests): replace shut-down gpt-4o-audio-preview with gpt-audio-1.5 (#28281)

* fix(tests): replace shut-down gpt-4o-audio-preview with gpt-audio-1.5

OpenAI shut down gpt-4o-audio-preview on 2026-05-07, so the live audio
calls in test_stream_chunk_builder_openai_audio_output_usage and
test_standard_logging_payload_audio now hard-fail with a model-not-found
error on every PR. The error was not "openai-internal", so the except
block swallowed it and execution fell through to an unbound
completion/response (UnboundLocalError).

Switch both tests to gpt-audio-1.5, OpenAI's recommended successor
(GA, not deprecated, already present in the litellm cost map so the
response_cost assertion still resolves). Also broaden the except to
skip with the real error in the reason instead of crashing, so a
transient upstream blip can't reintroduce the UnboundLocalError.

* fix(tests): narrow audio-test skip to model-not-found, re-raise the rest

Address review feedback: an unconditional skip on any exception would
silently mask a litellm-internal regression in the audio path (broken
param transformation, serialization, bad header) instead of failing CI.

Skip only on the upstream-unavailable class (model_not_found / "does not
exist" / openai-internal) and re-raise everything else, so genuine
regressions still fail loudly. The UnboundLocalError is still fixed
because the handler either skips or raises - it never falls through.

* fix(tests): add budget_exceeded to expected Interaction status enum

Staging added budget_exceeded to the Interaction OpenAPI status enum; the staging merge into this branch picked up the spec change but not the matching test update, so test_status_enum_values failed in CI. Align the test's expected list (exact-match by design) with the live spec.

* fix(tests): mock HTTP fetch in test_img_url_token_counter

The test parameterized a live third-party image URL (blog.purpureus.net) which now 404s, causing get_image_dimensions to fall through to its base64 decode path and crash with 'not enough values to unpack' on every PR run. Mock safe_get with a tiny 1x1 PNG so the URL branch is still exercised without any network dependency.

* fix(tests): swap gpt-4o-audio-preview to gpt-audio-1.5 in test_gpt4o_audio

OpenAI shut down gpt-4o-audio-preview on 2026-05-07, so both live tests in test_gpt4o_audio.py (test_audio_output_from_model and test_audio_input_to_model) hard-fail model_not_found on every PR. Swap the hardcoded model to OpenAI's successor gpt-audio-1.5 (same chat-completions audio surface; already in the litellm cost map). Mirror the narrowed-skip pattern from the prior audio fixes: skip on model_not_found / does-not-exist / openai-internal, re-raise everything else so genuine litellm regressions still fail CI loudly.

(cherry picked from commit 92de7423ef)

* fix(tests): migrate realtime + rerank tests off shut-down upstream models (#28191)

* fix(tests): use gpt-realtime in realtime guardrails test

OpenAI shut down gpt-4o-realtime-preview-2024-12-17 on 2026-05-07, so
the live OpenAI realtime guardrails integration test now fails with
model_not_found (session.created never arrives, _wait_for_event times
out). Point OPENAI_REALTIME_URL at the current GA model, gpt-realtime.

Scope limited to this test: the pricing-catalog JSON keeps the retired
entries intentionally (historical cost calc + separate Azure timeline),
and the Azure realtime cost-calc test is unaffected.

* fix(tests): mock nvidia_nim rerank instead of hitting EOL'd endpoint

NVIDIA reached end-of-life for the hosted nvidia/llama-3.2-nv-rerankqa-1b-v2
rerank API on 2026-05-18 with no published replacement, so the live
BaseLLMRerankTest.test_basic_rerank for nvidia_nim now returns HTTP 410
("Gone"). NVIDIA's hosted catalog rotates on a schedule, so swapping in
another live model would only defer the failure.

Override test_basic_rerank in TestNvidiaNim to mock the sync/async HTTP
transport (same pattern as test_nvidia_nim_rerank_ranking_endpoint in this
file) and inject a fake NVIDIA_NIM_API_KEY via monkeypatch. The
request/response transformation and cost calculation stay covered offline.
Scope limited to nvidia_nim; other BaseLLMRerankTest providers untouched.

* fix(tests): migrate remaining realtime tests off shut-down gpt-4o-realtime-preview

OpenAI's 2026-05-07 shutdown removed the entire gpt-4o-realtime-preview
family, including the undated 'gpt-4o-realtime-preview' alias (not just the
dated snapshot fixed earlier). Three live tests still connected with the
dead alias and failed with messages_received=1 (an error event instead of
session.created):

- test_openai_realtime_simple.py: get_model() -> gpt-realtime (drives
  TestOpenAIRealtime.test_realtime_connection / test_realtime_with_query_params)
- test_openai_realtime.py: test_openai_realtime_direct_call_no_intent and
  test_openai_realtime_direct_call_with_intent -> openai/gpt-realtime
  (the with_intent test shares the same dead alias even though it was not
  in the failing set this run)

Mocked unit tests (test_realtime_query_params_construction,
test_realtime_query_params_use_normalized_model_name) are left as-is: they
never hit the network and assert string plumbing only.

Also fixes test_text_message_blocked_by_guardrail_no_ai_response, which now
connects (the earlier URL swap worked) but tripped a model-wording-brittle
assertion. The guardrail flow asks the model to voice the block message
verbatim; gpt-4o-realtime-preview complied (output contained 'blocked'),
gpt-realtime refuses verbatim-repeat instructions ('I'm sorry, but I can't
repeat that message.'). Since the original user message is blocked before
it reaches OpenAI, the refusal is still a safe outcome. Assertion #3 now
accepts both voicing and refusal, and adds a hard check that the blocked
phrase never leaks into AI output.

(cherry picked from commit ce87c411bf)

* fix(model_prices): register mistral/ministral-8b-2512

Mistral's API now returns model='ministral-8b-2512' when 'mistral-tiny'
is requested, so test_completion_mistral_api fails with 'This model
isn't mapped yet'. Adding the entry so completion_cost can resolve the
cost for that response.

Author: Claude <noreply@anthropic.com>

* fix(mcp,auth): address greptile review concerns

- handle_sse_mcp now calls _raise_preemptive_401_for_unauthenticated_servers
  so SSE clients to pass-through OAuth MCP servers receive the RFC 9728
  401 + WWW-Authenticate challenge that the streamable-HTTP path already emits.
- get_request_route strips a trailing slash from root_path before length-based
  prefix removal so non-canonical ASGI root_path values like "/litellm/"
  don't strip the leading slash from the returned route.
- _mcp_oauth_user_api_key_auth's cookie JWT decode now passes
  options={"verify_aud": False} so a future revision of the UI session
  JWT containing an aud claim cannot silently downgrade the request to
  unauthenticated.

Co-authored-by: Claude <claude@anthropic.com>

* fix(tests): backfill local model_cost into remote-fetched map

litellm.model_cost is loaded at import time from LITELLM_MODEL_COST_MAP_URL
(pinned to main), so pricing entries that exist only in this branch (e.g.
mistral/ministral-8b-2512, freshly added because Mistral's API now returns
this id from mistral-tiny) are absent at test time and completion_cost
lookups raise 'This model isn't mapped yet'. Backfill the in-tree backup
into litellm.model_cost in the local_testing conftest so cassette-driven
cost calculations resolve against the entries that ship with the branch
under test.

Fixes local_testing_part1 failures on test_completion_mistral_api and
test_completion_mistral_api_modified_input.

* fix(mcp,jwt): address greptile concurrency and code-quality concerns

- _apply_issuer_claim_mappings now builds a new dict and reads from the
  original token, rather than mutating its input. The change is
  behaviour-preserving (caller passes a fresh jwt.decode result), but
  avoids the surprise-mutation pattern flagged by greptile.
- is_network_error uses isinstance(exc, httpx.TransportError) instead of
  matching type(exc).__name__ against a hand-maintained string set, so
  ReadError / WriteError / ProxyError / etc. are also treated as
  transport-level failures and surfaced as HTTP 502.
- fetch_upstream_oauth_protected_resource now coalesces concurrent
  discovery requests per (server_id, resource_url) through an
  asyncio.Lock so concurrent .well-known calls share a single upstream
  fetch + cache write.
- Drop the redundant 'if trusted_ranges:' branch in get_mcp_client_ip;
  it is always true on the path that reaches it (the prior 'if not
  trusted_ranges:' early-returns).

Co-authored-by: Claude <claude@anthropic.com>

* fix(jwt,mcp): fall back to global JWKS on unknown issuer; prune fetch locks

- handle_jwt._get_configured_issuer now returns None for tokens whose 'iss'
  is not in the configured issuers list, letting auth_jwt fall through to
  the legacy JWT_PUBLIC_KEY_URL path instead of hard-raising. This keeps
  existing tokens from non-configured IdPs working when an operator adds
  the new 'issuers' list to a live deployment.

- discoverable_endpoints._prune_oauth_metadata_cache now also prunes
  entries in _OAUTH_METADATA_FETCH_LOCKS whose cache entry has been
  evicted and whose lock isn't currently held, bounding the locks dict
  to match the cache it guards.

Co-authored-by: Claude <claude@anthropic.com>

* fix(mcp,auth): restore client_ip in oauth2 target check, drop from delegate check

The merge of staging into the PR branch (d42a66adb6) misplaced the
client_ip=client_ip kwarg: it landed inside _target_servers_delegate_auth_to_upstream
(which never accepted client_ip and isn't called with it), while the
sibling _target_servers_use_oauth2 has client_ip in its signature but
stopped passing it through to get_mcp_server_by_name. That left ruff
flagging F821 on the undefined name and lint failing.

Move client_ip back into _target_servers_use_oauth2's lookup (matching
the call site that already forwards IPAddressUtils.get_mcp_client_ip)
and drop it from _target_servers_delegate_auth_to_upstream so its body
matches its signature again.

* fix(mcp): respect client ip for delegated auth

* fix(auth): address remaining greptile style findings

- get_request_route: require root_path to match whole path segments before
  stripping, so '/apifoo' isn't truncated to 'foo' when root_path='/api'.
- get_mcp_client_ip: collapse the two trusted-proxy validation branches into
  a single is_request_from_trusted_proxy call so the return value drives
  control flow instead of being discarded for the side-effect warning.

Co-authored-by: Claude <claude@anthropic.com>

* fix(jwt): strip internal _litellm_* claims in global JWKS auth path

Prevents identity spoofing where a token signed by the global JWKS
could inject _litellm_jwt_issuer and other _litellm_* claims that
downstream getters trust. The issuer-scoped path already strips these
via _apply_issuer_claim_mappings; mirror that behavior for the global
fallback path.

Co-authored-by: Yassin Kortam <yassin@berri.ai>

* fix(mcp): surface MCPUpstreamAuthError as 401 in SSE/HTTP transport handlers

Both handle_sse_mcp and handle_streamable_http_mcp only caught
HTTPException to preserve 401 + WWW-Authenticate challenges, but
MCPUpstreamAuthError (raised when a pass-through server's upstream
rejects a bearer token mid-session) inherits from Exception. It was
falling through to the generic handler and surfacing as an opaque 500.

Mirror the REST endpoint behavior: translate MCPUpstreamAuthError into
an HTTPException(status_code=e.status_code) with the upstream
www-authenticate header so standards-compliant MCP clients trigger the
upstream OAuth flow.

Co-authored-by: Yassin Kortam <yassin@berri.ai>

* fix(mcp): add upstream auth pre-flight in SSE handler

Mirror handle_streamable_http_mcp by calling _check_passthrough_upstream_auth
after the cold-start 401 emitter so expired/invalid upstream tokens surface a
proper 401 + WWW-Authenticate challenge before the SSE session commits 200
headers, instead of letting list_tools silently return [] when the upstream
rejects the token.

Co-authored-by: Claude <noreply@anthropic.com>

* fix(mcp): tighten cold-start bypass against CSV paths + dedupe upstream auth probe

- Return None from _parse_mcp_server_names_from_path for CSV multi-server
  paths (/mcp/a,b). The regex previously truncated at the first comma and
  silently passed a single server name to the cold-start gate.
- Switch _is_mcp_passthrough_cold_start to all-targets semantics, matching
  _target_servers_use_oauth2: one non-passthrough target in a co-targeted
  set must not flip the anonymous-admission bypass open for the others.
- Drop the redundant HTTPStatusError block in _extract_upstream_auth_failure
  - any HTTPStatusError carries a .response, so the preceding generic block
  already handles 401/403 detection.

Co-authored-by: Yassin Kortam <yassin@berri.ai>

* fix(mcp,tests): sync stubs and cold-start assertions with delegate-check

The merge of base-branch _target_servers_delegate_auth_to_upstream
into process_mcp_request inserts an additional
get_mcp_server_by_name(name) lookup ahead of the cold-start path,
which breaks two test patterns:

1. lookup_by_name(name) side-effect stubs in
   TestMCPDelegateAuthToUpstream are called positionally by the
   delegate check, then again by the cold-start path with
   client_ip=... — raising TypeError: unexpected keyword argument
   'client_ip'. Accept **_kwargs to match the real signature.

2. TestMCPPassthroughColdStartAdmission assertions count the lookup
   exactly once with client_ip=..., but the delegate check now adds
   a positional-only call ahead of it. Switch assert_called_once_with
   to assert_any_call for the cold-start invocation, and assert
   client_ip was *not* passed for the aggregate /mcp test where
   cold-start must not fire.

Both updates align with CLAUDE.md guidance to keep monkeypatch stubs in
sync with the real signature when an optional parameter is added.

Co-authored-by: Claude <claude@anthropic.com>

* fix(mcp): correct passthrough probe 401 + slashed-name cold start parser

- _check_passthrough_upstream_auth now emits
  'Bearer resource_metadata="..."' pointing at the gateway's
  oauth-protected-resource well-known URL, mirroring the
  pre-emptive 401 path. Pass-through servers don't use the gateway
  as an authorization server, so the previous 'authorization_uri='
  challenge sent clients to the wrong metadata endpoint.

- _parse_mcp_server_names_from_path now accepts server names that
  contain a single slash (e.g. custom_solutions/user_123), mirroring
  MCPRequestHandler._extract_target_server_names_from_path. Without
  this, the cold-start bypass missed slashed-name servers and the
  generic admission error propagated instead of the spec-compliant
  401 challenge.

- _is_mcp_passthrough_cold_start drops the unused scope parameter
  from its signature.

Co-authored-by: Yassin Kortam <yassin@berri.ai>

* style(mcp): format discoverable endpoints

* refactor(mcp): dedupe MCPUpstreamAuthError->HTTPException + thread client_ip into delegate-auth gate

Co-authored-by: Yassin Kortam <yassin@berri.ai>

* fix(mcp): handle passthrough OAuth metadata and startup auth errors

- discoverable_endpoints: For pass-through MCP servers, when upstream
  oauth-protected-resource returns a non-200/non-dict response, raise
  HTTP 502 instead of falling through to default gateway metadata.
  Falling through would direct MCP clients at the gateway, which is
  not the authorization server for pass-through configs.

- mcp_server_manager: Wrap _get_tools_from_server in startup tool name
  mapping with try/except. Since _get_tools_from_server now re-raises
  MCPUpstreamAuthError, an upstream 401 from a pass-through server at
  startup (when no user token is present) would otherwise abort the
  loop and leave subsequent servers unmapped.

Co-authored-by: Yassin Kortam <yassin@berri.ai>

* fix(mcp): restrict passthrough probe challenge to OAuth passthrough servers

The probe filter previously matched any server with Authorization in
extra_headers, including gateway-managed OAuth2 servers. Those would
then receive the resource_metadata= WWW-Authenticate challenge meant
for pass-through servers, instead of the authorization_uri= challenge
pointing at the gateway AS metadata. Use srv.is_oauth_passthrough so
only genuine pass-through servers get the resource-metadata challenge.

Co-authored-by: Yassin Kortam <yassin@berri.ai>

* test(proxy): cover issuer-scoped JWT auth

* fix(mcp): use resource metadata for passthrough reauth

* fix(mcp,tests): assert cold-start helper directly for aggregate /mcp

Threading client_ip into _target_servers_delegate_auth_to_upstream
made get_mcp_server_by_name(name, client_ip=...) also fire from the
delegate-auth check, so the call_args_list assertion on
client_ip-in-kwargs no longer uniquely signals a cold-start lookup.
Patch _is_mcp_passthrough_cold_start and assert it is not invoked,
which is the actual contract the test is pinning.

* fix(mcp,jwt): drop unneeded async helper + suppress misleading unscoped JWT warning

- _build_oauth_authorization_server_response: revert to sync (no awaits in body).
  The function only does dict construction and synchronous registry lookups;
  async added coroutine creation overhead per discovery call without need.
- _build_decode_kwargs: accept has_issuer_config so the global path's
  'JWT auth is unscoped' warning is suppressed when LiteLLM_JWTAuth.issuers
  provides per-issuer scoping. Previously the warning fired spuriously for
  admins who intentionally use only the new issuers config.

* fix(jwt,mcp): clarify issuers fallthrough + add TTL on mcp permission cache

- LiteLLM_JWTAuth.issuers docs now state explicitly that unlisted
  issuers fall back to the global JWT_AUDIENCE/JWT_ISSUER path; the
  field is additive routing, not an allow-list. Matches actual
  control flow in handle_jwt.auth_jwt and the regression tests
  asserting backwards compatibility with the global JWKS path.
- MCPRequestHandler._get_{org,agent}_object_permission now pass
  ttl=DEFAULT_MANAGEMENT_OBJECT_IN_MEMORY_CACHE_TTL on async_set_cache,
  mirroring the auth_checks.py pattern so the cache TTL is explicit
  on both DualCache layers.

* fix(tests): align merged JWT and MCP cold-start assertions

Update the tests carried over from PR #28008 to match the assertions on
the staging branch:

- tests/test_litellm/proxy/auth/test_handle_jwt.py: unknown issuers now
  fall back to the legacy JWT_PUBLIC_KEY_URL path (per
  litellm_feat/v1.84.0-mcp-gateway-jwt-auth's
  '\''fall back to global JWKS on unknown issuer'\''), and mapped issuer
  claims that are absent no longer fail closed — they simply leave the
  normalised LiteLLM internal claim absent.

- tests/test_litellm/proxy/_experimental/mcp_server/auth/test_user_api_key_auth_mcp.py:
  the aggregate '\''/mcp'\'' route still triggers the delegate-auth-to-upstream
  lookup once for the header-supplied server name; cold-start admission
  must NOT fire on top of that. Tighten the assertion to
  assert_called_once_with so a future regression that re-enters cold-start
  is caught.

Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>

* fix(jwt): guard litellm_jwtauth access in auth_jwt global path

JWTHandler() can be constructed without update_environment() being
called (tests do this directly), in which case self.litellm_jwtauth
does not exist. Accessing it raises AttributeError before getattr can
fall back. Use the same safe pattern other call sites use.

* Gate MCP OAuth pass-through on delegate_auth_to_upstream flag

Sameer's review on #28356/#28008 flagged that the new pass-through
behaviors (preemptive 401 challenges, /.well-known/oauth-protected-
resource proxying, upstream 401/403 propagation as MCPUpstreamAuthError,
and Authorization-stripping when no x-litellm-api-key is supplied)
were implicitly enabled for every server with auth_type=none plus
Authorization in extra_headers. Existing users doing static bearer
pass-through for non-OAuth reasons would have silently regressed.

Make the detection rule explicit: extend the existing
delegate_auth_to_upstream flag (previously oauth2-only) to also gate
is_oauth_passthrough. Now requires flag + auth_type=None + Authorization
in extra_headers, per Sameer's suggested detection rule. The UI toggle
now appears for both modes (oauth2 PKCE passthrough and auth_type=none
OAuth pass-through) with mode-appropriate copy.

Update test fixtures to set the flag where the test intent is to
exercise OAuth pass-through behavior, and add negative tests covering
the new default-false case.

* fix(mcp): route org object_permission lookup through shared auth helpers

Replace the bespoke litellm_organizationtable.find_unique + dedicated
cache key in _get_org_object_permission with get_org_object +
get_object_permission so MCP requests share the same user_api_key_cache
entries as the rest of the proxy and no longer fragment org-row caching.

* fix(mcp): wrap get_object_permission call in shared try/except

Ensure exceptions from get_object_permission in _get_org_object_permission are caught and return None, preserving the original fail-safe semantics.

Co-authored-by: Yassin Kortam <yassin@berri.ai>

* fix(jwt): validate issuer audience at config load + dedicated key-miss exception

- Move JWTIssuerConfig audience-required guard into a Pydantic model_validator
  so misconfiguration fails at startup instead of on the first request.
- Replace the string-match `No matching public key found` filter in
  get_public_key's multi-URL fallback with a dedicated
  NoMatchingJWTPublicKeyError; only that specific exception triggers
  continuation, every other error still surfaces.

* fix(mcp): admit and forward Authorization for passthrough OAuth return

For pass-through MCP servers (auth_type=none with delegate_auth_to_upstream)
the RFC 9728 cold-start flow sends the client back with only
"Authorization: Bearer <upstream-token>" after upstream OAuth discovery.
Previously this path 1) was rejected in process_mcp_request because the
oauth2_headers fallback only covered auth_type=oauth2 targets, and 2) had
the Authorization header stripped by _prepare_mcp_server_headers when no
x-litellm-api-key was present, treating the upstream token as a potential
LiteLLM key leak.

- Extend the elif oauth2_headers fallback to also admit anonymously when
  every target is a pass-through server.
- Pass user_api_key_auth into _prepare_mcp_server_headers so it can
  forward Authorization for pass-through servers when admission did not
  consume the bearer as a LiteLLM key (api_key is unset).

Co-authored-by: Yassin Kortam <yassin@berri.ai>

* fix(mcp): consistent www-authenticate casing + SSE toolset scoping

- Normalize the WWW-Authenticate header key emitted by
  _check_passthrough_upstream_auth to lowercase to match the other 401
  emitters in the OAuth pass-through flow.
- Mirror the streamable HTTP handler's toolset scoping in handle_sse_mcp:
  strip client-supplied x-mcp-toolset-id and apply _apply_toolset_scope
  before _check_passthrough_upstream_auth so the upstream probe list is
  derived from the fully-authorized server set.
- Tighten _has_client_supplied_mcp_auth signature so
  mcp_server_auth_headers is Optional, matching its caller in
  process_mcp_request.

Co-authored-by: Yassin Kortam <yassin@berri.ai>

* security(mcp): strip Authorization in call_tool when LiteLLM admission used legacy header

Mirror the OAuth pass-through admission check from _prepare_mcp_server_headers
(list-tools path) in _call_regular_mcp_tool (tool-call path): when the server
is OAuth pass-through and the caller did not supply x-litellm-api-key,
Authorization on the inbound request may itself be the LiteLLM API key — so
strip it before forwarding instead of leaking the gateway credential upstream.

When x-litellm-api-key is present, admission is unambiguous and Authorization
continues to carry the upstream OAuth bearer (transparent pass-through).

* refactor(mcp): centralize caller Authorization strip decision

Extracted the security-sensitive logic that decides whether the caller's
Authorization header is forwarded to (or stripped from) an outgoing MCP
request into a single helper, _should_strip_caller_authorization, in
mcp_server_manager.py.

Previously the same condition was duplicated across
_call_regular_mcp_tool (mcp_server_manager.py) and
_prepare_mcp_server_headers (server.py). Keeping two copies of this
check risked future divergence and credential-leak / broken-passthrough
bugs. Both call sites now share the helper, preserving exact behavior.

Co-authored-by: Yassin Kortam <yassin@berri.ai>

* log MCP OAuth discovery diagnostics for unmatched paths and non-transport upstream errors

* fix(jwt): include issuer-normalized team id in get_all_jwt_team_ids

The aggregator for team IDs only consulted the issuer-normalized claim
for the plural (team_ids) path and fell back to the global config for
the singular path. When an operator configures team_id_jwt_field only
at the issuer level, get_team_id correctly returned the mapped value
but get_all_jwt_team_ids silently dropped it, causing membership
reconciliation to disagree with request routing.

Co-authored-by: Yassin Kortam <yassin@berri.ai>

* fix(mcp/jwt): dedupe cold-start path parser; reject conflicting audience flags

- _parse_mcp_server_names_from_path now delegates to
  MCPRequestHandler._extract_target_server_names_from_path so the
  names used by the cold-start passthrough bypass cannot drift from the
  names used by downstream routing.
- JWTIssuerConfig now rejects the combination of audience and
  disable_audience_validation=True at validation time instead of
  silently ignoring the flag.

* fix(mcp): restrict passthrough cold-start bypass to 401 only

The new elif passthrough cold-start branch reused is_auth_error which
matches both 401 and 403. A 403 from user_api_key_auth indicates the
LiteLLM key WAS recognized but is forbidden (e.g. over budget / rate
limited); falling through to anonymous UserAPIKeyAuth() in that case
bypasses spend and rate-limit controls on passthrough servers.

Only trigger the cold-start anonymous admission on 401, which is the
signal that the bearer is an upstream OAuth token rather than a
recognized LiteLLM key.

Co-authored-by: Yassin Kortam <yassin@berri.ai>

* fix(jwt/mcp): warn on unscoped JWT fallback; route agent permission lookup through shared helper

- _build_decode_kwargs no longer suppresses the unscoped-fallback warning
  when LiteLLM_JWTAuth.issuers is set: tokens whose iss does not match
  any configured issuer still fall through to the global path, and that
  fallback is itself unscoped when JWT_AUDIENCE/JWT_ISSUER are absent.

- _get_agent_object_permission now caches the agent_id ->
  object_permission_id mapping and delegates the permission lookup to
  the shared get_object_permission helper, so the agent path reuses the
  same cache entries as the org / team / key paths.

* fix(mcp): fabricate resource_metadata challenge when upstream 401 omits WWW-Authenticate

When an upstream pass-through MCP server returns 401 without a
WWW-Authenticate header (non-compliant per RFC 7235 §3.1),
to_http_exception() now produces a synthetic Bearer challenge pointing
at the gateway's standard-pattern oauth-protected-resource well-known
endpoint for that server. This keeps MCP clients on the RFC 9728
discovery flow instead of receiving a bare 401 with no recovery hint.

* fix(jwt): make _get_decode_options explicitly control verify_iss

Previously, _get_decode_options only set verify_aud based on whether
audience was provided. The issuer JWT path relied on always passing
issuer=issuer_config.issuer to trigger PyJWT's default verify_iss=True,
making the helper's behavior implicitly dependent on caller behavior.

Now _get_decode_options accepts issuer as well, mirroring the verify_aud
handling and matching the dimensions handled by _build_decode_kwargs.

Co-authored-by: Yassin Kortam <yassin@berri.ai>

* fix(mcp): emit absolute resource_metadata URI in fabricated 401 challenge

Per RFC 9728 §3.2 the resource_metadata Bearer challenge must be an
absolute URI; strict MCP clients reject relative URIs and fail to
initiate discovery. MCPUpstreamAuthError.to_http_exception now accepts
the gateway base URL and prepends it when the upstream omitted
WWW-Authenticate, and all four call sites (streamable HTTP, SSE, and
the two REST tool-list paths) supply it.

* fix(mcp): correct 403 detail text and remove dead _list_tools_for_single_server duplicate

- MCPUpstreamAuthError.to_http_exception() now returns detail='Forbidden' for
  403 upstream responses (and 'Unauthorized' for 401), matching the
  _check_passthrough_upstream_auth pre-flight probe.
- Remove the shadowed first definition of _list_tools_for_single_server in
  rest_endpoints.py; the second definition was the live one and the dead copy
  was a maintenance trap.

Co-authored-by: Yassin Kortam <yassin@berri.ai>

* fix: address potential bugs in auth_utils, mcp discoverable endpoints, and mcp auth

- auth_utils.get_request_route: return '/' instead of empty string when
  raw_path exactly equals root_path so downstream route allowlist checks
  still see a leading slash
- discoverable_endpoints.fetch_upstream_oauth_protected_resource: also
  cache negative results (no upstream metadata) for a shorter TTL so we
  don't re-fetch on every discovery request and so the per-key fetch
  lock can be pruned
- user_api_key_auth_mcp: guard the oauth2_headers 401 cold-start
  passthrough bypass with _has_client_supplied_mcp_auth, matching the
  parallel bypass in the no-Authorization branch so MCP-auth-bearing
  requests don't silently downgrade to anonymous admission

Co-authored-by: Yassin Kortam <yassin@berri.ai>

* test(vertex): tolerate transient InternalServerError in google maps tool test

test_gemini_google_maps_tool_simple makes live calls to Vertex AI's Google
Maps grounding backend, which intermittently returns 500 INTERNAL ("Please
retry") — a transient upstream failure, not a LiteLLM bug. The test already
passes on RateLimitError; treat InternalServerError the same way so transient
Vertex-side failures don't fail CI.

* refactor(mcp): drop redundant has_client_credentials filter on passthrough probe

is_oauth_passthrough already requires auth_type in (None, MCPAuth.none),
which is mutually exclusive with has_client_credentials (auth_type ==
MCPAuth.oauth2), so the extra guard was always True and only added
confusion about whether a server could be both passthrough and M2M.

Co-authored-by: Yassin Kortam <yassin@berri.ai>

* fix: restore unreachable InternalServerError skip handler in vertex test

Co-authored-by: Yassin Kortam <yassin@berri.ai>

* feat(mcp): add dedicated oauth_passthrough flag for non-oauth2 pass-through

Previously is_oauth_passthrough reused delegate_auth_to_upstream — a flag
scoped to oauth2 servers (PKCE bypass) — to gate OAuth pass-through for
auth_type=none servers. Overloading it risked regressing existing
deployments that set delegate_auth_to_upstream, since the same flag would
silently start driving pass-through (discovery proxying, 401 challenges,
upstream 401/403 propagation) on non-oauth2 servers.

Introduce a separate oauth_passthrough opt-in so the two behaviors never
imply each other:
- MCPServer.is_oauth_passthrough now requires oauth_passthrough (not
  delegate_auth_to_upstream).
- Persist oauth_passthrough on LiteLLM_MCPServerTable (new column +
  migration) and wire it through config/DB load and API responses.
- UI splits the single toggle into two: "Delegate auth to upstream (PKCE
  passthrough)" for oauth2 and "OAuth pass-through" for auth_type=none
  servers forwarding Authorization.

Adds backend tests (property, round-trip, and a regression guard that
delegate_auth_to_upstream alone never enables pass-through) and UI tests
for the toggle split.

* fix(mcp): reconcile cold-start bypass with x-mcp-servers header and skip non-absolute WWW-Authenticate fabrication

- _parse_mcp_server_names_from_path now fails closed when the
  x-mcp-servers header introduces any target not present in the
  path-derived target set, closing a header/path mismatch where the
  cold-start passthrough bypass could otherwise admit anonymously
  while the header advertises a non-passthrough server.
- MCPUpstreamAuthError.to_http_exception no longer emits a relative
  resource_metadata URI when base_url is missing; per RFC 9728 3.2
  the URI must be absolute, so we skip fabrication entirely rather
  than send a challenge strict MCP clients will reject.

Co-authored-by: Yassin Kortam <yassin@berri.ai>

* fix(mcp): fabricate path-aware resource_metadata URI for upstream 401

When MCPUpstreamAuthError.to_http_exception fabricates a
`WWW-Authenticate: Bearer resource_metadata=...` challenge (because
the upstream 401 omitted one), the URL now matches the inbound MCP
transport pattern the client originally used:

  - /mcp/{server_name}      -> /.well-known/oauth-protected-resource/mcp/{server_name}
  - /{server_name}/mcp      -> /.well-known/oauth-protected-resource/{server_name}/mcp

This mirrors the path-aware behaviour of
_get_passthrough_resource_metadata_url in server.py so strict
RFC 9728 \xA73.2 clients on legacy routes get a resource_metadata URI
aligned with the resource pattern they originally targeted.

Co-authored-by: Yassin Kortam <yassin@berri.ai>

* fix(jwt+mcp): tighten issuer-scoped claim type handling, RFC-quote authorization_uri, surface MCP upstream auth errors, defense-in-depth on decode options

- handle_jwt: when an issuer-scoped _litellm_team_ids claim exists but
  has an unexpected type, return [] instead of falling through to the
  global team_ids_jwt_field path (different claim semantically).
- handle_jwt: _get_decode_options/_decode_jwt_with_public_key now take
  an explicit disable_audience_validation flag; passing audience=None
  without it raises, so audience checks can't silently disappear if the
  model validator is ever bypassed. _auth_jwt_with_issuer forwards the
  flag from JWTIssuerConfig.
- mcp_server: quote the authorization_uri WWW-Authenticate parameter
  value (RFC 6750 / 9728 auth-param must be quoted-string), matching
  the pass-through path.
- mcp_server: in _fetch_and_filter_server_tools, re-raise
  MCPUpstreamAuthError so the outer streamable-HTTP handler can surface
  a proper 401 + WWW-Authenticate challenge instead of returning an
  empty tool list.

Co-authored-by: Yassin Kortam <yassin@berri.ai>

* chore(docker): align Dockerfile.non_root/Dockerfile.database to current wolfi-base SHA

The older sha256:3258be... pin has been intermittently returning 500/not-found
from cgr.dev, breaking the test-server-root-path GitHub Action and the
build_docker_database_image CircleCI job. Move both Dockerfiles onto the
same sha256:31da65... digest already in use by Dockerfile, gateway/Dockerfile,
backend/Dockerfile, and migrations/Dockerfile so the base image is consistent
across the repo.

* ci(docker): bump wolfi-base pin to current working digest

The previously aligned sha256:31da6565f35a... and the older sha256:3258be...
both return HTTP 500 from cgr.dev's manifest endpoint, breaking the
build_docker_database_image CircleCI job and test-server-root-path GitHub
Action. The current 'latest' tag resolves to sha256:5743937d521c... which
serves manifests normally, so move docker/Dockerfile.database and
docker/Dockerfile.non_root onto that digest.

* ci(docker): retry apk add in Dockerfile.database for apk.cgr.dev flakes

Mirror the retry-loop pattern from #28888 (which fixed backend/Dockerfile,
gateway/Dockerfile, and migrations/Dockerfile) into docker/Dockerfile.database.
The build_docker_database_image CI job has been intermittently failing with
"remote server returned error (try 'apk update')" when apk.cgr.dev flakes
mid-fetch; bumping the wolfi-base SHA doesn't address the mirror, only a
retry does.

Same explicit-failure form as #28888: exit non-zero on the 3rd miss instead
of silently succeeding because `sleep 5` was the last command in the
`&& break || sleep 5` chain.

* fix(mcp): scope preemptive 401 to toolset-narrowed server set

Move _raise_preemptive_401_for_unauthenticated_servers after toolset
scoping in both the StreamableHTTP and SSE handlers, and add an
optional allowed_server_ids parameter so passthrough/oauth2 servers
that the active toolset excludes no longer trigger a spurious 401
challenge. Without this, a client targeting a toolset whose scope
excludes a passthrough server could be pushed into an OAuth flow for
a server it would be 403'd on immediately after authentication.

Co-authored-by: Yassin Kortam <yassin@berri.ai>

* revert(docker): drop unrelated Wolfi bump and apk retry loop from MCP/JWT PR

These Docker changes are out of scope for the MCP OAuth passthrough + JWT
auth work and duplicate the build-reliability fix already merged to
litellm_internal_staging in #28888, which adds the same apk retry loop on
the componentized backend/gateway/migrations Dockerfiles and also fixes the
underlying nodeenv/libatomic root cause. Restoring docker/Dockerfile.database
and docker/Dockerfile.non_root to the base so this PR is purely the MCP/JWT
change.

* fix(mcp): surface upstream 403 challenges from REST tools/list

The single-server pass-through path converted an upstream MCPUpstreamAuthError
into an HTTPException, but list_tool_rest_api only re-raised 401s; an upstream
403 (valid token, insufficient scope) collapsed into a 200 response with
error=unexpected_error, so clients never saw the status or WWW-Authenticate
challenge needed to refresh scopes. Let MCPUpstreamAuthError propagate and
convert it once in list_tool_rest_api so both 401 and 403 reach the client,
while internal access/IP 403s keep the legacy error-dict shape.

* fix(mcp): fail closed for IP access control when XFF trusted ranges unset

When use_x_forwarded_for is enabled but mcp_trusted_proxy_ranges is not
configured, get_mcp_client_ip previously fell back to the direct peer IP.
Behind an internal reverse proxy that peer is the proxy's private address,
so every external caller was classified as internal and could reach MCP
servers with available_on_public_internet=false. Return an empty string in
that case so is_internal_ip treats the caller as external.

---------

Co-authored-by: Yuneng Jiang <yuneng@berri.ai>
Co-authored-by: Milan <milan@berri.ai>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
Co-authored-by: Krrish Dholakia <krrish+github@berri.ai>
Co-authored-by: ryan-crabbe-berri <ryan@berri.ai>
Co-authored-by: stuxf <70670632+stuxf@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: veria-ai[bot] <224490171+veria-ai[bot]@users.noreply.github.com>
Co-authored-by: gym-cmd <186399764+gym-cmd@users.noreply.github.com>
Co-authored-by: Artem Dudarev <artem.dudarev@justeattakeaway.com>
Co-authored-by: Claude <claude@anthropic.com>
Co-authored-by: Yassin Kortam <yassin@berri.ai>
2026-06-02 12:22:04 -07:00
Sameer Kankute
83fcacad08
fix(mcp): ignore stale ids on key save (#29128)
* fix(mcp): ignore stale server ids during key permission validation

Prevent virtual key save failures when object_permission still includes MCP server IDs that were deleted from the registry. The validator now drops stale IDs before team scope checks and adds coverage for stale-ID and active unauthorized-ID behavior.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(mcp): use DB find_many to detect stale server IDs, add tool-permission test

Addresses greptile feedback on PR #29128:

P1 – Race-condition / authoritative-signal concern:
  Extract _get_stale_mcp_server_ids() helper that performs a single
  DB find_many (LiteLLM_MCPServerTable) when prisma_client is available,
  making server-existence checks authoritative even during in-memory
  registry warm-up. Falls back to the in-memory registry only when no
  DB client is present (config-only deployments / unit tests).
  Pass prisma_client through both call sites in key_management_endpoints.

P2 – Missing test coverage for mcp_tool_permissions:
  Add test_validate_stale_ids_in_mcp_tool_permissions_silently_dropped
  to confirm stale server IDs that appear only as keys in
  mcp_tool_permissions are also stripped without raising 403.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(mcp): also check in-memory registry when DB is present to avoid misclassifying config-file servers as stale

Config-file MCP servers are never written to LiteLLM_MCPServerTable, so a
DB-only query would classify them as stale and silently drop their IDs from
the authorization check, allowing unauthorized key access.

Fix: treat a server ID as stale only when it is absent from BOTH the DB and
the in-memory registry (which holds config-file servers).

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(mcp): normalize server aliases on key save

Co-authored-by: Sameer Kankute <Sameerlite@users.noreply.github.com>

* fix: remove unused stale MCP helper and capture normalized object_permission on key generation

Co-authored-by: Yassin Kortam <yassin@berri.ai>

* test(mcp): keep validate_key_mcp_servers_against_team stub in sync with return contract

validate_key_mcp_servers_against_team now returns the (normalized) object_permission, and the key-generation helper assigns that return value back into the request data. The two key-generation tests stubbed the function with a bare AsyncMock, whose default MagicMock return value clobbered object_permission and skipped permission-record creation. Make the stubs pass object_permission through unchanged.

* fix(mcp): preserve provided object_permission fields on key update

The key-update path reconstructs data.object_permission from a full
model_dump(), which marks every field as set. Downstream
model_dump(exclude_unset=True) then emits models/blocked_tools/
search_tools as None, and those are non-nullable array columns, so the
Prisma write fails whenever the UI submits object_permission with only a
subset of fields populated (e.g. a TPM/RPM-only edit). Build the dict
with exclude_unset=True so the normalized object_permission keeps the
caller's original field set.

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Sameer Kankute <Sameerlite@users.noreply.github.com>
Co-authored-by: Yassin Kortam <yassin@berri.ai>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-05-29 20:50:24 -07:00
michelligabriele
68852ef165
fix(teams): expose keys_count on /v2/team/list and wire UI Resources badge (#28502) 2026-05-29 17:39:07 +05:30
ryan-crabbe-berri
f75a7c6b22
fix(model-edit): allow clearing custom pricing on wildcard models (#28719)
* fix(model-edit): allow clearing custom input/output cost on wildcard deployments

A user-set pricing override on a `/model/*` wildcard deployment could not
be removed: clearing the Input/Output Cost fields in the UI succeeded
visually, but the next read still showed the old values because both
`litellm_params` and `model_info` (mirrored via `SPECIAL_MODEL_INFO_PARAMS`)
retained the original rates.

UI: when the pricing field is touched but left empty, send `null` instead
of dropping it from the payload so the backend sees the clear intent. The
cache-read-cost fallback now guards against `null` as well as `undefined`
so a cleared input cost cannot silently wipe the cache-read override.

Backend: `update_db_model` honors explicit-null clears, but ONLY for
`SPECIAL_MODEL_INFO_PARAMS` (the 4 pricing fields). Restricting the
null-clear path prevents a team-scoped caller from using this codepath to
null out privileged fields like `team_id` or access groups.

Tests cover both clear paths (`litellm_params` and `model_info`), the
SPECIAL_MODEL_INFO_PARAMS mirror, PATCH semantics for omitted fields, and
the security guard that non-pricing nulls don't reach the merged dict.

Resolves LIT-3250

* fix(model-edit): run null-clears after both merges, not interleaved

The previous version cleared `model_info` from inside the litellm_params
merge block, but the subsequent `model_info.update(...)` re-injected the
old pricing because the UI's PATCH carries the full model_info blob with
the stale values still in it. Move the explicit-null clear pass to after
both merges so a model_info passthrough cannot resurrect cleared fields.

Adds a regression test for the realistic UI submit shape (both blobs in
the patch, model_info still holding the old pricing).

* test(e2e): clear-custom-pricing flow with create/delete cleanup

Covers the dashboard model edit form's pricing-clear flow end-to-end:
seeds a deployment with custom input/output pricing, drives the UI to
clear both fields, asserts the outgoing PATCH sends explicit nulls,
and confirms via /v2/model/info that the override is gone from both
litellm_params and model_info.

The dashboard DB persists across this suite, so beforeEach creates a
uniquely-named deployment and afterEach POSTs /model/delete to leave
the DB clean regardless of test outcome.

* fix(model-edit): extend pricing clear to cache_read and cache_write costs

Pre-existing parallel of the wildcard input/output cost bug: cleared
cache_read_input_token_cost and cache_creation_input_token_cost overrides
silently persisted because the UI omitted the key (delete or fallback) and
the backend null-clear allowlist did not cover them.

- types/router.py: add cache_read_input_token_cost and
  cache_creation_input_token_cost to SPECIAL_MODEL_INFO_PARAMS, so they are
  mirrored between litellm_params and model_info by Deployment.__init__ and
  honoured by the null-clear loop in update_db_model.
- model_info_view.tsx: emit explicit null for touched-but-empty cache_read
  and cache_write fields. Preserve the input_cost->cache_read mirror only
  when cache_read itself was not touched.
- model_management_endpoints.py: update the allowlist comment.
- Tests: three new unit tests for cache clear paths and a preserve check;
  the e2e spec now seeds, clears, and asserts null PATCH + key-absence for
  all four pricing fields.
2026-05-26 09:37:23 -07:00
yuneng-jiang
5f73ad4fe7
fix(team): refresh team cache on team_model_add/delete (LIT-3244) (#28683)
* fix(team): refresh team cache on team_model_add/delete (LIT-3244)

team_model_add and team_model_delete wrote to the DB but did not
invalidate the in-memory LiteLLM_TeamTableCachedObj used by
common_checks. After the v1.83.14 common_checks centralization made
team.models authoritative on /v1/files and /v1/vector_stores/*,
adding a Team-BYOK model silently failed to grant the new public
model name to team members until the cache TTL expired (and a
removed model kept working until then on the symmetric path).

Extract the cache-refresh snippet from update_team into a small
helper and apply it consistently at all three team-write sites.

* test: also assert updated models in team-cache-refresh pin

Strengthens the LIT-3244 regression test to also assert
`call_kwargs["team_table"].models` matches the updated row,
not just `team_id`. Both `existing_team` and `updated_team`
share `team_id` in the test setup, so the previous assertion
would have passed even if the implementation accidentally cached
the pre-mutation row.

Greptile review feedback.

* fix(team): hydrate object_permission on cache-refreshing team updates

The Prisma update calls in update_team, team_model_add, and
team_model_delete returned a team row with object_permission_id set
but object_permission=None (the relation was not requested via
include=). _refresh_cached_team then wrote that to the in-memory
LiteLLM_TeamTableCachedObj, and the cache-hit path in get_team_object
returns the cached object without re-hydrating. Downstream consumers
(validate_key_search_tools_against_team, the MCP/agent authz paths)
treat a missing object_permission as no team-level restriction, so
a team-write op silently dropped object-permission enforcement until
the cache TTL expired or a DB-fetch path re-hydrated it.

Add include={"object_permission": True} to all three updates so the
refresh writes a complete cached team. Extend the LIT-3244 regression
test to pin both the cached object_permission and the include shape
on the Prisma call.

Surfaced in PR review of LIT-3244.
2026-05-23 16:41:05 -07:00
Michael-RZ-Berri
3b2ce201d8
encrypt callback_vars in key/team metadata at rest (#27141)
Co-authored-by: Michael Riad Zaky <michaelr@Michaels-MacBook-Air.local>
Co-authored-by: Yuneng Jiang <yuneng@berri.ai>
2026-05-23 12:15:44 -07:00
Mateo Wang
b0b25ae4b9
Include team alias in CLI JWT token (#28621) 2026-05-22 10:40:59 -07:00
Sameer Kankute
50a3f10a92
feat(proxy): persist allowlisted OIDC claims in CLI SSO poll (#28463)
* feat(proxy): persist allowlisted OIDC claims in CLI SSO poll

Map CLI_SSO_CLAIM_MAP sources into user metadata and return scalar
attribution_metadata from /sso/cli/poll. Build SSOUserDefinedValues in
cli_sso_callback so first-time CLI logins can upsert users. Add mock OIDC
scripts and tests for claim extraction and poll exposure.

Co-authored-by: Cursor <cursoragent@cursor.com>

* docs(proxy): document CLI SSO attribution_metadata in client README

Co-authored-by: Cursor <cursoragent@cursor.com>

* Delete scripts/mock_oidc_server_for_cli_sso.py

* Delete scripts/test_cli_sso_claims_e2e.py

* fix(ui_sso): preserve claim types and avoid metadata. prefix stripping

- Replace _update_dictionary with a local recursive merge so string
  OIDC claim values that happen to look numeric are not silently coerced
  to int/float when persisting CLI SSO attribution metadata.
- Use a local dot-path resolver in _extract_sso_claim_value so that
  source claim paths beginning with 'metadata.' are not silently stripped
  by get_nested_value (which is designed for LiteLLM JWT metadata, not
  arbitrary OIDC claims).

Co-authored-by: Yassin Kortam <yassin@berri.ai>

* Remove redundant metadata. prefix strip in _set_nested_metadata_value

The _parse_cli_sso_claim_map already strips the metadata. prefix from
dest keys before reaching the setter. The duplicate strip in
_set_nested_metadata_value was a no-op in normal flow but could
mis-place values for dest keys like metadata.metadata.foo.

Co-authored-by: Yassin Kortam <yassin@berri.ai>

* Fix greptile

* Fix ruff

* Move CLI SSO user defined values build inside try/except for consistent error handling

Co-authored-by: Yassin Kortam <yassin@berri.ai>

* fix(proxy): enforce restricted SSO group on CLI SSO callback

Apply verify_user_in_restricted_sso_group before CLI session completion
and user upsert, matching the UI SSO path. Re-raise ProxyException so
restricted-group denials return 403 instead of 500.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(proxy): replace recursive CLI SSO metadata helpers with iterative merge

Use stack-based flatten/merge to satisfy recursive_detector CI. Fix mypy
types for UserApiKeyCache and user_id on CLI SSO session completion.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix: resolve nested CustomOpenID extra_fields in CLI SSO claim extraction

When GENERIC_USER_EXTRA_ATTRIBUTES captures a parent object (e.g. org_info),
extra_fields stores it as {"org_info": {"department": "..."}}. A CLI claim
map entry using a dotted path like org_info.department would silently fail
because the lookup only checked the exact flat key. Fall back to dotted-path
resolution on extra_fields before model_dump().

Co-authored-by: Yassin Kortam <yassin@berri.ai>

* fix(sso): update CLI SSO test for new received_response kwarg and remove redundant 'token' secret fragment

Co-authored-by: Yassin Kortam <yassin@berri.ai>

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Yassin Kortam <yassin@berri.ai>
2026-05-22 09:58:50 -07:00
Sameer Kankute
988196911a
Litellm oss staging 1 (#28337)
* feat: add Xiaomi MiMo-V2.5-Pro and MiMo-V2.5 OpenRouter model entries (#27700)

Squash-merged by litellm-agent from TorvaldUtne's PR.

* fix(ui): trim whitespace from MCP inspector tool call inputs (#28203)

Co-authored-by: shin-berri <shin-laptop@berri.ai>
Co-authored-by: yuneng-jiang <yuneng@berri.ai>

* gemini-3.1-flash-lite pricing (#27933)

* feat(model_prices): add gemini-3.1-flash-lite pricing with standard/batch/flex/priority tiers

* fix pricing

* add service tier

---------

Co-authored-by: shin-berri <shin-laptop@berri.ai>

* fix: incorrect /v1/agents request example (#28131)

* fix(anthropic): accept dict-shape reasoning_effort from Responses bridge (#28201)

* fix(anthropic): accept dict-shape reasoning_effort from Responses bridge

Issue #28196 — the Responses->Chat parser (transformation.py:184-200) keeps the full dict as reasoning_effort when summary is set; that branch was added in #25359. But the Anthropic transformation here still guarded on isinstance(value, str), silently dropping the param. Result: callers using the standard Reasoning(effort, summary) OpenAI-shaped object on Anthropic lose thinking entirely (0 reasoning_tokens, no thinking_blocks).

Coerce dict -> string before mapping. Same shape tolerance that gpt_5_transformation._normalize_reasoning_effort_for_chat_completion already implements. summary is irrelevant for Anthropic's thinking_blocks.

Adds two regression tests: one parametrized over string + dict shapes (with and without summary), one covering unparseable dict inputs (drops silently, no crash).

* test(anthropic): add non-adaptive model coverage for dict-shape reasoning_effort

Per Greptile feedback on PR #28198: the original regression test only exercised the adaptive (4.6+) path. Add a parametrized test for the non-adaptive branch (claude-sonnet-4-5) verifying that dict-shape reasoning_effort still maps to thinking.type='enabled' + budget_tokens, and that output_config is NOT set on pre-4.6 models.

* test(anthropic): convert unparseable-dict test to @pytest.mark.parametrize

Per @greptile-apps inline review on PR #28201 — matches the parametrize style of the two adjacent dict-shape tests and produces clearer failure messages (test ID per case instead of one collapsing for-loop).

* feat: add pricing entry for openrouter/google/gemini-3.1-flash-lite (#28280)

Squash-merged by litellm-agent from ro31337's PR.

* fix(router): wrap aresponses streaming iterator for mid-stream fallbacks (#28215)

Squash-merged by litellm-agent from cwang-otto's PR.

* fix(router): unblock staging — mypy + coverage for aresponses streaming fallback (#28318)

Squash-merged by litellm-agent from cwang-otto's PR.

* fix(responses): forward timeout on completion transformation path (Anthropic, Bedrock, Vertex) (#28133)

Squash-merged by litellm-agent from cwang-otto's PR.

* feat(ui): add pause/resume Switch to the models table (#28151)

Squash-merged by litellm-agent from Cyberfilo's PR.

* fix(responses): merge sync completion kwargs to avoid duplicate keys

Double-splatting litellm_completion_request and kwargs raised TypeError
when metadata or service_tier were set. Match the async merge pattern.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Use proxy base URL for CLI SSO form action (#28271)

Co-authored-by: shin-berri <shin-laptop@berri.ai>
Co-authored-by: yuneng-jiang <yuneng@berri.ai>

* fix(tests): add mistral/ministral-8b-2512 to cost map and backfill in conftest

Mistral rotated the 'mistral/mistral-tiny' alias to return
'ministral-8b-2512' as the response model, which was missing from the
cost map. This caused test_completion_mistral_api and
test_completion_mistral_api_modified_input to fail in
litellm.completion_cost lookup.

- Add mistral/ministral-8b-2512 entry to both the in-tree
  model_prices_and_context_window.json and the bundled
  litellm/model_prices_and_context_window_backup.json (mirrors the
  existing openrouter/mistralai/ministral-8b-2512 pricing).

- litellm.model_cost is loaded at import time from the URL pinned to
  main, so the new backup entry isn't visible at test runtime until
  it also lands on main. Backfill any entries missing from the
  remote-fetched map into litellm.model_cost in the local_testing
  conftest so cost-calculator lookups succeed on this branch.

* fix(tests): drop unnecessary del of conftest backfill loop vars

* fix(router): harden streaming fallback wrapper for bridge iterators

- FallbackResponsesStreamWrapper now uses getattr fallbacks when copying
  attributes from the source iterator. The bridge path
  (LiteLLMCompletionStreamingIterator used by Anthropic/Bedrock/Vertex)
  does not call super().__init__ and is missing response, logging_obj
  (it uses litellm_logging_obj), responses_api_provider_config,
  start_time, request_data, call_type, and _hidden_params. Previously,
  wrapper construction raised AttributeError for any streaming fallback
  on the bridge path.
- _aresponses_with_streaming_fallbacks now deep-copies the
  litellm_metadata (and metadata) dicts into fallback_kwargs. The
  primary attempt mutates this dict in place via
  _update_kwargs_with_deployment, so a shallow copy of kwargs was
  leaking primary-deployment fields (deployment, model_info, api_base)
  into the mid-stream fallback request.

Co-authored-by: Yassin Kortam <yassin@berri.ai>

* fix(router): use safe_deep_copy for fallback metadata snapshot

The ban_copy_deepcopy_kwargs CI check rejects copy.deepcopy() on any
variable whose name contains 'kwargs' (incl. fallback_kwargs). Swap
the two copy.deepcopy(fallback_kwargs[...]) calls for safe_deep_copy,
which handles non-picklable values (OTEL spans, etc.) by per-key
deepcopy with fallback to the original reference.

Co-authored-by: Yassin Kortam <yassin@berri.ai>

* test(ci): skip chronically flaky build_and_test integration tests

Both tests have been failing on every recent run of build_and_test
against this PR's HEAD (1686967, 1688402, 1689993, 1690877), and the
same two tests also fail intermittently on unrelated commits and other
branches, independent of any code change in this PR (which only touches
router fallback wrappers, the Anthropic Responses bridge, and unrelated
UI/cost-map files).

- tests.test_spend_logs.test_spend_logs: /spend/logs?request_id=...
  returns 500 even after a 20s wait for the spend log to be written.
  Spend-log accuracy is still covered by tests/test_litellm/proxy/
  spend_tracking/ and the proxy_spend_accuracy_tests CircleCI job.

- tests.test_team_members.test_add_multiple_members: /team/info?team_id=
  ... intermittently returns 404/400 mid-loop after add_team_member
  calls in the same fixture-created team. Single-member coverage in
  test_add_single_member already exercises the same endpoints, and
  team-member CRUD has dedicated unit coverage under
  tests/test_litellm/proxy/management_endpoints/.

Skipping unblocks the build_and_test job until the underlying race in
the dockerized integration setup is root-caused.

* fix: preserve explicit timeout=0 in responses API handler

Use 'timeout if timeout is not None else request_timeout' instead of
'timeout or request_timeout' so an explicit timeout=0/0.0 isn't silently
replaced by the default request_timeout.

Co-authored-by: Yassin Kortam <yassin@berri.ai>

* fix(ui): guard model_info access in pause Switch with optional chaining

* fix(ui): guard model_info access in pause Switch onChange handler

Mirror the optional-chaining guard already applied to the isPausing
check so a config-model row with a missing model_info cannot throw
when the toggle's onChange fires.

---------

Co-authored-by: TorvaldUtne <78661304+TorvaldUtne@users.noreply.github.com>
Co-authored-by: oss-agent-shin <ext-agent-shin@berri.ai>
Co-authored-by: shin-berri <shin-laptop@berri.ai>
Co-authored-by: yuneng-jiang <yuneng@berri.ai>
Co-authored-by: mubashir1osmani <mubashir.osmani777@gmail.com>
Co-authored-by: Isha <72744901+IshaMeera@users.noreply.github.com>
Co-authored-by: cwang-otto <chengxuan.wang@ottotheagent.com>
Co-authored-by: Roman Pushkin <roman.pushkin@gmail.com>
Co-authored-by: Filippo Menghi <113345637+Cyberfilo@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: boarder7395 <37314943+boarder7395@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Co-authored-by: Claude <claude@anthropic.com>
Co-authored-by: Yassin Kortam <yassin@berri.ai>
2026-05-20 17:27:03 -07:00
Sameer Kankute
36c494fdd2
Litellm oss staging (#28161)
* fix(opentelemetry): JSON-serialize dict metadata fields for OTEL span attributes (#27451) (#27455)

Squash-merged by litellm-agent from Anai-Guo's PR.

* feat(dashscope): add embeddings and reranks(qwen3-rerank) support via OpenAI-compatible endpoint (#27508)

Squash-merged by litellm-agent from yimao's PR.

* fix(vertex_ai/gemini): raise BadRequestError when image_url or url fi… (#24550)

Squash-merged by litellm-agent from krisxia0506's PR.

* fix(vertex_ai): raise error on mid-stream 429/error chunks instead of silently swallowing (#23711)

Squash-merged by litellm-agent from krisxia0506's PR.

* fix: raise BadRequestError for file content blocks missing 'file' sub… (#24503)

Squash-merged by litellm-agent from krisxia0506's PR.

* Fix Gemini MIME detection for extensionless GCS URIs (#27278)

Squash-merged by litellm-agent from krisxia0506's PR.

* fix(vertex_ai/partner_models): drop unused vertexai SDK gate from count_tokens (closes #28084) (#28107)

Squash-merged by litellm-agent from voidborne-d's PR.

* feat(chart): add support for autoscaling behavior in HPA (#27990)

Squash-merged by litellm-agent from FabrizioCafolla's PR.

* feat(proxy): add blocked flag to models for pause/resume from the UI (#27927)

Squash-merged by litellm-agent from Cyberfilo's PR.

* fix: pass socket timeouts to Redis cluster clients (#27920)

Squash-merged by litellm-agent from tomdee's PR.

* Fix/cache token (#28009)

Squash-merged by litellm-agent from escon1004's PR.

* fix(deepseek): forward reasoning_content in multi-turn thinking mode conversations (#28080)

Squash-merged by litellm-agent from Divyansh8321's PR.

* fix(guardrails): return HTTP 400 instead of 500 for blocked requests (#27617)

* fix: reset org and tag budgets (#27326)

* reset org budgets

* reset tag budgets

---------

Co-authored-by: Michael Riad Zaky <michaelr@Mac.localdomain>

* fix(ui): omit allowed_routes from key edit save when unchanged (#27553)

* fix(ui): omit allowed_routes from key edit save when unchanged

When a team admin opens Edit Settings on a key with key_type=AI APIs and
saves without changing anything, the UI re-sends the existing allowed_routes
value, which the backend's _check_allowed_routes_caller_permission gate
rejects for non-proxy-admins (LIT-2681).

Strip allowed_routes from the patch in handleSubmit when it deep-equals the
original keyData.allowed_routes. The backend treats absence as "leave alone,"
so no-op saves now succeed for non-admins. Admins explicitly editing the
field still send the new value.

* fix(ui): order-insensitive allowed_routes diff + cover null-original case

Address Greptile review:

- Switch the "is allowed_routes unchanged" check to a Set-based comparison so
  a server-side reorder of the array doesn't register as a user edit and
  re-trigger LIT-2681.
- Add two regression tests: (1) keyData.allowed_routes is null and the form
  is untouched — patch should strip the field; (2) server returned routes in
  a different order than the user originally entered — patch should still
  recognize the value as unchanged.

* chore(ui): strip ticket refs and tighten comments in key edit fix

- Remove internal-tracker references from in-code comments
- Tighten the WHY comment in handleSubmit to two lines
- Drop redundant test-block comments — test names already describe the case

* fix(ui): annotate Set<string> generic in allowed_routes diff to fix tsc

* fix(guardrails): return HTTP 400 instead of 500 for guardrail-blocked requests

GuardrailRaisedException and BlockedPiiEntityError both lacked a
status_code attribute.  When these exceptions reached the proxy
exception handler (getattr(e, 'status_code', 500)), the fallback
defaulted to HTTP 500 — making intentional guardrail blocks
indistinguishable from server errors and causing unnecessary client
retries.

Changes:
- Add status_code=400 (keyword-only) to GuardrailRaisedException
- Add status_code=400 (keyword-only) to BlockedPiiEntityError
- Update _is_guardrail_intervention() to recognize both exceptions
  so downstream loggers record 'guardrail_intervened' instead of
  'guardrail_failed_to_respond'
- Add 6 unit tests for default/custom status codes and getattr pattern
- Strengthen existing blocked-action test with status_code assertion

Fixes #24348

---------

Co-authored-by: Michael-RZ-Berri <michael@berri.ai>
Co-authored-by: Michael Riad Zaky <michaelr@Mac.localdomain>
Co-authored-by: ryan-crabbe-berri <ryan@berri.ai>
Co-authored-by: Krrish Dholakia <krrish+github@berri.ai>

* fix(router/proxy): address Greptile P1+P2 review comments on PR #28161

- router: raise ServiceUnavailableError (503) instead of RouterRateLimitErrorBasic (429)
  when a specifically-addressed deployment is administratively blocked; 429 misleads
  retry-enabled clients into spinning forever against a paused model
- proxy_server: compute get_fully_blocked_model_names() once before both branches in
  model_list() instead of duplicating the call in each branch
- deepseek: upgrade silent debug log to warning when injecting placeholder
  reasoning_content so callers are clearly notified of degraded multi-turn quality
- tests: update two blocked-deployment assertions to expect ServiceUnavailableError

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix: address bug detection findings (cache token order, mutable defaults)

Co-authored-by: Yassin Kortam <yassin@berri.ai>

* fix: address bugs in async pass-through, anthropic cache token detection, rerank tests

- async_get_available_deployment_for_pass_through: enforce blocked check on specific deployments
- cost_calculator: detect anthropic-style usage by attribute presence (not truthiness) to avoid mixing OpenAI cached_tokens into anthropic normalization when read=0
- dashscope rerank tests: pass request to httpx.Response constructions for consistency

Co-authored-by: Yassin Kortam <yassin@berri.ai>

* fix code qa

* fix(vertex_ai/gemini): strip MIME parameters from GCS contentType

GCS object metadata's contentType field can include parameters such as
'text/html; charset=utf-8'. Strip them in _apply_gemini_mime_type_aliases
so downstream get_file_extension_from_mime_type sees a bare MIME type.

Co-authored-by: Yassin Kortam <yassin@berri.ai>

* fix(vertex_ai/gemini): clarify mime-type error message string concatenation

Co-authored-by: Yassin Kortam <yassin@berri.ai>

---------

Co-authored-by: Tai An <antai12232931@outlook.com>
Co-authored-by: Vincent <yimao1231@gmail.com>
Co-authored-by: Kris Xia <xiajiayi0506@gmail.com>
Co-authored-by: d 🔹 <liusway405@gmail.com>
Co-authored-by: Fabrizio Cafolla <developer@fabriziocafolla.com>
Co-authored-by: Filippo Menghi <113345637+Cyberfilo@users.noreply.github.com>
Co-authored-by: Tom Denham <tom@tomdee.co.uk>
Co-authored-by: escon1004 <70471150+escon1004@users.noreply.github.com>
Co-authored-by: Divyansh Singhal <97736786+Divyansh8321@users.noreply.github.com>
Co-authored-by: robin-fiddler <robin@fiddler.ai>
Co-authored-by: Michael-RZ-Berri <michael@berri.ai>
Co-authored-by: Michael Riad Zaky <michaelr@Mac.localdomain>
Co-authored-by: ryan-crabbe-berri <ryan@berri.ai>
Co-authored-by: Krrish Dholakia <krrish+github@berri.ai>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Yassin Kortam <yassin@berri.ai>
2026-05-18 16:27:44 -07:00
ryan-crabbe-berri
fe63650ebd
fix(proxy): gate team allowed_passthrough_routes to proxy admins (#28097)
* fix(proxy): gate team allowed_passthrough_routes to proxy admins

allowed_passthrough_routes short-circuits the role-based route gate, so
the keys endpoints already restrict it to proxy admins. The team writers
(/team/new, /team/update) had no equivalent check, letting an org admin
(a non-proxy-admin who clears the route gate and _verify_team_access)
self-grant pass-through routes on their team. Lift the keys check into a
shared helper and apply it to both team endpoints.

Resolves LIT-3019

* docs(proxy): note view-only admins are intentionally excluded from passthrough gate

Clarifies the proxy-admin guard per review feedback; no behavior change.

Refs LIT-3019
2026-05-18 08:56:25 -07:00
Sameer Kankute
5aabfccf57
fix(mcp): allow delegate PKCE bypass for internal MCP servers
Remove available_on_public_internet gating from delegate-auth-to-upstream
paths so oauth2 + delegate_auth_to_upstream interactive servers behave
the same when marked internal. Keeps M2M exclusion. Updates tests.
2026-05-15 08:32:09 +05:30
Krrish Dholakia
410ce761dc
fix: tighten budget field validation and authorization checks (#27897)
* fix: block NaN/Inf budget bypass and add missing non-admin guards

Addresses three security issues:

GHSA-wvg4-6222-3q4r: /user/update exposes max_budget, soft_budget, spend
to self-editing non-admin users with no server-side guard. Non-admin callers
now receive HTTP 403 if any of those fields appear in the update payload.

GHSA-q775-qw9r-2r4g: _enforce_upperbound_key_params returned early (no-op)
when upperbound_key_generate_params was absent from config, letting any
authenticated user generate a key with unlimited max_budget. Fix adds a
delegated-authority ceiling in _common_key_generation_helper: non-admins
cannot grant a key more budget than their own key carries.

GHSA-2rv4-xv66-fpjg: float('nan') passes every `value < 0` guard because
nan < 0 is False in Python, and spend >= nan is always False, permanently
disabling budget enforcement for any entity carrying a NaN max_budget.
All write-time budget guards now use `not math.isfinite(v) or v < 0`.
_enforce_upperbound_key_params validates finiteness unconditionally (before
the early-return). All spend-enforcement comparisons in auth_checks.py are
now guarded with math.isfinite(max_budget) as defense-in-depth.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

* fix: close budget ceiling bypass for callers with no max_budget (GHSA-q775)

Non-admin callers whose API key has no explicit max_budget (None) could
bypass the delegated-authority ceiling and create keys with arbitrary
budgets. Now blocks budget assignment when caller has no budget configured.
Also removes redundant inline import of LitellmUserRoles.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: only apply budget ceiling to explicitly requested max_budget

Capture the caller-supplied max_budget before _enforce_upperbound_key_params
can fill it with a default, so auto-filled defaults don't trigger the
ceiling guard for non-admin users with no budget on their own key.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: capture requested max_budget before any defaults are applied

Move _requested_max_budget capture before both default_key_generate_params
and upperbound_key_generate_params mutations, so auto-filled values don't
trigger the ceiling check for non-admin users.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: allow unlimited-budget callers to delegate any budget

Callers with max_budget=None (unlimited) can legitimately create
budget-capped keys. Only block when caller has an explicit budget
and the requested budget exceeds it.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: yuneng-jiang <yuneng@berri.ai>
Co-authored-by: ryan-crabbe-berri <ryan@berri.ai>
Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-05-14 05:41:13 +00:00
Krrish Dholakia
8bbc61e03c
fix: harden /key/update authorization checks (#27878)
* fix: patch Host-header auth bypass in get_request_route

Starlette reconstructs request.url from the Host header. A malformed
Host like `localhost/?x=1` causes Starlette to build the full URL as
`http://localhost/?x=1/health`, which url-parses to path="/". Since "/"
is in LiteLLMRoutes.public_routes, all protected routes became reachable
without authentication.

Fix: read scope["path"] (set by uvicorn from the HTTP request line,
not derivable from headers) instead of request.url.path. Sub-path
deployments are handled via scope["app_root_path"] / scope["root_path"],
mirroring Starlette's own base_url construction logic.

Affected variants confirmed fixed:
  Host: localhost/?x=1
  Host: localhost:4000/?x=1
  Host: localhost/#test
  Host: localhost:4000/#test

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

* style: reduce comments in route fix

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

* fix: block credential fields in RAG ingest vector_store options

Credential fields (vertex_credentials, aws_access_key_id, api_key, etc.)
in ingest_options.vector_store are now rejected at the API boundary with
a 400 error. Credentials must be configured server-side.

Previously any authenticated user could supply a vertex_credentials dict
with type=external_account pointing credential_source.file at an
arbitrary path (e.g. /proc/1/environ) and token_url at an
attacker-controlled server. google-auth's identity_pool.Credentials
refresh() would read the file and POST its contents to the attacker.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

* fix: block /key/update self-escalation by assigned users

Non-admin users who were assigned a key (created_by != caller) could
update any non-budget field — models, rpm_limit, guardrails, etc. —
without admin authorization, allowing privilege self-escalation.

Gate: only the key creator (created_by == caller) may edit their own
key without admin check; budget changes always require admin regardless
of creator status. All other callers must pass _check_key_admin_access.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

* fix: block user-controlled api_base in RAG ingest vector_store options

A user-supplied api_base in ingest_options.vector_store caused the server
to forward its configured provider credentials (Gemini, OpenAI) to an
attacker-controlled endpoint via SSRF.

Add api_base to the blocked credential params set alongside api_key and
the existing credential fields.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

* fix: restrict /utils/transform_request to PROXY_ADMIN and apply body safety check

Any authenticated internal_user could POST arbitrary provider config
(aws_sts_endpoint, api_base, etc.) to /utils/transform_request and have
the server forward its credentials to an attacker-controlled endpoint.

- Gate the endpoint on PROXY_ADMIN role (403 for all other roles)
- Call is_request_body_safe() to reject banned params even for admins
- Convert ValueError from safety check to HTTP 400

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

* fix: apply banned-param check to /utils/transform_request

Without is_request_body_safe(), any authenticated user could pass
aws_sts_endpoint, api_base, or aws_web_identity_token to
/utils/transform_request and have the server forward its configured
provider credentials to an attacker-controlled endpoint during SDK
credential resolution.

Applies the same banned-param blocklist already used by LLM endpoints.
Endpoint remains accessible to all authenticated users.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

* fix: block SSRF via api_base in /prompts/test dotprompt YAML frontmatter

Any frontmatter key not in ["model","input","output"] flowed into
optional_params and was merged into the LLM call data dict, bypassing
is_request_body_safe. An attacker with any bearer key could set
api_base in YAML to redirect the outbound LLM request — including the
provider API key — to an attacker-controlled host.

Fix: call is_request_body_safe on the constructed data dict after
optional_params are merged, before invoking ProxyBaseLLMRequestProcessing.
ValueError from the banned-param check is surfaced as HTTP 400.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

* Update litellm/proxy/rag_endpoints/endpoints.py

Co-authored-by: veria-ai[bot] <224490171+veria-ai[bot]@users.noreply.github.com>

* fix: coerce nested config strings before banned-param check

_NESTED_CONFIG_KEYS descent used isinstance(nested, dict) which silently
skipped litellm_embedding_config when delivered as a JSON string via
multipart/form-data. Banned params (api_base, aws_sts_endpoint, etc.)
nested inside the stringified value were invisible to is_request_body_safe.

_NESTED_METADATA_KEYS already used _coerce_metadata_to_dict which parses
JSON strings before checking. Apply the same coercion to _NESTED_CONFIG_KEYS.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

* fix: replace substring match with prefix match in is_llm_api_route

mapped_pass_through_routes used `_llm_passthrough_route in route` (substring)
so any admin-only path whose URL contained a provider name (openai, anthropic,
azure, bedrock, etc.) was misclassified as an LLM API route and bypassed the
admin gate in non_proxy_admin_allowed_routes_check.

Confirmed live: non-admin key could GET /credentials/by_name/openai (read
masked provider API key) and DELETE /credentials/openai (delete credential).

Fix: use exact match or startswith(prefix + "/") — the same pattern used
everywhere else in RouteChecks — so only routes that actually start with a
passthrough prefix are allowed through.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

* fix: stabilize PR #27878 test failures

- key_management_endpoints: extend can_skip_admin_check to team keys so
  team members with /key/update permission can update non-budget fields.
  can_team_member_execute_key_management_endpoint already validates team
  membership + permission and raises if unauthorized; reaching the admin
  check on a team key means the caller was authorized.

- test: set created_by on mock key in
  test_update_key_non_budget_fields_allowed_for_internal_user so
  caller_is_creator resolves correctly (MagicMock default ≠ user_id).

- auth_utils.get_request_route: guard against non-dict request.scope
  (e.g. MagicMock in unit tests) to prevent a MagicMock leaking into
  UserAPIKeyAuth.request_route and failing Pydantic validation.

- ci: assign test_multipart_bypass_repro.py to the proxy-runtime shard
  in test-unit-proxy-db.yml to satisfy the shard-coverage check.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

* fix(lint): add explicit str() cast in get_request_route for MyPy

scope.get() returns Any|None which MyPy cannot coerce to str implicitly.
Wrap both scope.get() calls in str() to satisfy the type checker.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

* fix: guard bare-/ root_path strip + make total_spend migration idempotent

auth_utils.get_request_route: when Starlette sets scope["app_root_path"]
to "/" (e.g. behind some middleware), the old stripping logic would
remove the leading slash from every path ("/team/new" → "team/new"),
breaking route matching and causing auth to misclassify protected routes.
Skip stripping when root_path is bare "/".

migration: add IF NOT EXISTS to total_spend ALTER TABLE so the migration
is safe to replay when a prior partial run already created the column.
Without this guard, prisma migrate deploy fails on CI DBs that were
partially migrated, causing all subsequent DB operations (including
/team/new) to 500.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

* fix: require creator still owns key for personal-key bypass in /key/update

caller_is_creator now requires both created_by == caller AND user_id ==
caller. Previously checking only created_by let a demoted admin who
originally created a key for another user continue editing non-budget
fields on it after reassignment, bypassing _check_key_admin_access.

Adds regression test: creator whose key was reassigned is blocked (403).

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

* fix: extract auth checks to fix PLR0915 + broaden max_budget assertion

internal_user_endpoints._update_single_user_helper exceeded 50 statements
(PLR0915). Extract authorization checks into _check_user_update_authz helper
to bring statement count under the limit.

test_validate_max_budget: assert "negative" (substring of both the local
"cannot be negative" and the CI "non-negative finite number" messages) so
the test is stable regardless of which exact wording the function uses.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: veria-ai[bot] <224490171+veria-ai[bot]@users.noreply.github.com>
2026-05-14 04:16:04 +00:00
Sameer Kankute
18f77ff7bc
feat(mcp): add delegate_auth_to_upstream flag for PKCE passthrough (#27834)
* feat(mcp): add delegate_auth_to_upstream flag for PKCE passthrough

Adds an opt-in per-server flag that lets clients (e.g. VS Code) complete
PKCE directly with an upstream OAuth2 MCP server, instead of LiteLLM
double-gating with its own API-key/SSO check. Only honored when
auth_type=oauth2 and the operator explicitly sets the flag; mixed-target
or non-oauth2 requests fail closed.

- Adds the field to Pydantic models, Prisma schema, and a migration
- New MCPRequestHandler._target_servers_delegate_auth_to_upstream gate
  that runs only when no x-litellm-api-key is present, so authenticated
  users still get user_id resolution + stored-credential lookup
- Anonymous callers now see delegate servers in get_allowed_mcp_servers
  (scoped to delegate servers only; the upstream still enforces auth)
- mcp_management_endpoints: allow anonymous /authorize and /token for
  delegate servers so VS Code can complete PKCE without a LiteLLM session
- UI toggle (shown only for oauth2) + payload/view wiring
- Tests covering: oauth2 on/off, non-oauth2 with flag, mixed targets,
  no resolvable target, explicit key precedence, and 401 emission

Co-authored-by: Cursor <cursoragent@cursor.com>

* Enforce oauth2 for delegated MCP auth bypass

Co-authored-by: Yassin Kortam <yassin@berri.ai>

* fix(mcp): close secondary Authorization bypass for delegate servers

The delegate-auth bypass gated only on the primary `x-litellm-api-key`
header, so a LiteLLM key sent via `Authorization: Bearer sk-...` (the
secondary header) was silently dropped — skipping spend tracking and
rate limiting. Gate on the resolved litellm_api_key (which considers
both headers) so the bypass fires only when neither is present.

Also update the existing "Authorization header present" test to reflect
that an upstream OAuth token now flows through the existing oauth2
fallback (LiteLLM auth attempt → fail → anonymous), not via the
delegate branch.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Avoid duplicate MCP OAuth credential lookup

Co-authored-by: Yassin Kortam <yassin@berri.ai>

* fix(mcp): block delegate bypass for M2M and internal-only servers

Two security issues flagged in code review:

1. High – client_credentials (M2M) servers must not be delegatable:
   LiteLLM auto-fetches the upstream token using stored credentials, so
   allowing anonymous bypass would let any external caller invoke tools
   authenticated as LiteLLM's service account.
   Fix: check `server.has_client_credentials` in
   `_target_servers_delegate_auth_to_upstream`, the anonymous
   allow-list in `get_allowed_mcp_servers`, and `_mcp_oauth_user_api_key_auth`.

2. Medium – internal-only servers exposed to public internet:
   The anonymous delegate allow-list was not filtering by
   `available_on_public_internet`, so external callers with an upstream
   OAuth token could invoke tools on servers marked internal-only.
   Fix: add `available_on_public_internet` guard to the anonymous
   delegate server list in `get_allowed_mcp_servers`.

Tests added for both cases.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Require public MCP delegate auth servers

Co-authored-by: Yassin Kortam <yassin@berri.ai>

* fix(mcp): align delegate auth path parsing with downstream routing

`_extract_target_server_names_from_path` used a naive segments-based
split while `server.py::_get_mcp_servers_in_path` uses a regex that
allows server names with one embedded slash and comma-separated lists.
With the old parser, a request to `/mcp/<delegated>/<garbage>` was
parsed as targeting `<delegated>` by the auth gate (bypassing LiteLLM
auth) while the routing layer parsed it as `<delegated>/<garbage>` —
when that name did not resolve, the request fell back to the anonymous
allow-list, which can include `allow_all_keys` servers that normally
require a LiteLLM key.

Replace the parser with the same regex logic as
`_get_mcp_servers_in_path` so auth gating sees the exact target name(s)
downstream routing sees. Add regression tests covering parser parity
and the specific extra-path-segment bypass attempt.

https://claude.ai/code/session_01SjyPmwfmrq8fveFgw9iHW9

* fix(mcp): close header/path TOCTOU in MCP delegate auth gate

`_target_servers_delegate_auth_to_upstream` and
`_target_servers_use_oauth2` trusted the `x-mcp-servers` header when
present, but `server.py::extract_mcp_auth_context` overrides that
header with the path-derived list for `/mcp/...` routes. An attacker
could set `x-mcp-servers: <delegated>` while pointing the URL path at
a non-delegate server, flipping the auth gate without changing the
target downstream routing actually uses.

Extract a shared `_resolve_target_server_names` helper that mirrors
the downstream override (path-derived names for `/mcp/...` routes,
header value otherwise). Add regression tests covering the TOCTOU
attempt and the helper's path-vs-header precedence.

https://claude.ai/code/session_01SjyPmwfmrq8fveFgw9iHW9

* Fix delegated MCP OAuth test mock

Co-authored-by: Yassin Kortam <yassin@berri.ai>

* fix(mcp): drop unreachable /{server}/mcp branch in auth path parser

`_extract_target_server_names_from_path` also matched the
``/{server_name}/mcp`` form, but the downstream parser
``_get_mcp_servers_in_path`` only handles ``/mcp/...`` — and
``dynamic_mcp_route`` in ``proxy_server`` rewrites ``/{name}/mcp``
to ``/mcp/{name}`` on the scope before the MCP handler runs. Parsing
the un-rewritten form on the auth side was therefore unreachable in
production, and contradicted the docstring's claim of mirroring the
downstream parser — exactly the kind of mismatch that risks a future
header/path TOCTOU if any new entry point skips the rewrite.

Drop the branch; the canonical ``/mcp/...`` path matches both
parsers. Update the regression test to assert the new behavior.

https://claude.ai/code/session_01SjyPmwfmrq8fveFgw9iHW9

* Fix MCP path auth target resolution

Co-authored-by: Yassin Kortam <yassin@berri.ai>

* fix(mcp): require auth for refresh_token grants on delegate-auth servers

`_mcp_oauth_user_api_key_auth` gates the unauthenticated PKCE flow for
``delegate_auth_to_upstream`` servers, but the bypass applied to BOTH
``/authorize`` and ``/token`` regardless of grant type. ``mcp_token``
accepts ``grant_type=refresh_token`` as well as ``authorization_code``,
and ``exchange_token_with_server`` attaches the server's stored
``client_secret`` to whatever is forwarded upstream. An unauthenticated
caller holding a refresh token issued to that OAuth client could mint
fresh upstream access tokens through LiteLLM.

Limit the anonymous bypass on ``/token`` to ``grant_type=authorization_code``
(the only grant PKCE actually protects via ``code_verifier``); fall
through to normal LiteLLM auth for ``refresh_token`` and any other grant.
``/authorize`` continues to allow anonymous PKCE redirects.

https://claude.ai/code/session_01SjyPmwfmrq8fveFgw9iHW9

* fix(ui): clear delegate_auth_to_upstream when switching off oauth2

The ``delegate_auth_to_upstream`` form field is rendered inside an
``isOAuth2 && (...)`` conditional, so the Form.Item unmounts when the
user changes ``auth_type`` away from ``oauth2``. The follow-up
``form.setFieldValue("delegate_auth_to_upstream", false)`` runs after
the field has already deregistered, so ``onFinish`` receives
``undefined`` and the fallback ``?? mcpServer.delegate_auth_to_upstream``
preserved the old ``true``. The flag then persisted in the database for
a non-oauth2 server and silently re-activated if ``auth_type`` was later
switched back to ``oauth2``.

In the edit payload, force the flag to ``false`` whenever
``auth_type !== oauth2``; only trust the form value (and the existing
DB fallback) when the server is actually oauth2. Backend defense-in-depth
already ignores the flag for non-oauth2 servers, but the DB state should
stay clean too.

https://claude.ai/code/session_01SjyPmwfmrq8fveFgw9iHW9

* Fix MCP delegate auth reset on edit

Co-authored-by: Yassin Kortam <yassin@berri.ai>

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Yassin Kortam <yassin@berri.ai>
Co-authored-by: Claude <claude@anthropic.com>
2026-05-13 12:06:13 -07:00
user
8f3056c9c9
chore(proxy): block explicit-null user_id in ownership rebind guard
``model_dump(exclude_unset=True)`` in ``prepare_key_update_data``
includes any field the caller explicitly set, even when the value is
``None``. The previous guard short-circuited on ``getattr(data,
'user_id', None) is None``, which conflated "field omitted" (safe)
with "field explicitly set to null" (writes NULL to the token row,
detaching the key from its user and bypassing user-row role
checks).

Switch the omitted-vs-set distinction to ``data.model_fields_set``;
treat explicit-null and explicit-empty-string identically as a
removal attempt, both 403-rejected for non-admin callers.

Parametrized regression adds ``explicit_null_blocked`` alongside the
existing ``rebind_blocked`` / ``empty_blocked`` / ``same_user_id_allowed``
cases.
2026-05-13 00:46:28 +00:00
user
d5d73f1d6a
chore(proxy): clarify ownership-rebind error message (actor vs target)
Previous wording read "User=<new_owner> is not allowed to update the
key to belong to user=<current_owner>" — easy to misread as "caller
wants to keep the key on its current owner". Reframe as
"Non-admin caller is not allowed to rebind the key from
user=<existing> to user=<incoming>" so the direction of the failed
operation is unambiguous.

Same shape preserved (HTTPException 403); only the ``detail`` string
changes. Regression test substring updated.
2026-05-13 00:41:08 +00:00
user
7738b03501
chore(proxy): close /key/regenerate ownership-rebind + premium-gate bypass
A non-admin caller could rebind their own key's ``user_id`` via
``/key/regenerate``. ``_execute_virtual_key_regeneration`` had org/team
guards but no ``user_id`` guard, and ``prepare_key_update_data`` did not
strip the field — it survived ``model_dump(exclude_unset=True)`` into
the Prisma update. On the next request,
``_return_user_api_key_auth_obj`` resolved the rebound ``user_id``
against ``litellm_usertable`` and returned ``PROXY_ADMIN`` whenever
the target row's ``user_role`` was admin (e.g. the default
``user_id="default_user_id"`` created on first password-UI login).

``/key/update`` had the equivalent guard inline at
``_validate_update_key_data``; extract it to a shared helper
``_validate_caller_can_change_key_ownership`` and call from both
``/key/update`` and ``_execute_virtual_key_regeneration``. Future
regenerate-style endpoints inherit the guard for free.

Also tighten the premium gate that allowed the master-key rotation
branch to skip the enterprise check. The previous predicate was
``data.new_master_key is not None`` — a field-presence test, not an
identity check. Any non-premium caller could send any value in that
field and the premium check would no-op. Verify the caller actually
holds the master key via ``_is_master_key`` before allowing the
non-premium path.

Tests:
- ``test_regenerate_user_id_rebind_guard`` — parametrized table over
  cross-user rebind (blocked), empty-string removal (blocked), and
  same-user no-op rebind (allowed).
- ``test_regenerate_premium_gate_requires_actual_master_key`` /
  ``test_regenerate_premium_gate_allows_actual_master_key_holder`` —
  ensure the premium check requires the caller actually present the
  master key, and that legitimate master-key rotation still works.
2026-05-13 00:27:34 +00:00
ryan-crabbe-berri
63a2d1ddc9
fix(tests): use canonical litellm_enterprise import path (#27699)
The enterprise package is installed as `litellm_enterprise` (per
enterprise/pyproject.toml), but several tests imported it as
`enterprise.litellm_enterprise.*` — a path that only resolves
because the repo root happens to sit on sys.path, letting Python's
implicit namespace package machinery discover `enterprise/` as a
directory.

This breaks any test runner that relocates source (e.g. the
mutation-testing workflow, which copies tests under `mutants/`) and
also caused two `patch()` strings to target a module path that does
not match what production code imports — meaning those mocks were
never actually patching the production module's attribute.

Replace `from enterprise.litellm_enterprise.` with the canonical
`from litellm_enterprise.` across 6 test files, and fix two
`patch()` target strings (and one `sys.modules` patch key in the
SSO test) to match.
2026-05-12 12:32:57 -07:00
ryan-crabbe-berri
9c4faeabc9
feat(ui): search teams by team ID alongside name (#27684)
* feat(ui): search teams by team ID alongside name

The Teams page search box only matched team_alias, so pasting a team UUID
returned zero results. Detect when the input is a full UUID and route it
to the team_id filter instead; otherwise keep the existing alias substring
search. Placeholder now reads "Search teams by name or ID...".

Resolves LIT-2648

* refactor: search teams via backend OR clause, drop client-side UUID detection

Adds a `search` query param to /v2/team/list that ORs across team_id
(exact) and team_alias (case-insensitive contains), so the search box
sends one param regardless of input format. Removes the isLikelyTeamId
helper and the client-side branching it fed.
2026-05-12 11:14:29 -07:00
Sameer Kankute
aa9e7b9808
feat: litellm shin agent oss staging 05 10 2026 (#27631)
* fix: invalidate cached tag object on tag budget reset (#27481) (#27572)

Squash-merged by litellm-agent from oss-agent-shin's PR.

* chore(mcp): tighten stdio server registration paths (#27570)

Squash-merged by litellm-agent from stuxf's PR.

* fix(proxy): clear MCP OpenAPI mappings on server eviction; widen budget cache invalidation

Evict OpenAPI tools from global_mcp_tool_registry and strip tool_name_to_mcp_server_name_mapping entries when a server leaves the runtime registry (remove_server and approval-status eviction). Invalidate user_api_key_cache for keys, orgs, and team members on budget-tier spend resets alongside tags.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(mcp): align update_server eviction with remove_server name fallback

Document budget-reset test assertion flip (cross-pod cache staleness).

Greptile: eviction now pops by server_id then server_name like remove_server;
test docstring explains assert_not_awaited -> assert_any_await change.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Fix org budget cache invalidation

---------

Co-authored-by: oss-agent-shin <ext-agent-shin@berri.ai>
Co-authored-by: stuxf <70670632+stuxf@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-11 20:31:43 -07:00
ishaan-berri
12e59c8798
Fix internal tag usage scoping (#27315)
* Scope internal tag usage to own keys

Co-authored-by: ishaan-berri <ishaan-berri@users.noreply.github.com>

* Add internal tag usage unowned key regression test

Co-authored-by: ishaan-berri <ishaan-berri@users.noreply.github.com>

* Handle empty internal tag usage scopes safely

Co-authored-by: ishaan-berri <ishaan-berri@users.noreply.github.com>

* Add tag activity database guard

Co-authored-by: ishaan-berri <ishaan-berri@users.noreply.github.com>

---------

Co-authored-by: oss-agent-shin <279349115+oss-agent-shin@users.noreply.github.com>
Co-authored-by: ishaan-berri <ishaan-berri@users.noreply.github.com>
2026-05-11 10:44:50 -07:00
Sameer Kankute
aa587bd9d3
Merge pull request #27549 from BerriAI/shin_agent_oss_staging_05_09_2026
[litellm-agent] Staging → litellm_internal_staging (5/9/2026)
2026-05-11 11:58:26 +05:30
Sameer Kankute
9ed99037d2
Merge pull request #27422 from BerriAI/shin_agent_oss_staging_05_07_2026
[litellm-agent] Staging → litellm_internal_staging (5/7/2026)
2026-05-11 11:58:05 +05:30
Michael-RZ-Berri
b834817785
[Feat] Add endpoint for bulk key updates for team (#26468)
Squash-merged by litellm-agent from Michael-RZ-Berri's PR.
2026-05-09 19:32:16 +00:00
ryan-crabbe-berri
13a193367f
feat(sso): show full IdP claims in /sso/debug/callback (#27498)
* feat(sso): show full IdP claims in /sso/debug/callback

The debug callback only displayed the proxy-parsed OpenID summary, so
customers couldn't verify what custom claims (team_id, team_alias, roles,
etc.) the IdP was actually returning. Render two new sections — Raw
Claims (userinfo) and Access Token Claims (decoded JWT) — alongside the
existing parsed view. Strip bearer tokens defense-in-depth in case a
non-conforming IdP places them in its userinfo response.

Resolves LIT-2838

* Update litellm/proxy/management_endpoints/ui_sso.py

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* fix(sso): hoist json.dumps out of f-string for py3.10 ruff

---------

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-05-08 17:39:39 -07:00
oss-pr-review-agent-shin[bot]
158b0c28c0
[litellm-agent] Staging → litellm_internal_staging (5/7/2026) (#27375)
Squash-merged by litellm-agent from oss-pr-review-agent-shin[bot]'s PR.
2026-05-07 21:29:47 +00:00
oss-agent-shin
d90cf56245
Fix SCIM user lookup filters (#27308)
* Fix SCIM Okta userName lookup

Co-authored-by: ishaan-berri <ishaan-berri@users.noreply.github.com>

* fix scim user filter typing

Co-authored-by: ishaan-berri <ishaan-berri@users.noreply.github.com>

---------

Co-authored-by: oss-agent-shin <279349115+oss-agent-shin@users.noreply.github.com>
Co-authored-by: ishaan-berri <ishaan-berri@users.noreply.github.com>
2026-05-06 11:58:47 -07:00
Yassin Kortam
bd1ea0252a
perf(proxy): run daily activity aggregation off the event loop (#27264)
Some checks are pending
Unit Tests: Proxy DB Operations / custom-logging (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / logging-misc (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / assert-shard-coverage (push) Waiting to run
Unit Tests: Proxy DB Operations / auth-checks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / budgets (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / db-and-spend (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / endpoints-and-responses (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / guardrails-hooks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / jwt-and-keys (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / key-generation (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-runtime (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-server-core (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / schema-migration (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-utils (push) Blocked by required conditions
Unit Tests: Security / security (push) Waiting to run
Unit Tests: Caching (Redis) / caching-redis (push) Waiting to run
Co-authored-by: Yassin Kortam <yassinkortam@g.ucla.edu>
2026-05-05 20:19:28 -07:00
Dennis Henry
73de892654
fix: replace user api key auth with authorization or cookie for mcp server creation (#27190)
* fix: replace user api key auth with authorization or cookie for mcp server creation

* updated tests
2026-05-05 18:36:22 -07:00
yuneng-jiang
be5f217aaf
Merge pull request #26861 from BerriAI/litellm_fix_scim_virtual_key_deactivation
fix(scim): revoke virtual keys when SCIM deprovisions a user
2026-05-04 19:03:55 -07:00
Yassin Kortam
a56256e5ee feat: routing groups ui 2026-05-04 18:09:14 -07:00
yuneng-jiang
2aa4301fe7
Merge pull request #27027 from stuxf/fix/mcp-server-url-redact-non-admin
fix(proxy): redact MCP server URL and headers for non-admin viewers (VERIA-8)
2026-05-04 15:05:05 -07:00
yuneng-jiang
e1083b1353
Merge pull request #27014 from stuxf/fix/activity-endpoint-tenant-scoping
fix(proxy): scope team and agent activity endpoints per-entity (VERIA-43)
2026-05-04 14:51:45 -07:00
yuneng-jiang
e995156462
Merge pull request #26953 from stuxf/chore/audit-log-cache-vault-settings
chore(audit): audit-log /cache/settings + /config_overrides/hashicorp_vault mutations
2026-05-04 14:42:23 -07:00
yuneng-jiang
c064170a18
Merge pull request #26819 from stuxf/fix/team-callback-idor
chore(team): require team-management role on /team/{id}/callback endpoints
2026-05-04 14:31:59 -07:00
user
31d0d8599f fix(mcp): redact command fields for non-admin server views 2026-05-01 21:47:18 -07:00
Cursor Agent
8cf3bf11bd
fix(scim): preserve scim_active on PUT when client omits the field
Some checks failed
Unit Tests: Caching (Redis) / caching-redis (push) Has been cancelled
Unit Tests: Proxy DB Operations / assert-shard-coverage (push) Has been cancelled
Unit Tests: Security / security (push) Has been cancelled
Unit Tests: Proxy DB Operations / auth-checks (push) Has been cancelled
Unit Tests: Proxy DB Operations / budgets (push) Has been cancelled
Unit Tests: Proxy DB Operations / custom-logging (push) Has been cancelled
Unit Tests: Proxy DB Operations / db-and-spend (push) Has been cancelled
Unit Tests: Proxy DB Operations / endpoints-and-responses (push) Has been cancelled
Unit Tests: Proxy DB Operations / guardrails-hooks (push) Has been cancelled
Unit Tests: Proxy DB Operations / jwt-and-keys (push) Has been cancelled
Unit Tests: Proxy DB Operations / key-generation (push) Has been cancelled
Unit Tests: Proxy DB Operations / logging-misc (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-runtime (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-server-core (push) Has been cancelled
Unit Tests: Proxy DB Operations / schema-migration (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-utils (push) Has been cancelled
A SCIM PUT may legally omit `active` (full-replace with the field
absent). Pydantic fills the SCIMUser.active default of True, so the PUT
handler was overwriting metadata.scim_active with True even when the
client never sent it — silently reactivating a previously SCIM-blocked
user and unblocking their keys.

Use model_fields_set to detect whether the client actually sent
`active`. If omitted, preserve the prior scim_active value and skip
the cascade to virtual keys.

Also drop comments added in this PR that just narrate what the code
does; keep only the docstrings and the SQL-NULL pitfall note that
explain non-obvious behaviour.

Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
2026-05-02 04:15:25 +00:00