mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-06 02:48:13 +00:00
588 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
7a72b1062d
|
Merge pull request #29528 from aanchal22/litellm_byok-alias-merge
fix(proxy): atomic merge for team model aliases and team.models on BYOK create
(cherry picked from commit
|
||
|
|
5204ab5639
|
fix(mcp): allow team access-group grants in OAuth authorize/token access check (#30041)
* fix(mcp): honor team access-group grants in OAuth authorize/token access check
* test(mcp): mock build_effective_auth_contexts in non-admin authorize tests for isolation
(cherry picked from commit
|
||
|
|
4ec4ab99d0
|
feat(mcp): per-server env vars with global + per-user scopes (#28917) | ||
|
|
53cf3d8416
|
fix(proxy): drop deleted team BYOK model name from team.models (#29820)
Deleting a team-scoped BYOK model left its public name in team.models, so /models with a team key kept listing the now-deleted "ghost" model. delete_model stripped team.models using only litellm_modeltable alias lookups, but models added via /model/new with a team_id never create an alias row; their public name lives only in team.models and model_info.team_public_model_name, so it was never removed. The team cache was also left stale because the delete path skipped _refresh_cached_team. The cleanup now keys off team_public_model_name (falling back to alias keys), runs after the deployment row is deleted, and strips a public name only when no remaining team deployment still backs it, so a load-balanced replica is not revoked and concurrent deletes cannot leave a ghost. The updated team row is refreshed in cache so /models reflects the change immediately |
||
|
|
89f177b7b6
|
fix(galileo): use ingest traces API and standard logging payload (#29651)
* fix(galileo): use ingest traces API and standard logging payload
Switch hosted Galileo logging to /ingest/traces with nested trace/span payloads, read metrics from standard_logging_object, and include cost and total tokens on trace metrics.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(galileo): route username/password auth to v2 traces ingest
Hosted Galileo no longer serves /observe/ingest; JWT login should post the same trace payload to /v2/projects/{project_id}/traces.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(galileo): address Greptile review on logging and timestamps
Use debug-level logs for per-request Galileo callback messages and fall back to start_time/end_time when standard_logging_object omits startTime/endTime.
Co-authored-by: Cursor <cursoragent@cursor.com>
* feat(galileo): add Galileo to proxy UI callback configuration
Expose Galileo in the admin callback selector and config APIs so credentials can be configured through the dashboard instead of YAML only.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(galileo): align response type logging with Langfuse
Mirror Langfuse input/output handling for rerank, speech, transcription,
realtime, pass-through, and other response types so Galileo ingest no longer
skips supported call types.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(galileo): redact trace payload in debug logs and format with black
Avoid logging prompts and model responses in flush debug output while
keeping structural metadata for troubleshooting.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(galileo): stop logging full trace payload in debug output
Log only flush URL and trace count so prompts and model responses are not
written to application logs when debug logging is enabled.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Fix Galileo token totals and prompt messages
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
|
||
|
|
3f79222350
|
fix(proxy): persist oauth2_flow on MCP server registration (#29690) | ||
|
|
56aa55b991
|
fix(proxy): stop team BYOK model name corruption on model edit (#29731)
* fix(proxy): stop team model name corruption on edit (#28382) (#29001) Team-scoped ("Team-BYOK") models store an internal routing key model_name_{team_id}_{uuid} in the model_name column and the user-facing name in model_info.team_public_model_name. The internal name leaked into /v1, /v2, and /model/info responses; the dashboard bound its edit form to it, so any non-rename save (e.g. a TPM tweak) PATCHed the internal name back. The update path then treated it as a rename, overwriting team_public_model_name and rewriting the team's models[] ACL with the mangled string -- breaking team key calls with team_model_access_denied. Two-layer fix: - Read path (root cause): add _translate_model_name_for_response and apply it in model_info_v2 and _get_proxy_model_info so /v1, /v2, and /model/info surface the public name for team-scoped rows. The DB column and router index keep the internal name as the routing key; this is a presentation-layer swap on a shallow copy (never mutates input). - Write path (defense in depth): harden _get_public_model_name so a value matching the internal shape, or a no-op against the current DB column, is never treated as a rename -- for both the top-level model_name and an explicit model_info.team_public_model_name. Tests: regression for the reported scenario, full branch coverage of _get_public_model_name, two internal-shape guard cases, an end-to-end PATCH through _update_team_model_in_db (asserts the team ACL is untouched), and four response-translation cases. 60 passed (model management), 181 passed (proxy server). * fix(ui): key Agent Builder agent selection on model_info.id (#29729) * fix(ui): key Agent Builder agent selection on model_info.id Once team-scoped BYOK models can share a public name (the backend now returns the public name on /model/info instead of the internal routing key), selecting agents by model_name collides. Key selection, create, update and delete on the stable model_info.id instead, falling back to model_name only for config-defined agents that have no id. * fix(ui): add name-match fallback to post-create agent selection If the just-created agent's id is not yet present in the re-fetched list, try matching by name before falling back to the first agent. Addresses greptile review on #29729. --------- Co-authored-by: tushar8408 <32977767+tushar8408@users.noreply.github.com> |
||
|
|
e9417603a3
|
fix(key_generate): scope session-token team-key budget exemption to caller-supplied team_id (#29641)
Some checks failed
Unit Tests: Proxy DB Operations / assert-shard-coverage (push) Has been cancelled
Unit Tests: Security / security (push) Has been cancelled
Unit Tests: Proxy DB Operations / auth-checks (push) Has been cancelled
Unit Tests: Proxy DB Operations / budgets (push) Has been cancelled
Unit Tests: Proxy DB Operations / custom-logging (push) Has been cancelled
Unit Tests: Proxy DB Operations / db-and-spend (push) Has been cancelled
Unit Tests: Proxy DB Operations / endpoints-and-responses (push) Has been cancelled
Unit Tests: Proxy DB Operations / guardrails-hooks (push) Has been cancelled
Unit Tests: Proxy DB Operations / jwt-and-keys (push) Has been cancelled
Unit Tests: Proxy DB Operations / key-generation (push) Has been cancelled
Unit Tests: Proxy DB Operations / logging-misc (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-runtime (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-server-core (push) Has been cancelled
Unit Tests: Proxy DB Operations / schema-migration (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-utils (push) Has been cancelled
#29612 exempts UI/CLI session tokens from the key budget ceiling when they create a team key, keyed on data.team_id. That value is read after the default_key_generate_params loop can populate team_id, so on deployments that set default_key_generate_params.team_id a request the caller did not scope to a team is treated as a team key and skips the ceiling. Capture _requested_team_id before defaults run and key the exemption off it, mirroring how _requested_max_budget is already captured. Requests the caller did not scope to a team keep the ceiling. |
||
|
|
97ba7e1a30
|
fix(key_generate): exempt UI/CLI session tokens from the budget ceiling for team keys (#29612)
Non-admin users creating a team key through the UI were rejected with "max_budget cannot exceed the caller's own max_budget (0.25)". The request is authenticated by a UI/CLI session token whose max_budget is the per-session chat spend cap (max_ui_session_budget, default $0.25), and the delegated-authority budget ceiling (GHSA-q775-qw9r-2r4g) treated that cap as a delegation limit. Skip the ceiling only when a session token creates a team key (data.team_id set); that key's spend is bounded by the team budget at request time. Personal keys and every other non-admin caller keep the ceiling, so a session token cannot mint an arbitrary-budget personal key. |
||
|
|
b11833c737
|
fix(key_generate): allow team members to create keys on org-scoped teams (#29310)
* fix(key_generate): allow team members to create keys on org-scoped teams When a virtual key is created for a team, enterprise logic inherits the team's organization_id onto the key (add_team_organization_id). Since the VERIA-55 org-IDOR fix, /key/generate then required the caller to be an explicit LiteLLM_OrganizationMembership member of that org, returning 403 "Caller is not a member of organization_id=<uuid>". Admins normally only add users to teams (not orgs), so self-serve key creation regressed for any user on an org-scoped team (regression since v1.84.0-rc.1). Skip the org-membership check when organization_id was inherited from the key's team (organization_id == team_table.organization_id). Team-level authorization already gates this path, so team membership is sufficient. The membership check still runs when a caller assigns an organization_id that did not come from the key's team, preserving the IDOR protection. Adds regression tests covering both the team-inherited (allowed) and foreign-org (still blocked) cases. Co-authored-by: Cursor <cursoragent@cursor.com> * test(key_generate): cover mismatched team org IDOR path on generate Add test_generate_key_foreign_org_with_mismatched_team_still_enforces_membership for the case where a team is present but request organization_id differs from team_table.organization_id. Enterprise inheritance is no-op'd in the test so the guard is exercised directly; membership validation must still run. Addresses Greptile review on #29310. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
|
0a767ed14f
|
fix(auth): let internal users view search tools (#29542)
* fix(auth): let internal users view search tools Internal users could not see search tools in the UI even when an admin created them, while vector stores were visible. The Search Tools page rendered but its list calls 403'd because the read routes were not in internal_user_routes. Grant internal users read-only access to the listing and provider routes; create/update/delete stay admin-only. Resolves LIT-3150 * fix(auth): scope /search_tools/list to caller-allowed tools Adding the read routes to internal_user_routes let any internal user call /search_tools/list, which returned every configured tool's id, provider, api_base, and metadata regardless of the caller's object_permission.search_tools allowlist. The api_key was masked, so this was metadata disclosure rather than a credential leak, but it ignored the key/team scoping that /search already enforces. Filter the listing through the same can_key_call_search_tool / can_team_call_search_tool checks (exposed as a boolean can_user_view_search_tool), so non-admin callers only see tools they may invoke; admins still see all. Mirrors how /vector_store/list scopes results. * fix(types): resolve mypy errors in list_search_tools The config and DB build loops reused one loop variable, so mypy pinned it to the config element type (SearchToolTypedDict); its .get() calls returned object and the DB element (SearchTool) failed the reuse assignment. Give each loop its own variable so each gets its real type, and coerce the config tool's SearchToolInfoTypedDict to a plain dict to match SearchToolInfoResponse.search_tool_info. |
||
|
|
4a81ec4982
|
feat(proxy): add per-MCP-server RPM rate limiting for keys and teams (#29482)
* feat(proxy): add per-MCP-server RPM rate limiting for keys and teams
Adds mcp_rpm_limit, a dict keyed by MCP server name (alias if set, else the
configured name) that caps requests per minute per server for a key or team.
The v3 rate limiter builds a per-server descriptor only when a limit is
configured for the server being called, so other servers stay uncapped and no
TPM reservation is engaged. Server identity is surfaced into the request data
via mcp_rate_limit_server_name so the limiter can resolve it.
* fix(proxy): gate MCP rpm descriptors on call_mcp_tool; document mcp_rpm_limit param
Only honor mcp_server_name when the call is an actual MCP tool call. Without
this, a normal LLM request could inject mcp_server_name in its body to consume
a target server's MCP quota and 429 legitimate tool calls. Also adds the
mcp_rpm_limit parameter docstring to update_key, new_user, and user_update so
the API docs validator passes.
* Fix MCP rate limit quota handling
* Delete scripts/test_mcp_rpm_limit.sh
* docs(proxy): clarify mcp_rpm_limit is enforced for keys and teams, not per user
* fix(proxy): accept mcp_rpm_limit in generate_key_helper_fn
NewUserRequest and GenerateKeyRequest inherit mcp_rpm_limit from
GenerateRequestBase, so /user/new and /key/generate forwarded the field
to generate_key_helper_fn, which did not accept it and returned a 500
("unexpected keyword argument 'mcp_rpm_limit'"). Accept the param and
store it in metadata, matching model_rpm_limit/model_tpm_limit, so the
limit is persisted where get_key_mcp_rpm_limit reads it.
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
|
||
|
|
6d6eda8101
|
[internal copy of #28008] Support MCP OAuth passthrough and issuer-scoped JWT auth (#28356)
* fix(proxy): point /metrics 401 at the opt-out flag Operators upgrading past |
||
|
|
83fcacad08
|
fix(mcp): ignore stale ids on key save (#29128)
* fix(mcp): ignore stale server ids during key permission validation Prevent virtual key save failures when object_permission still includes MCP server IDs that were deleted from the registry. The validator now drops stale IDs before team scope checks and adds coverage for stale-ID and active unauthorized-ID behavior. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(mcp): use DB find_many to detect stale server IDs, add tool-permission test Addresses greptile feedback on PR #29128: P1 – Race-condition / authoritative-signal concern: Extract _get_stale_mcp_server_ids() helper that performs a single DB find_many (LiteLLM_MCPServerTable) when prisma_client is available, making server-existence checks authoritative even during in-memory registry warm-up. Falls back to the in-memory registry only when no DB client is present (config-only deployments / unit tests). Pass prisma_client through both call sites in key_management_endpoints. P2 – Missing test coverage for mcp_tool_permissions: Add test_validate_stale_ids_in_mcp_tool_permissions_silently_dropped to confirm stale server IDs that appear only as keys in mcp_tool_permissions are also stripped without raising 403. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(mcp): also check in-memory registry when DB is present to avoid misclassifying config-file servers as stale Config-file MCP servers are never written to LiteLLM_MCPServerTable, so a DB-only query would classify them as stale and silently drop their IDs from the authorization check, allowing unauthorized key access. Fix: treat a server ID as stale only when it is absent from BOTH the DB and the in-memory registry (which holds config-file servers). Co-authored-by: Cursor <cursoragent@cursor.com> * fix(mcp): normalize server aliases on key save Co-authored-by: Sameer Kankute <Sameerlite@users.noreply.github.com> * fix: remove unused stale MCP helper and capture normalized object_permission on key generation Co-authored-by: Yassin Kortam <yassin@berri.ai> * test(mcp): keep validate_key_mcp_servers_against_team stub in sync with return contract validate_key_mcp_servers_against_team now returns the (normalized) object_permission, and the key-generation helper assigns that return value back into the request data. The two key-generation tests stubbed the function with a bare AsyncMock, whose default MagicMock return value clobbered object_permission and skipped permission-record creation. Make the stubs pass object_permission through unchanged. * fix(mcp): preserve provided object_permission fields on key update The key-update path reconstructs data.object_permission from a full model_dump(), which marks every field as set. Downstream model_dump(exclude_unset=True) then emits models/blocked_tools/ search_tools as None, and those are non-nullable array columns, so the Prisma write fails whenever the UI submits object_permission with only a subset of fields populated (e.g. a TPM/RPM-only edit). Build the dict with exclude_unset=True so the normalized object_permission keeps the caller's original field set. --------- Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Sameer Kankute <Sameerlite@users.noreply.github.com> Co-authored-by: Yassin Kortam <yassin@berri.ai> Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |
||
|
|
68852ef165
|
fix(teams): expose keys_count on /v2/team/list and wire UI Resources badge (#28502) | ||
|
|
f75a7c6b22
|
fix(model-edit): allow clearing custom pricing on wildcard models (#28719)
* fix(model-edit): allow clearing custom input/output cost on wildcard deployments A user-set pricing override on a `/model/*` wildcard deployment could not be removed: clearing the Input/Output Cost fields in the UI succeeded visually, but the next read still showed the old values because both `litellm_params` and `model_info` (mirrored via `SPECIAL_MODEL_INFO_PARAMS`) retained the original rates. UI: when the pricing field is touched but left empty, send `null` instead of dropping it from the payload so the backend sees the clear intent. The cache-read-cost fallback now guards against `null` as well as `undefined` so a cleared input cost cannot silently wipe the cache-read override. Backend: `update_db_model` honors explicit-null clears, but ONLY for `SPECIAL_MODEL_INFO_PARAMS` (the 4 pricing fields). Restricting the null-clear path prevents a team-scoped caller from using this codepath to null out privileged fields like `team_id` or access groups. Tests cover both clear paths (`litellm_params` and `model_info`), the SPECIAL_MODEL_INFO_PARAMS mirror, PATCH semantics for omitted fields, and the security guard that non-pricing nulls don't reach the merged dict. Resolves LIT-3250 * fix(model-edit): run null-clears after both merges, not interleaved The previous version cleared `model_info` from inside the litellm_params merge block, but the subsequent `model_info.update(...)` re-injected the old pricing because the UI's PATCH carries the full model_info blob with the stale values still in it. Move the explicit-null clear pass to after both merges so a model_info passthrough cannot resurrect cleared fields. Adds a regression test for the realistic UI submit shape (both blobs in the patch, model_info still holding the old pricing). * test(e2e): clear-custom-pricing flow with create/delete cleanup Covers the dashboard model edit form's pricing-clear flow end-to-end: seeds a deployment with custom input/output pricing, drives the UI to clear both fields, asserts the outgoing PATCH sends explicit nulls, and confirms via /v2/model/info that the override is gone from both litellm_params and model_info. The dashboard DB persists across this suite, so beforeEach creates a uniquely-named deployment and afterEach POSTs /model/delete to leave the DB clean regardless of test outcome. * fix(model-edit): extend pricing clear to cache_read and cache_write costs Pre-existing parallel of the wildcard input/output cost bug: cleared cache_read_input_token_cost and cache_creation_input_token_cost overrides silently persisted because the UI omitted the key (delete or fallback) and the backend null-clear allowlist did not cover them. - types/router.py: add cache_read_input_token_cost and cache_creation_input_token_cost to SPECIAL_MODEL_INFO_PARAMS, so they are mirrored between litellm_params and model_info by Deployment.__init__ and honoured by the null-clear loop in update_db_model. - model_info_view.tsx: emit explicit null for touched-but-empty cache_read and cache_write fields. Preserve the input_cost->cache_read mirror only when cache_read itself was not touched. - model_management_endpoints.py: update the allowlist comment. - Tests: three new unit tests for cache clear paths and a preserve check; the e2e spec now seeds, clears, and asserts null PATCH + key-absence for all four pricing fields. |
||
|
|
5f73ad4fe7
|
fix(team): refresh team cache on team_model_add/delete (LIT-3244) (#28683)
* fix(team): refresh team cache on team_model_add/delete (LIT-3244)
team_model_add and team_model_delete wrote to the DB but did not
invalidate the in-memory LiteLLM_TeamTableCachedObj used by
common_checks. After the v1.83.14 common_checks centralization made
team.models authoritative on /v1/files and /v1/vector_stores/*,
adding a Team-BYOK model silently failed to grant the new public
model name to team members until the cache TTL expired (and a
removed model kept working until then on the symmetric path).
Extract the cache-refresh snippet from update_team into a small
helper and apply it consistently at all three team-write sites.
* test: also assert updated models in team-cache-refresh pin
Strengthens the LIT-3244 regression test to also assert
`call_kwargs["team_table"].models` matches the updated row,
not just `team_id`. Both `existing_team` and `updated_team`
share `team_id` in the test setup, so the previous assertion
would have passed even if the implementation accidentally cached
the pre-mutation row.
Greptile review feedback.
* fix(team): hydrate object_permission on cache-refreshing team updates
The Prisma update calls in update_team, team_model_add, and
team_model_delete returned a team row with object_permission_id set
but object_permission=None (the relation was not requested via
include=). _refresh_cached_team then wrote that to the in-memory
LiteLLM_TeamTableCachedObj, and the cache-hit path in get_team_object
returns the cached object without re-hydrating. Downstream consumers
(validate_key_search_tools_against_team, the MCP/agent authz paths)
treat a missing object_permission as no team-level restriction, so
a team-write op silently dropped object-permission enforcement until
the cache TTL expired or a DB-fetch path re-hydrated it.
Add include={"object_permission": True} to all three updates so the
refresh writes a complete cached team. Extend the LIT-3244 regression
test to pin both the cached object_permission and the include shape
on the Prisma call.
Surfaced in PR review of LIT-3244.
|
||
|
|
3b2ce201d8
|
encrypt callback_vars in key/team metadata at rest (#27141)
Co-authored-by: Michael Riad Zaky <michaelr@Michaels-MacBook-Air.local> Co-authored-by: Yuneng Jiang <yuneng@berri.ai> |
||
|
|
b0b25ae4b9
|
Include team alias in CLI JWT token (#28621) | ||
|
|
50a3f10a92
|
feat(proxy): persist allowlisted OIDC claims in CLI SSO poll (#28463)
* feat(proxy): persist allowlisted OIDC claims in CLI SSO poll
Map CLI_SSO_CLAIM_MAP sources into user metadata and return scalar
attribution_metadata from /sso/cli/poll. Build SSOUserDefinedValues in
cli_sso_callback so first-time CLI logins can upsert users. Add mock OIDC
scripts and tests for claim extraction and poll exposure.
Co-authored-by: Cursor <cursoragent@cursor.com>
* docs(proxy): document CLI SSO attribution_metadata in client README
Co-authored-by: Cursor <cursoragent@cursor.com>
* Delete scripts/mock_oidc_server_for_cli_sso.py
* Delete scripts/test_cli_sso_claims_e2e.py
* fix(ui_sso): preserve claim types and avoid metadata. prefix stripping
- Replace _update_dictionary with a local recursive merge so string
OIDC claim values that happen to look numeric are not silently coerced
to int/float when persisting CLI SSO attribution metadata.
- Use a local dot-path resolver in _extract_sso_claim_value so that
source claim paths beginning with 'metadata.' are not silently stripped
by get_nested_value (which is designed for LiteLLM JWT metadata, not
arbitrary OIDC claims).
Co-authored-by: Yassin Kortam <yassin@berri.ai>
* Remove redundant metadata. prefix strip in _set_nested_metadata_value
The _parse_cli_sso_claim_map already strips the metadata. prefix from
dest keys before reaching the setter. The duplicate strip in
_set_nested_metadata_value was a no-op in normal flow but could
mis-place values for dest keys like metadata.metadata.foo.
Co-authored-by: Yassin Kortam <yassin@berri.ai>
* Fix greptile
* Fix ruff
* Move CLI SSO user defined values build inside try/except for consistent error handling
Co-authored-by: Yassin Kortam <yassin@berri.ai>
* fix(proxy): enforce restricted SSO group on CLI SSO callback
Apply verify_user_in_restricted_sso_group before CLI session completion
and user upsert, matching the UI SSO path. Re-raise ProxyException so
restricted-group denials return 403 instead of 500.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(proxy): replace recursive CLI SSO metadata helpers with iterative merge
Use stack-based flatten/merge to satisfy recursive_detector CI. Fix mypy
types for UserApiKeyCache and user_id on CLI SSO session completion.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix: resolve nested CustomOpenID extra_fields in CLI SSO claim extraction
When GENERIC_USER_EXTRA_ATTRIBUTES captures a parent object (e.g. org_info),
extra_fields stores it as {"org_info": {"department": "..."}}. A CLI claim
map entry using a dotted path like org_info.department would silently fail
because the lookup only checked the exact flat key. Fall back to dotted-path
resolution on extra_fields before model_dump().
Co-authored-by: Yassin Kortam <yassin@berri.ai>
* fix(sso): update CLI SSO test for new received_response kwarg and remove redundant 'token' secret fragment
Co-authored-by: Yassin Kortam <yassin@berri.ai>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Yassin Kortam <yassin@berri.ai>
|
||
|
|
988196911a
|
Litellm oss staging 1 (#28337)
* feat: add Xiaomi MiMo-V2.5-Pro and MiMo-V2.5 OpenRouter model entries (#27700) Squash-merged by litellm-agent from TorvaldUtne's PR. * fix(ui): trim whitespace from MCP inspector tool call inputs (#28203) Co-authored-by: shin-berri <shin-laptop@berri.ai> Co-authored-by: yuneng-jiang <yuneng@berri.ai> * gemini-3.1-flash-lite pricing (#27933) * feat(model_prices): add gemini-3.1-flash-lite pricing with standard/batch/flex/priority tiers * fix pricing * add service tier --------- Co-authored-by: shin-berri <shin-laptop@berri.ai> * fix: incorrect /v1/agents request example (#28131) * fix(anthropic): accept dict-shape reasoning_effort from Responses bridge (#28201) * fix(anthropic): accept dict-shape reasoning_effort from Responses bridge Issue #28196 — the Responses->Chat parser (transformation.py:184-200) keeps the full dict as reasoning_effort when summary is set; that branch was added in #25359. But the Anthropic transformation here still guarded on isinstance(value, str), silently dropping the param. Result: callers using the standard Reasoning(effort, summary) OpenAI-shaped object on Anthropic lose thinking entirely (0 reasoning_tokens, no thinking_blocks). Coerce dict -> string before mapping. Same shape tolerance that gpt_5_transformation._normalize_reasoning_effort_for_chat_completion already implements. summary is irrelevant for Anthropic's thinking_blocks. Adds two regression tests: one parametrized over string + dict shapes (with and without summary), one covering unparseable dict inputs (drops silently, no crash). * test(anthropic): add non-adaptive model coverage for dict-shape reasoning_effort Per Greptile feedback on PR #28198: the original regression test only exercised the adaptive (4.6+) path. Add a parametrized test for the non-adaptive branch (claude-sonnet-4-5) verifying that dict-shape reasoning_effort still maps to thinking.type='enabled' + budget_tokens, and that output_config is NOT set on pre-4.6 models. * test(anthropic): convert unparseable-dict test to @pytest.mark.parametrize Per @greptile-apps inline review on PR #28201 — matches the parametrize style of the two adjacent dict-shape tests and produces clearer failure messages (test ID per case instead of one collapsing for-loop). * feat: add pricing entry for openrouter/google/gemini-3.1-flash-lite (#28280) Squash-merged by litellm-agent from ro31337's PR. * fix(router): wrap aresponses streaming iterator for mid-stream fallbacks (#28215) Squash-merged by litellm-agent from cwang-otto's PR. * fix(router): unblock staging — mypy + coverage for aresponses streaming fallback (#28318) Squash-merged by litellm-agent from cwang-otto's PR. * fix(responses): forward timeout on completion transformation path (Anthropic, Bedrock, Vertex) (#28133) Squash-merged by litellm-agent from cwang-otto's PR. * feat(ui): add pause/resume Switch to the models table (#28151) Squash-merged by litellm-agent from Cyberfilo's PR. * fix(responses): merge sync completion kwargs to avoid duplicate keys Double-splatting litellm_completion_request and kwargs raised TypeError when metadata or service_tier were set. Match the async merge pattern. Co-authored-by: Cursor <cursoragent@cursor.com> * Use proxy base URL for CLI SSO form action (#28271) Co-authored-by: shin-berri <shin-laptop@berri.ai> Co-authored-by: yuneng-jiang <yuneng@berri.ai> * fix(tests): add mistral/ministral-8b-2512 to cost map and backfill in conftest Mistral rotated the 'mistral/mistral-tiny' alias to return 'ministral-8b-2512' as the response model, which was missing from the cost map. This caused test_completion_mistral_api and test_completion_mistral_api_modified_input to fail in litellm.completion_cost lookup. - Add mistral/ministral-8b-2512 entry to both the in-tree model_prices_and_context_window.json and the bundled litellm/model_prices_and_context_window_backup.json (mirrors the existing openrouter/mistralai/ministral-8b-2512 pricing). - litellm.model_cost is loaded at import time from the URL pinned to main, so the new backup entry isn't visible at test runtime until it also lands on main. Backfill any entries missing from the remote-fetched map into litellm.model_cost in the local_testing conftest so cost-calculator lookups succeed on this branch. * fix(tests): drop unnecessary del of conftest backfill loop vars * fix(router): harden streaming fallback wrapper for bridge iterators - FallbackResponsesStreamWrapper now uses getattr fallbacks when copying attributes from the source iterator. The bridge path (LiteLLMCompletionStreamingIterator used by Anthropic/Bedrock/Vertex) does not call super().__init__ and is missing response, logging_obj (it uses litellm_logging_obj), responses_api_provider_config, start_time, request_data, call_type, and _hidden_params. Previously, wrapper construction raised AttributeError for any streaming fallback on the bridge path. - _aresponses_with_streaming_fallbacks now deep-copies the litellm_metadata (and metadata) dicts into fallback_kwargs. The primary attempt mutates this dict in place via _update_kwargs_with_deployment, so a shallow copy of kwargs was leaking primary-deployment fields (deployment, model_info, api_base) into the mid-stream fallback request. Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix(router): use safe_deep_copy for fallback metadata snapshot The ban_copy_deepcopy_kwargs CI check rejects copy.deepcopy() on any variable whose name contains 'kwargs' (incl. fallback_kwargs). Swap the two copy.deepcopy(fallback_kwargs[...]) calls for safe_deep_copy, which handles non-picklable values (OTEL spans, etc.) by per-key deepcopy with fallback to the original reference. Co-authored-by: Yassin Kortam <yassin@berri.ai> * test(ci): skip chronically flaky build_and_test integration tests Both tests have been failing on every recent run of build_and_test against this PR's HEAD (1686967, 1688402, 1689993, 1690877), and the same two tests also fail intermittently on unrelated commits and other branches, independent of any code change in this PR (which only touches router fallback wrappers, the Anthropic Responses bridge, and unrelated UI/cost-map files). - tests.test_spend_logs.test_spend_logs: /spend/logs?request_id=... returns 500 even after a 20s wait for the spend log to be written. Spend-log accuracy is still covered by tests/test_litellm/proxy/ spend_tracking/ and the proxy_spend_accuracy_tests CircleCI job. - tests.test_team_members.test_add_multiple_members: /team/info?team_id= ... intermittently returns 404/400 mid-loop after add_team_member calls in the same fixture-created team. Single-member coverage in test_add_single_member already exercises the same endpoints, and team-member CRUD has dedicated unit coverage under tests/test_litellm/proxy/management_endpoints/. Skipping unblocks the build_and_test job until the underlying race in the dockerized integration setup is root-caused. * fix: preserve explicit timeout=0 in responses API handler Use 'timeout if timeout is not None else request_timeout' instead of 'timeout or request_timeout' so an explicit timeout=0/0.0 isn't silently replaced by the default request_timeout. Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix(ui): guard model_info access in pause Switch with optional chaining * fix(ui): guard model_info access in pause Switch onChange handler Mirror the optional-chaining guard already applied to the isPausing check so a config-model row with a missing model_info cannot throw when the toggle's onChange fires. --------- Co-authored-by: TorvaldUtne <78661304+TorvaldUtne@users.noreply.github.com> Co-authored-by: oss-agent-shin <ext-agent-shin@berri.ai> Co-authored-by: shin-berri <shin-laptop@berri.ai> Co-authored-by: yuneng-jiang <yuneng@berri.ai> Co-authored-by: mubashir1osmani <mubashir.osmani777@gmail.com> Co-authored-by: Isha <72744901+IshaMeera@users.noreply.github.com> Co-authored-by: cwang-otto <chengxuan.wang@ottotheagent.com> Co-authored-by: Roman Pushkin <roman.pushkin@gmail.com> Co-authored-by: Filippo Menghi <113345637+Cyberfilo@users.noreply.github.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: boarder7395 <37314943+boarder7395@users.noreply.github.com> Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> Co-authored-by: Claude <claude@anthropic.com> Co-authored-by: Yassin Kortam <yassin@berri.ai> |
||
|
|
36c494fdd2
|
Litellm oss staging (#28161)
* fix(opentelemetry): JSON-serialize dict metadata fields for OTEL span attributes (#27451) (#27455) Squash-merged by litellm-agent from Anai-Guo's PR. * feat(dashscope): add embeddings and reranks(qwen3-rerank) support via OpenAI-compatible endpoint (#27508) Squash-merged by litellm-agent from yimao's PR. * fix(vertex_ai/gemini): raise BadRequestError when image_url or url fi… (#24550) Squash-merged by litellm-agent from krisxia0506's PR. * fix(vertex_ai): raise error on mid-stream 429/error chunks instead of silently swallowing (#23711) Squash-merged by litellm-agent from krisxia0506's PR. * fix: raise BadRequestError for file content blocks missing 'file' sub… (#24503) Squash-merged by litellm-agent from krisxia0506's PR. * Fix Gemini MIME detection for extensionless GCS URIs (#27278) Squash-merged by litellm-agent from krisxia0506's PR. * fix(vertex_ai/partner_models): drop unused vertexai SDK gate from count_tokens (closes #28084) (#28107) Squash-merged by litellm-agent from voidborne-d's PR. * feat(chart): add support for autoscaling behavior in HPA (#27990) Squash-merged by litellm-agent from FabrizioCafolla's PR. * feat(proxy): add blocked flag to models for pause/resume from the UI (#27927) Squash-merged by litellm-agent from Cyberfilo's PR. * fix: pass socket timeouts to Redis cluster clients (#27920) Squash-merged by litellm-agent from tomdee's PR. * Fix/cache token (#28009) Squash-merged by litellm-agent from escon1004's PR. * fix(deepseek): forward reasoning_content in multi-turn thinking mode conversations (#28080) Squash-merged by litellm-agent from Divyansh8321's PR. * fix(guardrails): return HTTP 400 instead of 500 for blocked requests (#27617) * fix: reset org and tag budgets (#27326) * reset org budgets * reset tag budgets --------- Co-authored-by: Michael Riad Zaky <michaelr@Mac.localdomain> * fix(ui): omit allowed_routes from key edit save when unchanged (#27553) * fix(ui): omit allowed_routes from key edit save when unchanged When a team admin opens Edit Settings on a key with key_type=AI APIs and saves without changing anything, the UI re-sends the existing allowed_routes value, which the backend's _check_allowed_routes_caller_permission gate rejects for non-proxy-admins (LIT-2681). Strip allowed_routes from the patch in handleSubmit when it deep-equals the original keyData.allowed_routes. The backend treats absence as "leave alone," so no-op saves now succeed for non-admins. Admins explicitly editing the field still send the new value. * fix(ui): order-insensitive allowed_routes diff + cover null-original case Address Greptile review: - Switch the "is allowed_routes unchanged" check to a Set-based comparison so a server-side reorder of the array doesn't register as a user edit and re-trigger LIT-2681. - Add two regression tests: (1) keyData.allowed_routes is null and the form is untouched — patch should strip the field; (2) server returned routes in a different order than the user originally entered — patch should still recognize the value as unchanged. * chore(ui): strip ticket refs and tighten comments in key edit fix - Remove internal-tracker references from in-code comments - Tighten the WHY comment in handleSubmit to two lines - Drop redundant test-block comments — test names already describe the case * fix(ui): annotate Set<string> generic in allowed_routes diff to fix tsc * fix(guardrails): return HTTP 400 instead of 500 for guardrail-blocked requests GuardrailRaisedException and BlockedPiiEntityError both lacked a status_code attribute. When these exceptions reached the proxy exception handler (getattr(e, 'status_code', 500)), the fallback defaulted to HTTP 500 — making intentional guardrail blocks indistinguishable from server errors and causing unnecessary client retries. Changes: - Add status_code=400 (keyword-only) to GuardrailRaisedException - Add status_code=400 (keyword-only) to BlockedPiiEntityError - Update _is_guardrail_intervention() to recognize both exceptions so downstream loggers record 'guardrail_intervened' instead of 'guardrail_failed_to_respond' - Add 6 unit tests for default/custom status codes and getattr pattern - Strengthen existing blocked-action test with status_code assertion Fixes #24348 --------- Co-authored-by: Michael-RZ-Berri <michael@berri.ai> Co-authored-by: Michael Riad Zaky <michaelr@Mac.localdomain> Co-authored-by: ryan-crabbe-berri <ryan@berri.ai> Co-authored-by: Krrish Dholakia <krrish+github@berri.ai> * fix(router/proxy): address Greptile P1+P2 review comments on PR #28161 - router: raise ServiceUnavailableError (503) instead of RouterRateLimitErrorBasic (429) when a specifically-addressed deployment is administratively blocked; 429 misleads retry-enabled clients into spinning forever against a paused model - proxy_server: compute get_fully_blocked_model_names() once before both branches in model_list() instead of duplicating the call in each branch - deepseek: upgrade silent debug log to warning when injecting placeholder reasoning_content so callers are clearly notified of degraded multi-turn quality - tests: update two blocked-deployment assertions to expect ServiceUnavailableError Co-authored-by: Cursor <cursoragent@cursor.com> * fix: address bug detection findings (cache token order, mutable defaults) Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix: address bugs in async pass-through, anthropic cache token detection, rerank tests - async_get_available_deployment_for_pass_through: enforce blocked check on specific deployments - cost_calculator: detect anthropic-style usage by attribute presence (not truthiness) to avoid mixing OpenAI cached_tokens into anthropic normalization when read=0 - dashscope rerank tests: pass request to httpx.Response constructions for consistency Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix code qa * fix(vertex_ai/gemini): strip MIME parameters from GCS contentType GCS object metadata's contentType field can include parameters such as 'text/html; charset=utf-8'. Strip them in _apply_gemini_mime_type_aliases so downstream get_file_extension_from_mime_type sees a bare MIME type. Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix(vertex_ai/gemini): clarify mime-type error message string concatenation Co-authored-by: Yassin Kortam <yassin@berri.ai> --------- Co-authored-by: Tai An <antai12232931@outlook.com> Co-authored-by: Vincent <yimao1231@gmail.com> Co-authored-by: Kris Xia <xiajiayi0506@gmail.com> Co-authored-by: d 🔹 <liusway405@gmail.com> Co-authored-by: Fabrizio Cafolla <developer@fabriziocafolla.com> Co-authored-by: Filippo Menghi <113345637+Cyberfilo@users.noreply.github.com> Co-authored-by: Tom Denham <tom@tomdee.co.uk> Co-authored-by: escon1004 <70471150+escon1004@users.noreply.github.com> Co-authored-by: Divyansh Singhal <97736786+Divyansh8321@users.noreply.github.com> Co-authored-by: robin-fiddler <robin@fiddler.ai> Co-authored-by: Michael-RZ-Berri <michael@berri.ai> Co-authored-by: Michael Riad Zaky <michaelr@Mac.localdomain> Co-authored-by: ryan-crabbe-berri <ryan@berri.ai> Co-authored-by: Krrish Dholakia <krrish+github@berri.ai> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Yassin Kortam <yassin@berri.ai> |
||
|
|
fe63650ebd
|
fix(proxy): gate team allowed_passthrough_routes to proxy admins (#28097)
* fix(proxy): gate team allowed_passthrough_routes to proxy admins allowed_passthrough_routes short-circuits the role-based route gate, so the keys endpoints already restrict it to proxy admins. The team writers (/team/new, /team/update) had no equivalent check, letting an org admin (a non-proxy-admin who clears the route gate and _verify_team_access) self-grant pass-through routes on their team. Lift the keys check into a shared helper and apply it to both team endpoints. Resolves LIT-3019 * docs(proxy): note view-only admins are intentionally excluded from passthrough gate Clarifies the proxy-admin guard per review feedback; no behavior change. Refs LIT-3019 |
||
|
|
5aabfccf57
|
fix(mcp): allow delegate PKCE bypass for internal MCP servers
Remove available_on_public_internet gating from delegate-auth-to-upstream paths so oauth2 + delegate_auth_to_upstream interactive servers behave the same when marked internal. Keeps M2M exclusion. Updates tests. |
||
|
|
410ce761dc
|
fix: tighten budget field validation and authorization checks (#27897)
* fix: block NaN/Inf budget bypass and add missing non-admin guards
Addresses three security issues:
GHSA-wvg4-6222-3q4r: /user/update exposes max_budget, soft_budget, spend
to self-editing non-admin users with no server-side guard. Non-admin callers
now receive HTTP 403 if any of those fields appear in the update payload.
GHSA-q775-qw9r-2r4g: _enforce_upperbound_key_params returned early (no-op)
when upperbound_key_generate_params was absent from config, letting any
authenticated user generate a key with unlimited max_budget. Fix adds a
delegated-authority ceiling in _common_key_generation_helper: non-admins
cannot grant a key more budget than their own key carries.
GHSA-2rv4-xv66-fpjg: float('nan') passes every `value < 0` guard because
nan < 0 is False in Python, and spend >= nan is always False, permanently
disabling budget enforcement for any entity carrying a NaN max_budget.
All write-time budget guards now use `not math.isfinite(v) or v < 0`.
_enforce_upperbound_key_params validates finiteness unconditionally (before
the early-return). All spend-enforcement comparisons in auth_checks.py are
now guarded with math.isfinite(max_budget) as defense-in-depth.
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
* fix: close budget ceiling bypass for callers with no max_budget (GHSA-q775)
Non-admin callers whose API key has no explicit max_budget (None) could
bypass the delegated-authority ceiling and create keys with arbitrary
budgets. Now blocks budget assignment when caller has no budget configured.
Also removes redundant inline import of LitellmUserRoles.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: only apply budget ceiling to explicitly requested max_budget
Capture the caller-supplied max_budget before _enforce_upperbound_key_params
can fill it with a default, so auto-filled defaults don't trigger the
ceiling guard for non-admin users with no budget on their own key.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: capture requested max_budget before any defaults are applied
Move _requested_max_budget capture before both default_key_generate_params
and upperbound_key_generate_params mutations, so auto-filled values don't
trigger the ceiling check for non-admin users.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: allow unlimited-budget callers to delegate any budget
Callers with max_budget=None (unlimited) can legitimately create
budget-capped keys. Only block when caller has an explicit budget
and the requested budget exceeds it.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: yuneng-jiang <yuneng@berri.ai>
Co-authored-by: ryan-crabbe-berri <ryan@berri.ai>
Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
|
||
|
|
8bbc61e03c
|
fix: harden /key/update authorization checks (#27878)
* fix: patch Host-header auth bypass in get_request_route Starlette reconstructs request.url from the Host header. A malformed Host like `localhost/?x=1` causes Starlette to build the full URL as `http://localhost/?x=1/health`, which url-parses to path="/". Since "/" is in LiteLLMRoutes.public_routes, all protected routes became reachable without authentication. Fix: read scope["path"] (set by uvicorn from the HTTP request line, not derivable from headers) instead of request.url.path. Sub-path deployments are handled via scope["app_root_path"] / scope["root_path"], mirroring Starlette's own base_url construction logic. Affected variants confirmed fixed: Host: localhost/?x=1 Host: localhost:4000/?x=1 Host: localhost/#test Host: localhost:4000/#test Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * style: reduce comments in route fix Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * fix: block credential fields in RAG ingest vector_store options Credential fields (vertex_credentials, aws_access_key_id, api_key, etc.) in ingest_options.vector_store are now rejected at the API boundary with a 400 error. Credentials must be configured server-side. Previously any authenticated user could supply a vertex_credentials dict with type=external_account pointing credential_source.file at an arbitrary path (e.g. /proc/1/environ) and token_url at an attacker-controlled server. google-auth's identity_pool.Credentials refresh() would read the file and POST its contents to the attacker. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * fix: block /key/update self-escalation by assigned users Non-admin users who were assigned a key (created_by != caller) could update any non-budget field — models, rpm_limit, guardrails, etc. — without admin authorization, allowing privilege self-escalation. Gate: only the key creator (created_by == caller) may edit their own key without admin check; budget changes always require admin regardless of creator status. All other callers must pass _check_key_admin_access. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * fix: block user-controlled api_base in RAG ingest vector_store options A user-supplied api_base in ingest_options.vector_store caused the server to forward its configured provider credentials (Gemini, OpenAI) to an attacker-controlled endpoint via SSRF. Add api_base to the blocked credential params set alongside api_key and the existing credential fields. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * fix: restrict /utils/transform_request to PROXY_ADMIN and apply body safety check Any authenticated internal_user could POST arbitrary provider config (aws_sts_endpoint, api_base, etc.) to /utils/transform_request and have the server forward its credentials to an attacker-controlled endpoint. - Gate the endpoint on PROXY_ADMIN role (403 for all other roles) - Call is_request_body_safe() to reject banned params even for admins - Convert ValueError from safety check to HTTP 400 Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * fix: apply banned-param check to /utils/transform_request Without is_request_body_safe(), any authenticated user could pass aws_sts_endpoint, api_base, or aws_web_identity_token to /utils/transform_request and have the server forward its configured provider credentials to an attacker-controlled endpoint during SDK credential resolution. Applies the same banned-param blocklist already used by LLM endpoints. Endpoint remains accessible to all authenticated users. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * fix: block SSRF via api_base in /prompts/test dotprompt YAML frontmatter Any frontmatter key not in ["model","input","output"] flowed into optional_params and was merged into the LLM call data dict, bypassing is_request_body_safe. An attacker with any bearer key could set api_base in YAML to redirect the outbound LLM request — including the provider API key — to an attacker-controlled host. Fix: call is_request_body_safe on the constructed data dict after optional_params are merged, before invoking ProxyBaseLLMRequestProcessing. ValueError from the banned-param check is surfaced as HTTP 400. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * Update litellm/proxy/rag_endpoints/endpoints.py Co-authored-by: veria-ai[bot] <224490171+veria-ai[bot]@users.noreply.github.com> * fix: coerce nested config strings before banned-param check _NESTED_CONFIG_KEYS descent used isinstance(nested, dict) which silently skipped litellm_embedding_config when delivered as a JSON string via multipart/form-data. Banned params (api_base, aws_sts_endpoint, etc.) nested inside the stringified value were invisible to is_request_body_safe. _NESTED_METADATA_KEYS already used _coerce_metadata_to_dict which parses JSON strings before checking. Apply the same coercion to _NESTED_CONFIG_KEYS. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * fix: replace substring match with prefix match in is_llm_api_route mapped_pass_through_routes used `_llm_passthrough_route in route` (substring) so any admin-only path whose URL contained a provider name (openai, anthropic, azure, bedrock, etc.) was misclassified as an LLM API route and bypassed the admin gate in non_proxy_admin_allowed_routes_check. Confirmed live: non-admin key could GET /credentials/by_name/openai (read masked provider API key) and DELETE /credentials/openai (delete credential). Fix: use exact match or startswith(prefix + "/") — the same pattern used everywhere else in RouteChecks — so only routes that actually start with a passthrough prefix are allowed through. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * fix: stabilize PR #27878 test failures - key_management_endpoints: extend can_skip_admin_check to team keys so team members with /key/update permission can update non-budget fields. can_team_member_execute_key_management_endpoint already validates team membership + permission and raises if unauthorized; reaching the admin check on a team key means the caller was authorized. - test: set created_by on mock key in test_update_key_non_budget_fields_allowed_for_internal_user so caller_is_creator resolves correctly (MagicMock default ≠ user_id). - auth_utils.get_request_route: guard against non-dict request.scope (e.g. MagicMock in unit tests) to prevent a MagicMock leaking into UserAPIKeyAuth.request_route and failing Pydantic validation. - ci: assign test_multipart_bypass_repro.py to the proxy-runtime shard in test-unit-proxy-db.yml to satisfy the shard-coverage check. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * fix(lint): add explicit str() cast in get_request_route for MyPy scope.get() returns Any|None which MyPy cannot coerce to str implicitly. Wrap both scope.get() calls in str() to satisfy the type checker. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * fix: guard bare-/ root_path strip + make total_spend migration idempotent auth_utils.get_request_route: when Starlette sets scope["app_root_path"] to "/" (e.g. behind some middleware), the old stripping logic would remove the leading slash from every path ("/team/new" → "team/new"), breaking route matching and causing auth to misclassify protected routes. Skip stripping when root_path is bare "/". migration: add IF NOT EXISTS to total_spend ALTER TABLE so the migration is safe to replay when a prior partial run already created the column. Without this guard, prisma migrate deploy fails on CI DBs that were partially migrated, causing all subsequent DB operations (including /team/new) to 500. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * fix: require creator still owns key for personal-key bypass in /key/update caller_is_creator now requires both created_by == caller AND user_id == caller. Previously checking only created_by let a demoted admin who originally created a key for another user continue editing non-budget fields on it after reassignment, bypassing _check_key_admin_access. Adds regression test: creator whose key was reassigned is blocked (403). Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * fix: extract auth checks to fix PLR0915 + broaden max_budget assertion internal_user_endpoints._update_single_user_helper exceeded 50 statements (PLR0915). Extract authorization checks into _check_user_update_authz helper to bring statement count under the limit. test_validate_max_budget: assert "negative" (substring of both the local "cannot be negative" and the CI "non-negative finite number" messages) so the test is stable regardless of which exact wording the function uses. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: veria-ai[bot] <224490171+veria-ai[bot]@users.noreply.github.com> |
||
|
|
18f77ff7bc
|
feat(mcp): add delegate_auth_to_upstream flag for PKCE passthrough (#27834)
* feat(mcp): add delegate_auth_to_upstream flag for PKCE passthrough Adds an opt-in per-server flag that lets clients (e.g. VS Code) complete PKCE directly with an upstream OAuth2 MCP server, instead of LiteLLM double-gating with its own API-key/SSO check. Only honored when auth_type=oauth2 and the operator explicitly sets the flag; mixed-target or non-oauth2 requests fail closed. - Adds the field to Pydantic models, Prisma schema, and a migration - New MCPRequestHandler._target_servers_delegate_auth_to_upstream gate that runs only when no x-litellm-api-key is present, so authenticated users still get user_id resolution + stored-credential lookup - Anonymous callers now see delegate servers in get_allowed_mcp_servers (scoped to delegate servers only; the upstream still enforces auth) - mcp_management_endpoints: allow anonymous /authorize and /token for delegate servers so VS Code can complete PKCE without a LiteLLM session - UI toggle (shown only for oauth2) + payload/view wiring - Tests covering: oauth2 on/off, non-oauth2 with flag, mixed targets, no resolvable target, explicit key precedence, and 401 emission Co-authored-by: Cursor <cursoragent@cursor.com> * Enforce oauth2 for delegated MCP auth bypass Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix(mcp): close secondary Authorization bypass for delegate servers The delegate-auth bypass gated only on the primary `x-litellm-api-key` header, so a LiteLLM key sent via `Authorization: Bearer sk-...` (the secondary header) was silently dropped — skipping spend tracking and rate limiting. Gate on the resolved litellm_api_key (which considers both headers) so the bypass fires only when neither is present. Also update the existing "Authorization header present" test to reflect that an upstream OAuth token now flows through the existing oauth2 fallback (LiteLLM auth attempt → fail → anonymous), not via the delegate branch. Co-authored-by: Cursor <cursoragent@cursor.com> * Avoid duplicate MCP OAuth credential lookup Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix(mcp): block delegate bypass for M2M and internal-only servers Two security issues flagged in code review: 1. High – client_credentials (M2M) servers must not be delegatable: LiteLLM auto-fetches the upstream token using stored credentials, so allowing anonymous bypass would let any external caller invoke tools authenticated as LiteLLM's service account. Fix: check `server.has_client_credentials` in `_target_servers_delegate_auth_to_upstream`, the anonymous allow-list in `get_allowed_mcp_servers`, and `_mcp_oauth_user_api_key_auth`. 2. Medium – internal-only servers exposed to public internet: The anonymous delegate allow-list was not filtering by `available_on_public_internet`, so external callers with an upstream OAuth token could invoke tools on servers marked internal-only. Fix: add `available_on_public_internet` guard to the anonymous delegate server list in `get_allowed_mcp_servers`. Tests added for both cases. Co-authored-by: Cursor <cursoragent@cursor.com> * Require public MCP delegate auth servers Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix(mcp): align delegate auth path parsing with downstream routing `_extract_target_server_names_from_path` used a naive segments-based split while `server.py::_get_mcp_servers_in_path` uses a regex that allows server names with one embedded slash and comma-separated lists. With the old parser, a request to `/mcp/<delegated>/<garbage>` was parsed as targeting `<delegated>` by the auth gate (bypassing LiteLLM auth) while the routing layer parsed it as `<delegated>/<garbage>` — when that name did not resolve, the request fell back to the anonymous allow-list, which can include `allow_all_keys` servers that normally require a LiteLLM key. Replace the parser with the same regex logic as `_get_mcp_servers_in_path` so auth gating sees the exact target name(s) downstream routing sees. Add regression tests covering parser parity and the specific extra-path-segment bypass attempt. https://claude.ai/code/session_01SjyPmwfmrq8fveFgw9iHW9 * fix(mcp): close header/path TOCTOU in MCP delegate auth gate `_target_servers_delegate_auth_to_upstream` and `_target_servers_use_oauth2` trusted the `x-mcp-servers` header when present, but `server.py::extract_mcp_auth_context` overrides that header with the path-derived list for `/mcp/...` routes. An attacker could set `x-mcp-servers: <delegated>` while pointing the URL path at a non-delegate server, flipping the auth gate without changing the target downstream routing actually uses. Extract a shared `_resolve_target_server_names` helper that mirrors the downstream override (path-derived names for `/mcp/...` routes, header value otherwise). Add regression tests covering the TOCTOU attempt and the helper's path-vs-header precedence. https://claude.ai/code/session_01SjyPmwfmrq8fveFgw9iHW9 * Fix delegated MCP OAuth test mock Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix(mcp): drop unreachable /{server}/mcp branch in auth path parser `_extract_target_server_names_from_path` also matched the ``/{server_name}/mcp`` form, but the downstream parser ``_get_mcp_servers_in_path`` only handles ``/mcp/...`` — and ``dynamic_mcp_route`` in ``proxy_server`` rewrites ``/{name}/mcp`` to ``/mcp/{name}`` on the scope before the MCP handler runs. Parsing the un-rewritten form on the auth side was therefore unreachable in production, and contradicted the docstring's claim of mirroring the downstream parser — exactly the kind of mismatch that risks a future header/path TOCTOU if any new entry point skips the rewrite. Drop the branch; the canonical ``/mcp/...`` path matches both parsers. Update the regression test to assert the new behavior. https://claude.ai/code/session_01SjyPmwfmrq8fveFgw9iHW9 * Fix MCP path auth target resolution Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix(mcp): require auth for refresh_token grants on delegate-auth servers `_mcp_oauth_user_api_key_auth` gates the unauthenticated PKCE flow for ``delegate_auth_to_upstream`` servers, but the bypass applied to BOTH ``/authorize`` and ``/token`` regardless of grant type. ``mcp_token`` accepts ``grant_type=refresh_token`` as well as ``authorization_code``, and ``exchange_token_with_server`` attaches the server's stored ``client_secret`` to whatever is forwarded upstream. An unauthenticated caller holding a refresh token issued to that OAuth client could mint fresh upstream access tokens through LiteLLM. Limit the anonymous bypass on ``/token`` to ``grant_type=authorization_code`` (the only grant PKCE actually protects via ``code_verifier``); fall through to normal LiteLLM auth for ``refresh_token`` and any other grant. ``/authorize`` continues to allow anonymous PKCE redirects. https://claude.ai/code/session_01SjyPmwfmrq8fveFgw9iHW9 * fix(ui): clear delegate_auth_to_upstream when switching off oauth2 The ``delegate_auth_to_upstream`` form field is rendered inside an ``isOAuth2 && (...)`` conditional, so the Form.Item unmounts when the user changes ``auth_type`` away from ``oauth2``. The follow-up ``form.setFieldValue("delegate_auth_to_upstream", false)`` runs after the field has already deregistered, so ``onFinish`` receives ``undefined`` and the fallback ``?? mcpServer.delegate_auth_to_upstream`` preserved the old ``true``. The flag then persisted in the database for a non-oauth2 server and silently re-activated if ``auth_type`` was later switched back to ``oauth2``. In the edit payload, force the flag to ``false`` whenever ``auth_type !== oauth2``; only trust the form value (and the existing DB fallback) when the server is actually oauth2. Backend defense-in-depth already ignores the flag for non-oauth2 servers, but the DB state should stay clean too. https://claude.ai/code/session_01SjyPmwfmrq8fveFgw9iHW9 * Fix MCP delegate auth reset on edit Co-authored-by: Yassin Kortam <yassin@berri.ai> --------- Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Yassin Kortam <yassin@berri.ai> Co-authored-by: Claude <claude@anthropic.com> |
||
|
|
8f3056c9c9
|
chore(proxy): block explicit-null user_id in ownership rebind guard
``model_dump(exclude_unset=True)`` in ``prepare_key_update_data`` includes any field the caller explicitly set, even when the value is ``None``. The previous guard short-circuited on ``getattr(data, 'user_id', None) is None``, which conflated "field omitted" (safe) with "field explicitly set to null" (writes NULL to the token row, detaching the key from its user and bypassing user-row role checks). Switch the omitted-vs-set distinction to ``data.model_fields_set``; treat explicit-null and explicit-empty-string identically as a removal attempt, both 403-rejected for non-admin callers. Parametrized regression adds ``explicit_null_blocked`` alongside the existing ``rebind_blocked`` / ``empty_blocked`` / ``same_user_id_allowed`` cases. |
||
|
|
d5d73f1d6a
|
chore(proxy): clarify ownership-rebind error message (actor vs target)
Previous wording read "User=<new_owner> is not allowed to update the key to belong to user=<current_owner>" — easy to misread as "caller wants to keep the key on its current owner". Reframe as "Non-admin caller is not allowed to rebind the key from user=<existing> to user=<incoming>" so the direction of the failed operation is unambiguous. Same shape preserved (HTTPException 403); only the ``detail`` string changes. Regression test substring updated. |
||
|
|
7738b03501
|
chore(proxy): close /key/regenerate ownership-rebind + premium-gate bypass
A non-admin caller could rebind their own key's ``user_id`` via ``/key/regenerate``. ``_execute_virtual_key_regeneration`` had org/team guards but no ``user_id`` guard, and ``prepare_key_update_data`` did not strip the field — it survived ``model_dump(exclude_unset=True)`` into the Prisma update. On the next request, ``_return_user_api_key_auth_obj`` resolved the rebound ``user_id`` against ``litellm_usertable`` and returned ``PROXY_ADMIN`` whenever the target row's ``user_role`` was admin (e.g. the default ``user_id="default_user_id"`` created on first password-UI login). ``/key/update`` had the equivalent guard inline at ``_validate_update_key_data``; extract it to a shared helper ``_validate_caller_can_change_key_ownership`` and call from both ``/key/update`` and ``_execute_virtual_key_regeneration``. Future regenerate-style endpoints inherit the guard for free. Also tighten the premium gate that allowed the master-key rotation branch to skip the enterprise check. The previous predicate was ``data.new_master_key is not None`` — a field-presence test, not an identity check. Any non-premium caller could send any value in that field and the premium check would no-op. Verify the caller actually holds the master key via ``_is_master_key`` before allowing the non-premium path. Tests: - ``test_regenerate_user_id_rebind_guard`` — parametrized table over cross-user rebind (blocked), empty-string removal (blocked), and same-user no-op rebind (allowed). - ``test_regenerate_premium_gate_requires_actual_master_key`` / ``test_regenerate_premium_gate_allows_actual_master_key_holder`` — ensure the premium check requires the caller actually present the master key, and that legitimate master-key rotation still works. |
||
|
|
63a2d1ddc9
|
fix(tests): use canonical litellm_enterprise import path (#27699)
The enterprise package is installed as `litellm_enterprise` (per enterprise/pyproject.toml), but several tests imported it as `enterprise.litellm_enterprise.*` — a path that only resolves because the repo root happens to sit on sys.path, letting Python's implicit namespace package machinery discover `enterprise/` as a directory. This breaks any test runner that relocates source (e.g. the mutation-testing workflow, which copies tests under `mutants/`) and also caused two `patch()` strings to target a module path that does not match what production code imports — meaning those mocks were never actually patching the production module's attribute. Replace `from enterprise.litellm_enterprise.` with the canonical `from litellm_enterprise.` across 6 test files, and fix two `patch()` target strings (and one `sys.modules` patch key in the SSO test) to match. |
||
|
|
9c4faeabc9
|
feat(ui): search teams by team ID alongside name (#27684)
* feat(ui): search teams by team ID alongside name The Teams page search box only matched team_alias, so pasting a team UUID returned zero results. Detect when the input is a full UUID and route it to the team_id filter instead; otherwise keep the existing alias substring search. Placeholder now reads "Search teams by name or ID...". Resolves LIT-2648 * refactor: search teams via backend OR clause, drop client-side UUID detection Adds a `search` query param to /v2/team/list that ORs across team_id (exact) and team_alias (case-insensitive contains), so the search box sends one param regardless of input format. Removes the isLikelyTeamId helper and the client-side branching it fed. |
||
|
|
aa9e7b9808
|
feat: litellm shin agent oss staging 05 10 2026 (#27631)
* fix: invalidate cached tag object on tag budget reset (#27481) (#27572) Squash-merged by litellm-agent from oss-agent-shin's PR. * chore(mcp): tighten stdio server registration paths (#27570) Squash-merged by litellm-agent from stuxf's PR. * fix(proxy): clear MCP OpenAPI mappings on server eviction; widen budget cache invalidation Evict OpenAPI tools from global_mcp_tool_registry and strip tool_name_to_mcp_server_name_mapping entries when a server leaves the runtime registry (remove_server and approval-status eviction). Invalidate user_api_key_cache for keys, orgs, and team members on budget-tier spend resets alongside tags. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(mcp): align update_server eviction with remove_server name fallback Document budget-reset test assertion flip (cross-pod cache staleness). Greptile: eviction now pops by server_id then server_name like remove_server; test docstring explains assert_not_awaited -> assert_any_await change. Co-authored-by: Cursor <cursoragent@cursor.com> * Fix org budget cache invalidation --------- Co-authored-by: oss-agent-shin <ext-agent-shin@berri.ai> Co-authored-by: stuxf <70670632+stuxf@users.noreply.github.com> Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
|
12e59c8798
|
Fix internal tag usage scoping (#27315)
* Scope internal tag usage to own keys Co-authored-by: ishaan-berri <ishaan-berri@users.noreply.github.com> * Add internal tag usage unowned key regression test Co-authored-by: ishaan-berri <ishaan-berri@users.noreply.github.com> * Handle empty internal tag usage scopes safely Co-authored-by: ishaan-berri <ishaan-berri@users.noreply.github.com> * Add tag activity database guard Co-authored-by: ishaan-berri <ishaan-berri@users.noreply.github.com> --------- Co-authored-by: oss-agent-shin <279349115+oss-agent-shin@users.noreply.github.com> Co-authored-by: ishaan-berri <ishaan-berri@users.noreply.github.com> |
||
|
|
aa587bd9d3
|
Merge pull request #27549 from BerriAI/shin_agent_oss_staging_05_09_2026
[litellm-agent] Staging → litellm_internal_staging (5/9/2026) |
||
|
|
9ed99037d2
|
Merge pull request #27422 from BerriAI/shin_agent_oss_staging_05_07_2026
[litellm-agent] Staging → litellm_internal_staging (5/7/2026) |
||
|
|
b834817785
|
[Feat] Add endpoint for bulk key updates for team (#26468)
Squash-merged by litellm-agent from Michael-RZ-Berri's PR. |
||
|
|
13a193367f
|
feat(sso): show full IdP claims in /sso/debug/callback (#27498)
* feat(sso): show full IdP claims in /sso/debug/callback The debug callback only displayed the proxy-parsed OpenID summary, so customers couldn't verify what custom claims (team_id, team_alias, roles, etc.) the IdP was actually returning. Render two new sections — Raw Claims (userinfo) and Access Token Claims (decoded JWT) — alongside the existing parsed view. Strip bearer tokens defense-in-depth in case a non-conforming IdP places them in its userinfo response. Resolves LIT-2838 * Update litellm/proxy/management_endpoints/ui_sso.py Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> * fix(sso): hoist json.dumps out of f-string for py3.10 ruff --------- Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> |
||
|
|
158b0c28c0
|
[litellm-agent] Staging → litellm_internal_staging (5/7/2026) (#27375)
Squash-merged by litellm-agent from oss-pr-review-agent-shin[bot]'s PR. |
||
|
|
d90cf56245
|
Fix SCIM user lookup filters (#27308)
* Fix SCIM Okta userName lookup Co-authored-by: ishaan-berri <ishaan-berri@users.noreply.github.com> * fix scim user filter typing Co-authored-by: ishaan-berri <ishaan-berri@users.noreply.github.com> --------- Co-authored-by: oss-agent-shin <279349115+oss-agent-shin@users.noreply.github.com> Co-authored-by: ishaan-berri <ishaan-berri@users.noreply.github.com> |
||
|
|
bd1ea0252a
|
perf(proxy): run daily activity aggregation off the event loop (#27264)
Some checks are pending
Unit Tests: Proxy DB Operations / custom-logging (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / logging-misc (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / assert-shard-coverage (push) Waiting to run
Unit Tests: Proxy DB Operations / auth-checks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / budgets (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / db-and-spend (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / endpoints-and-responses (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / guardrails-hooks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / jwt-and-keys (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / key-generation (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-runtime (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-server-core (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / schema-migration (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-utils (push) Blocked by required conditions
Unit Tests: Security / security (push) Waiting to run
Unit Tests: Caching (Redis) / caching-redis (push) Waiting to run
Co-authored-by: Yassin Kortam <yassinkortam@g.ucla.edu> |
||
|
|
73de892654
|
fix: replace user api key auth with authorization or cookie for mcp server creation (#27190)
* fix: replace user api key auth with authorization or cookie for mcp server creation * updated tests |
||
|
|
be5f217aaf
|
Merge pull request #26861 from BerriAI/litellm_fix_scim_virtual_key_deactivation
fix(scim): revoke virtual keys when SCIM deprovisions a user |
||
|
|
a56256e5ee | feat: routing groups ui | ||
|
|
2aa4301fe7
|
Merge pull request #27027 from stuxf/fix/mcp-server-url-redact-non-admin
fix(proxy): redact MCP server URL and headers for non-admin viewers (VERIA-8) |
||
|
|
e1083b1353
|
Merge pull request #27014 from stuxf/fix/activity-endpoint-tenant-scoping
fix(proxy): scope team and agent activity endpoints per-entity (VERIA-43) |
||
|
|
e995156462
|
Merge pull request #26953 from stuxf/chore/audit-log-cache-vault-settings
chore(audit): audit-log /cache/settings + /config_overrides/hashicorp_vault mutations |
||
|
|
c064170a18
|
Merge pull request #26819 from stuxf/fix/team-callback-idor
chore(team): require team-management role on /team/{id}/callback endpoints
|
||
|
|
31d0d8599f | fix(mcp): redact command fields for non-admin server views | ||
|
|
8cf3bf11bd
|
fix(scim): preserve scim_active on PUT when client omits the field
Some checks failed
Unit Tests: Caching (Redis) / caching-redis (push) Has been cancelled
Unit Tests: Proxy DB Operations / assert-shard-coverage (push) Has been cancelled
Unit Tests: Security / security (push) Has been cancelled
Unit Tests: Proxy DB Operations / auth-checks (push) Has been cancelled
Unit Tests: Proxy DB Operations / budgets (push) Has been cancelled
Unit Tests: Proxy DB Operations / custom-logging (push) Has been cancelled
Unit Tests: Proxy DB Operations / db-and-spend (push) Has been cancelled
Unit Tests: Proxy DB Operations / endpoints-and-responses (push) Has been cancelled
Unit Tests: Proxy DB Operations / guardrails-hooks (push) Has been cancelled
Unit Tests: Proxy DB Operations / jwt-and-keys (push) Has been cancelled
Unit Tests: Proxy DB Operations / key-generation (push) Has been cancelled
Unit Tests: Proxy DB Operations / logging-misc (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-runtime (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-server-core (push) Has been cancelled
Unit Tests: Proxy DB Operations / schema-migration (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-utils (push) Has been cancelled
A SCIM PUT may legally omit `active` (full-replace with the field absent). Pydantic fills the SCIMUser.active default of True, so the PUT handler was overwriting metadata.scim_active with True even when the client never sent it — silently reactivating a previously SCIM-blocked user and unblocking their keys. Use model_fields_set to detect whether the client actually sent `active`. If omitted, preserve the prior scim_active value and skip the cascade to virtual keys. Also drop comments added in this PR that just narrate what the code does; keep only the docstrings and the SQL-NULL pitfall note that explain non-obvious behaviour. Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> |