- team-admin: assert Admin Settings is not visible (role-specific check)
- proxy-admin: use users[Role.ProxyAdmin].password from constants instead of duplicating the env var fallback inline
Pin all cosign public key references to the immutable commit hash
(0112e53) that first introduced the key, instead of fetching it from
the release tag. This addresses the concern that an attacker with push
access could replace the key on main/tags and re-sign tampered images.
Docs now show two verification methods: commit hash (recommended) and
release tag (convenience), with explanation of why the hash is stronger.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: batch-limit stale managed object cleanup to prevent 300K row UPDATE (#25257)
* Add STALE_OBJECT_CLEANUP_BATCH_SIZE constant
Configurable batch limit (default 1000) for stale managed object cleanup,
preventing unbounded UPDATE queries from hitting 300K+ rows at once.
* Batch-limit stale managed object cleanup with single bounded SQL query
Two fixes to _cleanup_stale_managed_objects:
1. Replace unbounded update_many with a single execute_raw using a
subquery LIMIT, capping each poll cycle to STALE_OBJECT_CLEANUP_BATCH_SIZE
rows. Zero rows loaded into Python memory — everything stays in Postgres.
Uses the same PostgreSQL raw-SQL pattern as spend_log_cleanup.py
(the proxy requires PostgreSQL per schema.prisma).
2. Extract _expire_stale_rows as a separate method for testability.
Keeps the file_purpose='response' filter to avoid incorrectly expiring
long-running batch or fine-tune jobs that legitimately exceed the
staleness cutoff.
* docs: add STALE_OBJECT_CLEANUP_BATCH_SIZE to env vars reference
* test: remove deprecated embed-english-v2.0 cohere embedding tests
Adds a new endpoint to bulk-update team_member_permissions across
teams. Supports apply_to_all_teams (with cursor-based pagination)
or a specific list of team_ids. Merges new permissions into each
team's existing set rather than overwriting.
Also fixes test isolation bug in test_get_prompt_info_by_base_id
where leaked prisma_client state from other tests caused a
TypeError on await.
* Remove redundant matrix unit test workflow
All test paths in test-litellm-matrix.yml are fully covered by the
newer semantic unit test workflows (test-unit-*.yml), making the
matrix workflow redundant CI spend.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Add Codecov coverage reporting to semantic unit test workflows
Add coverage collection (--cov) and Codecov OIDC upload to both
reusable base workflows and all 12 caller workflows, replacing the
coverage reporting that was previously only in the matrix workflow.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Move id-token/pull-requests permissions to job level for multi-job workflows
For workflows with multiple jobs (llm-providers, proxy-db), move
id-token: write and pull-requests: write from workflow level to job
level so permissions are scoped to only the jobs that need them.
Removes zizmor inline suppressions that were masking the issue.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* [Docs] Enforce Black formatting in contributor docs
Black formatting is now enforced in CI. Update CLAUDE.md, AGENTS.md,
and CONTRIBUTING.md to instruct contributors and AI agents to run
`poetry run black .` before committing, and add VS Code setup guidance.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: fixes
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The proxy_e2e_azure_batches_tests workflow is consistently flaky and
does not provide reliable signal on whether changes break anything.
Remove the workflow from both CircleCI and GitHub Actions, along with
the test directory it exclusively used.
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: multiple concurrent budget windows per API key and team (#24883)
* feat(proxy): add BudgetLimitEntry type and wire budget_limits into key/team models
* feat(schema): add budget_limits Json column to VerificationToken and TeamTable
* feat(migrations): add migration for budget_limits column on keys and teams
* feat(keys): initialize budget_limits windows with reset_at on key create/update
* feat(teams): initialize budget_limits windows with reset_at on team create/update
* feat(auth): add _virtual_key_multi_budget_check and _team_multi_budget_check
* feat(auth): call multi-budget checks from common_checks for keys and teams
* feat(proxy): increment per-window Redis spend counters after each request
* feat(budget): reset individual budget windows on schedule via reset_budget_job
* feat(ui): add hourly option to BudgetDurationDropdown
* feat(ui): add budget_limits field to KeyResponse type
* feat(ui): add Budget Windows editor to key edit view
* feat(ui): add Budget Windows editor to create key form
* fix(proxy): strip budget_limits=None before Prisma upsert to fix login 500
Prisma rejects nullable JSON fields (Json? without @default) when passed as
Python None — it needs the field omitted entirely so the DB stores NULL via
the column's nullable constraint. This was breaking /v2/login because the UI
session key creation path hit the upsert with budget_limits=None.
* ui(key-edit): use antd InputNumber+Button for budget windows, add reset hints
* ui(create-key): use antd InputNumber+Button for budget windows, add reset hints
* docs(users): add multiple budget windows section with API + dashboard walkthrough
* fix: BudgetExceededError returns HTTP 429 instead of 400
- Add status_code=429 to BudgetExceededError class
- auth_exception_handler hardcoded code=400 → code=429
* fix: no-op else branch in multi-budget auth checks causes KeyError
- BudgetLimitEntry objects must be coerced via model_dump() not left as-is
- Move _virtual_key_multi_budget_check into common_checks (was asymmetric
with _team_multi_budget_check which already lived there)
* fix: len() on JSON string returns char count not window count
Guard with isinstance check + json.loads() before iterating per-window
Redis counters in increment_spend_counters
* fix: silent except:pass hides Redis reset failures in reset_budget_windows
Log Redis counter reset failures as warnings so they are observable
* test: add unit tests for multi-budget window enforcement
5 tests covering: no budget_limits passes, under budget passes,
over hourly window raises 429, over monthly window raises 429,
BudgetLimitEntry objects coerced without KeyError
* fix: key per-window counters stable across reorders (duration key, not index)
* fix: team+key per-window spend increments use duration key, not index
* fix: budget window reset uses duration key; log failures instead of swallowing
* refactor: extract BudgetWindowsEditor to shared component
* refactor: key_edit_view imports BudgetWindowsEditor from shared component
* refactor: create_key_button imports BudgetWindowsEditor from shared component
---------
Co-authored-by: Ishaan Jaffer <ishaanjaffer0324@gmail.com>
* fix(reset_budget_job): extract _reset_expired_window helper to fix PLR0915 too many statements
* feat(skills): Skills Registry & Hub — register skills, browse in AI Hub, public skill hub (#25118)
* feat(skills): add domain and namespace fields to plugin types
* feat(skills): store and return domain/namespace inside manifest_json
* feat(skills): add /public/skill_hub endpoint for unauthenticated access
* feat(skills): whitelist /public/skill_hub from auth requirements
* feat(skills): add domain, namespace to Plugin and RegisterPluginRequest types
* feat(skills): smart URL parser — paste github URL, auto-detect source type and name
* feat(skills): replace enable toggle with Public badge, make rows clickable
* feat(skills): add skill detail view with Overview and How to Use tabs
* feat(skills): add MakeSkillPublicForm modal for publishing skills to the hub
* feat(skills): rename panel to Skills, wire in skill detail view on row click
* feat(skills): add skill hub table columns — name, description, domain, source, status
* feat(skills): add SkillHubDashboard with stats row, domain dropdown filter, and table
* feat(skills): add Skill Hub tab to AI Hub with Select Skills to Make Public button
* feat(skills): move Skills to top-level nav item directly under MCP Servers
* feat(skills): add skillHubPublicCall and NEXT_PUBLIC_BASE_URL support
* feat(skills): add Skill Hub tab to public AI Hub page
* feat(skills): add skills page routing in main app router
* feat(skills): add /skills page route
* chore: update package-lock after npm install
* docs(skills): add Skills Gateway doc page with mermaid architecture diagram
* docs(skills): add Skills Gateway to sidebar under Agent & MCP Gateway
* docs(skills): add loom walkthrough video to Skills Gateway doc
* chore: fixes
---------
Co-authored-by: Ishaan Jaffer <ishaanjaffer0324@gmail.com>
Co-authored-by: Yuneng Jiang <yuneng@berri.ai>
* fix(bedrock): strip [1m]/[200k] context window suffixes before cost lookup
* test(bedrock): add test for [1m] context window suffix stripping in cost lookup
* schema: add allowed_models to BudgetTable, default_team_member_models to TeamTable
* migration: add allowed_models and default_team_member_models columns
* types: add allowed_models to TeamMemberAddRequest, TeamMemberUpdateRequest, UpdateTeamRequest
* utils: add allowed_models param to add_new_member, persist to budget table
* common_utils: add allowed_models to _upsert_budget_and_membership
* team endpoints: seed allowed_models on member_add, persist on member_update and team/update
* auth: enforce per-member allowed_models at request time
* networking: add allowed_models to Member type and teamMemberUpdateCall
* TeamMemberTab: add Model Scope column showing per-member allowed_models
* EditMembership: add Allowed Models multi-select field
* TeamInfo: add default_team_member_models field in Settings tab
* chore: sync schema.prisma copies from root
* fix(team_member_update): update existing budget in-place instead of creating new one
When a member already has a budget_id, patch only the fields the caller
provided rather than always creating a fresh budget record. The old
code ignored existing_budget_id entirely, so updating only allowed_models
silently dropped the stored max_budget / tpm_limit / rpm_limit values.
* fix(auth): pass llm_router to _check_team_member_model_access
Without the router, _can_object_call_model cannot resolve wildcard model
names (e.g. openai/*) or access-group names in allowed_models, causing
legitimate requests to be denied. Thread the existing llm_router from
_run_common_checks through to the new member-scope check.
* feat(ui): add Team Member Settings accordion to Create Team modal
Groups default_team_member_models, member budget/key duration, and
tpm/rpm defaults into a single collapsible section. The model picker
is filtered to only show the models selected for the team, and the
copy distinguishes it from the team-level Models field.
* feat(ui): consolidate Team Member Settings into accordion in edit team form
Moves default_team_member_models + per-member budget/key/tpm/rpm fields
into a collapsible "Team Member Settings" panel. Keeps the top-level
form focused on team-wide settings (team models, team budget, tpm/rpm).
* fix(ui): use tremor Accordion for Team Member Settings in edit team form
* fix(ui): move Team Member Settings accordion above budget fields in Create Team
* chore: fixes
---------
Co-authored-by: Ishaan Jaffer <ishaanjaffer0324@gmail.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Yuneng Jiang <yuneng@berri.ai>
* Add STALE_OBJECT_CLEANUP_BATCH_SIZE constant
Configurable batch limit (default 1000) for stale managed object cleanup,
preventing unbounded UPDATE queries from hitting 300K+ rows at once.
* Batch-limit stale managed object cleanup with single bounded SQL query
Two fixes to _cleanup_stale_managed_objects:
1. Replace unbounded update_many with a single execute_raw using a
subquery LIMIT, capping each poll cycle to STALE_OBJECT_CLEANUP_BATCH_SIZE
rows. Zero rows loaded into Python memory — everything stays in Postgres.
Uses the same PostgreSQL raw-SQL pattern as spend_log_cleanup.py
(the proxy requires PostgreSQL per schema.prisma).
2. Extract _expire_stale_rows as a separate method for testability.
Keeps the file_purpose='response' filter to avoid incorrectly expiring
long-running batch or fine-tune jobs that legitimately exceed the
staleness cutoff.
Unit Tests: Proxy DB Operations / proxy-db (auth-checks, tests/proxy_unit_tests/test_auth_checks.py tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-db (remaining, tests/proxy_unit_tests --ignore=tests/proxy_unit_tests/test_key_generate_prisma.py --ignore=tests/proxy_unit_tests/test_auth_checks.py --ignore=tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Has been cancelled
* fix(ui): resolve login redirect loop when reverse proxy adds HttpOnly to cookies
When LiteLLM is behind nginx-ingress or similar with security-hardened
configs, the reverse proxy adds HttpOnly to all Set-Cookie headers. This
makes the JWT token unreadable by JavaScript, causing an infinite login
redirect loop. Fix by returning the JWT token in the /v2/login response
body so the frontend can set a JS-accessible cookie directly.
Fixes#19663
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: address Greptile review feedback
- Add window guard to setTokenCookie for SSR consistency with clearTokenCookies
- Add SSR test for window undefined case
- Add code comment explaining why JWT is included in response body
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: address second round of Greptile review feedback
- Add loginCall integration tests verifying setTokenCookie is called with
token and skipped when absent (backward-compatibility path)
- Use encodeURIComponent/decodeURIComponent in setTokenCookie/getCookie
for defense-in-depth against non-standard token formats
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Update ui/litellm-dashboard/src/utils/cookieUtils.ts
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* Update ui/litellm-dashboard/src/utils/cookieUtils.ts
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* fix(ui): use sessionStorage instead of cookie for login token storage
Replace setTokenCookie (which is a no-op when reverse proxy adds HttpOnly)
with storeLoginToken using sessionStorage. Add sessionStorage fallback to
getCookie so the token is found even when the cookie is HttpOnly. Also handle
'=' in cookie values with .slice(1).join("=") and clear sessionStorage on
logout.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(ui): use shared getCookie in page.tsx and user_dashboard.tsx
Replace local getCookie functions in page.tsx and user_dashboard.tsx
with the shared one from cookieUtils that has the sessionStorage
fallback. Without this, the HttpOnly cookie fix was incomplete —
page.tsx (the dashboard entry point) could not read the token,
causing the redirect loop to persist.
Also scope the sessionStorage fallback to the "token" key only,
and clear sessionStorage in page.tsx deleteCookie.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(ui): scope deleteCookie sessionStorage cleanup to token key only
Also document the sessionStorage cross-tab trade-off: per-tab scope
means users behind an HttpOnly proxy must log in once per tab, but
this is intentional to avoid localStorage XSS exposure.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Update ui/litellm-dashboard/src/utils/cookieUtils.ts
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* Apply suggestion from @greptile-apps[bot]
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* style: remove stray double blank line in user_dashboard.tsx
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(ui): guard storeLoginToken against empty/whitespace-only tokens
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(ui): preserve sessionStorage token across beforeunload clear
The existing beforeunload handler calls sessionStorage.clear() to
flush cached UI data on page refresh. This also wiped the token
stored by storeLoginToken, re-introducing the redirect loop after
any page refresh in the HttpOnly proxy scenario. Now the token is
saved and restored across the clear.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(ui): set JS-accessible cookie at /ui path as HttpOnly workaround
sessionStorage alone is unreliable. Also set the token via
document.cookie at path=/ui — nginx only adds HttpOnly to server-set
Set-Cookie headers, so a JS-set cookie is always readable.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(ui): use dynamic cookie path based on server_root_path
Hardcoded path=/ui breaks when LiteLLM is deployed with a custom
server_root_path. Now derives the cookie path from serverRootPath
so it works at /ui, /myapp/ui, etc.
Also reuse clearTokenCookies() in deleteCookie() to avoid duplication.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor(ui): remove circular dependency in cookieUtils.ts
Derive the UI cookie path from window.location.pathname instead of
importing serverRootPath from networking.tsx. This breaks the
cookieUtils → networking → cookieUtils cycle that could cause
serverRootPath to be undefined under certain bundler configurations.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(ui): harden getUiCookiePath regex and add missing tests
- Use regex /\/ui(?=\/|$)/ to match "/ui" only as a full path segment,
preventing false matches on paths like "/my-ui-tool/login".
- Add unit tests for storeLoginToken empty/whitespace guard and
cookie-at-/ui-path behavior.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* style: fix Black formatting in audit_logs.py
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix CI: formatting, test params, remove token from login JSON
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: reformat with Black 23.x to match CI
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: keep token in login JSON body for UI storeLoginToken flow
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: use storeLoginToken in exchangeLoginCode, add credentials include
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* revert: remove unrelated changes from HttpOnly cookie fix branch
Reset files not related to the login cookie fix back to main:
- prometheus.py, bedrock converse, guardrail handler
- auth_checks.py, reset_budget_job.py, audit_logs.py
- test_user_api_key_auth.py
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Revert "revert: remove unrelated changes from HttpOnly cookie fix branch"
This reverts commit 0684a1e275.
* Revert "fix: use storeLoginToken in exchangeLoginCode, add credentials include"
This reverts commit 866405f443.
* Revert "fix: keep token in login JSON body for UI storeLoginToken flow"
This reverts commit 086c41640c.
* Revert "fix: reformat with Black 23.x to match CI"
This reverts commit b2c3334c88.
* Revert "fix CI: formatting, test params, remove token from login JSON"
This reverts commit 2905d47bd4.
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Unit Tests: Proxy DB Operations / proxy-db (auth-checks, tests/proxy_unit_tests/test_auth_checks.py tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Waiting to run
Unit Tests: Proxy DB Operations / proxy-db (remaining, tests/proxy_unit_tests --ignore=tests/proxy_unit_tests/test_key_generate_prisma.py --ignore=tests/proxy_unit_tests/test_auth_checks.py --ignore=tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Waiting to run
* docs(blog): add cosign Docker image verification instructions
Add steps for verifying Docker images with cosign to three security blog posts:
CI/CD v2, Security Townhall, and Security Update.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs(proxy): add cosign verification to Docker/Helm/Terraform deploy page
Add image signature verification steps to the main deployment doc so
users pulling Docker images know how to verify them with cosign.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: fixes
* Update index.md
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* [Docs] Scope cosign signing docs to GHCR and specify starting version
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* [Docs] Add starting version callout to ci_cd_v2 blog post
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Krrish Dholakia <krrish+github@berri.ai>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Unit Tests: Proxy DB Operations / proxy-db (auth-checks, tests/proxy_unit_tests/test_auth_checks.py tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-db (remaining, tests/proxy_unit_tests --ignore=tests/proxy_unit_tests/test_key_generate_prisma.py --ignore=tests/proxy_unit_tests/test_auth_checks.py --ignore=tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-db (auth-checks, tests/proxy_unit_tests/test_auth_checks.py tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-db (remaining, tests/proxy_unit_tests --ignore=tests/proxy_unit_tests/test_key_generate_prisma.py --ignore=tests/proxy_unit_tests/test_auth_checks.py --ignore=tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-db (auth-checks, tests/proxy_unit_tests/test_auth_checks.py tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-db (remaining, tests/proxy_unit_tests --ignore=tests/proxy_unit_tests/test_key_generate_prisma.py --ignore=tests/proxy_unit_tests/test_auth_checks.py --ignore=tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-db (auth-checks, tests/proxy_unit_tests/test_auth_checks.py tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-db (remaining, tests/proxy_unit_tests --ignore=tests/proxy_unit_tests/test_key_generate_prisma.py --ignore=tests/proxy_unit_tests/test_auth_checks.py --ignore=tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Has been cancelled
Extend LATENCY_BUCKETS beyond 5 minutes so request/LLM latency metrics
can distinguish long runs up to the typical default LLM request timeout.
Made-with: Cursor
* fix(presidio): use correct text positions in anonymize_text (#24160)
The Presidio anonymizer endpoint returns items with start/end positions
that reference the *anonymized output* text, not the original input.
anonymize_text() was applying these positions to the original text,
causing garbled output with remnants of un-masked PII data.
When output_parse_pii is False, return redacted_text["text"] directly
from the anonymizer response instead of manually splicing.
When output_parse_pii is True, use analyze_results positions (which
correctly reference the original text) to build numbered replacement
tokens and the pii_tokens mapping.
* address review: remove dead code, fix token numbering order
- Remove unused `anon_item_by_entity` dict (Greptile P2)
- Number tokens left-to-right (<PERSON_1> first in text, not last)
- Add assertion for token numbering order in test
## Problem
When `get_cache_key(**kwargs)` is called with kwargs that already
contains `preset_cache_key` (which can happen when cache key is
recomputed in certain code paths), the call to
`_set_preset_cache_key_in_kwargs()` fails with:
```
TypeError: _set_preset_cache_key_in_kwargs() got multiple values
for keyword argument 'preset_cache_key'
```
This is because `preset_cache_key` is passed both explicitly:
```python
self._set_preset_cache_key_in_kwargs(
preset_cache_key=hashed_cache_key, **kwargs
)
```
And implicitly via `**kwargs` unpacking when `kwargs["preset_cache_key"]`
exists.
## Solution
Filter out `preset_cache_key` from kwargs before passing to
`_set_preset_cache_key_in_kwargs()`:
```python
kwargs_for_preset = {k: v for k, v in kwargs.items() if k != "preset_cache_key"}
self._set_preset_cache_key_in_kwargs(
preset_cache_key=hashed_cache_key, **kwargs_for_preset
)
```
## Testing
Added unit tests covering:
- kwargs with existing preset_cache_key (the bug case)
- kwargs without preset_cache_key (regression test)
- Verification that preset_cache_key is correctly set in litellm_params
Fixes#25081.
is_tool_name_prefixed() checked for the presence of MCP_TOOL_PREFIX_SEPARATOR
(default '-') anywhere in the tool name. Any non-MCP tool whose name
contains a hyphen (e.g. 'text-to-speech', 'code-review') was silently
misclassified as an MCP-prefixed tool. When the semantic tool filter is
enabled, these tools would be routed through semantic matching and
potentially dropped.
Fix: accept an optional known_server_prefixes set. When supplied, the
function extracts the candidate prefix (text before the first separator)
and checks it against the normalised set of registered server prefixes.
Only a genuine match returns True. Without the set, legacy behaviour is
preserved for backward compatibility.
Updated _get_mcp_server_from_tool_name() to build the prefix set from
the live registry and pass it through.
9 new tests.
Co-authored-by: d 🔹 <258577966+voidborne-d@users.noreply.github.com>
Unit Tests: Proxy DB Operations / proxy-db (auth-checks, tests/proxy_unit_tests/test_auth_checks.py tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Waiting to run
Unit Tests: Proxy DB Operations / proxy-db (remaining, tests/proxy_unit_tests --ignore=tests/proxy_unit_tests/test_key_generate_prisma.py --ignore=tests/proxy_unit_tests/test_auth_checks.py --ignore=tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Waiting to run
* bump litellm-proxy-extras version to 0.4.65
* bump litellm-proxy-extras==0.4.65 in pyproject.toml
* bump litellm-proxy-extras==0.4.65 in requirements.txt
* added support for metadata (#24261)
* added support for metadata
* fix: PR review - meta truthiness, BlobResourceContents mimeType, add Blob+empty meta tests
Made-with: Cursor
* pyproject to .25
* feat(teams): resolve access group models/MCPs/agents in team endpoints
Add access_group_models, access_group_mcp_server_ids, and
access_group_agent_ids to /team/info and /v2/team/list responses.
These fields contain resources inherited from access groups, kept
separate from direct assignments so the UI can distinguish the source.
Backend: _resolve_access_group_resources() helper resolves access
group resources via existing _get_*_from_access_groups() functions.
UI: Teams table and detail view show direct models as blue badges
and access-group-sourced models as green badges.
* perf(teams): single-pass access group resolution + asyncio.gather in list endpoint
- Fetch each access group object once and extract all 3 resource fields
in a single pass instead of 3 separate calls (3N → N lookups)
- Use asyncio.gather to resolve access groups across teams concurrently
in list_team_v2 instead of sequential awaits
- Add 5 unit tests for _resolve_access_group_resources
* docs: add default_team_params to config reference and update examples
- Add default_team_params to litellm_settings reference table in
config_settings.md with all sub-fields documented
- Update self_serve.md and msft_sso.md examples to include
team_member_permissions, tpm_limit, and rpm_limit
- Fix misleading comment that implied default_team_params only applies
to SSO auto-created teams — it applies to all /team/new calls
* docs: clarify that models sub-field only applies to SSO auto-created teams
* fix: lazy import get_access_object to break cyclic import + short-circuit all-proxy-models display
- Remove get_access_object from module-level import in team_endpoints.py
and use a lazy _get_access_object wrapper to avoid cyclic dependency
- Add _prisma_client is None early-exit guard in _resolve_access_group_resources
- Short-circuit UI to show "All Proxy Models" when team.models is empty
or contains "all-proxy-models", skipping access group model resolution
* add: making organizations a select instead of read only badges
* fix(ui): only send organization_id when changed and use raw initial value
* fix(ui): add paginated team search to usage page filter
Replace the static team dropdown on the usage page with a new
TeamMultiSelect component that uses the paginated v2/team/list
endpoint with debounced server-side search and infinite scroll.
* fix(ui): fix imports and update placeholder for team multi select
* fix(ui): wire team_id filter to key alias dropdown on Virtual Keys tab
The Key Alias dropdown on the Virtual Keys page was showing aliases from
all teams regardless of which team was selected. The team_id was never
passed through the frontend chain to the backend /key/aliases endpoint.
- Backend: add optional team_id query param to /key/aliases endpoint
- networking.tsx: add team_id param to keyAliasesCall
- useKeyAliases: accept and forward team_id to API call and query key
- filter.tsx: pass allFilters context to custom filter components
- PaginatedKeyAliasSelect: read Team ID from allFilters and pass to hook
* fix(tests): correct mock targets in TestResolveAccessGroupResources
Three tests were patching the non-existent `get_access_object` instead
of `_get_access_object` (the lazy-import wrapper), causing AttributeError.
Also added missing `prisma_client` mock so tests get past the early-exit
guard and actually exercise the resolution logic.
* fix: use direct attribute access with or [] fallback in _resolve_access_group_resources
Replace getattr(ag, "field", []) with ag.field or [] for cleaner
access and safe handling if a field is None.
* fix(ui): remove model source legend from team detail view
The blue/green color distinction is self-explanatory; the legend added
visual clutter without providing enough value.
* fix(ui): add missing access_group fields to TeamData.team_info type
The TeamData interface was missing access_group_models,
access_group_mcp_server_ids, and access_group_agent_ids fields,
causing a TypeScript build failure.
* perf(teams): batch-fetch access groups in single DB query
Replace per-ID _resolve_access_group_resources loop with a single
find_many call that deduplicates IDs across all teams. Removes the
N+1 query pattern on cold cache for the team list endpoint.
* refactor(proxy): extract helpers to fix PLR0915 violations
Extract `_apply_non_admin_alias_scope` from `key_aliases`,
`_resolve_team_access_group_resources` from `team_info`, and
`_enforce_list_team_v2_access` from `list_team_v2` to bring each
function under ruff's 50-statement limit. No behavior changes.
* test(ui): update tests to match new team_id / access-group signatures
- useKeyAliases, PaginatedKeyAliasSelect: add trailing `undefined` to
spy matchers for the new `team_id` param on `useInfiniteKeyAliases`
and `keyAliasesCall`.
- EntityUsage: mock new `TeamMultiSelect` child so QueryClientProvider
is not required for team-entity tests.
- ModelsCell: replace the overflow-accordion test with one that
verifies the new collapse-on-`all-proxy-models` behavior (no
accordion, single badge).
* fix(ui): send null (not '') for cleared organization_id on team update
AntD <Select allowClear> returns undefined when the user clears the
selection. Coalescing to "" caused the team-update payload to carry
organization_id: "" instead of null, relying on the backend to coerce
it. Send null directly so the intent is explicit at the source.
* poetry
* chore: regen poetry.lock for litellm-proxy-extras 0.4.64 bump
* chore: update Next.js build artifacts (2026-04-04 17:55 UTC, node v22.16.0)
---------
Co-authored-by: shivam <shivam@uni.minerva.edu>
Co-authored-by: Ryan Crabbe <ryan@berri.ai>
Co-authored-by: yuneng-jiang <yuneng@berri.ai>
* Tag query fix (#25094)
* feat(tag-spend): implement separate scheduler job for daily tag spend updates
* fix(docker): add g++ to build dependencies in Dockerfile
* initial test cases. TODO: check scheduler init and test cases in proxy_server related to it
* resolved QPS issue when redis transaction buffer is enabled
* resolving circular import error flagged by greptile
* fix(mypy): use Optional[str] for api_base in PydanticAI provider to match superclass signature
---------
Co-authored-by: Shivam Rawat <shivam@berri.ai>
Co-authored-by: shivam <shivam@uni.minerva.edu>
Co-authored-by: Ryan Crabbe <ryan@berri.ai>
Co-authored-by: yuneng-jiang <yuneng@berri.ai>
Co-authored-by: Harish <harishgokul01@gmail.com>
Co-authored-by: Ishaan Jaffer <ishaan@berri.ai>
Unit Tests: Proxy DB Operations / proxy-db (auth-checks, tests/proxy_unit_tests/test_auth_checks.py tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-db (remaining, tests/proxy_unit_tests --ignore=tests/proxy_unit_tests/test_key_generate_prisma.py --ignore=tests/proxy_unit_tests/test_auth_checks.py --ignore=tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Has been cancelled
The test was hitting "No QueryClient set" because TeamGuardrailsTab
(which pulls in useRegisterGuardrail) was not mocked alongside the
other tab children. Added a mock.
Also: the "+ Add New Guardrail" assertion was silently relying on
Tremor Tabs rendering all panels at once. antd Tabs only renders
the active tab's content, and defaultActiveKey is "submitted", so
the button in the "Guardrails" tab wasn't in the DOM. Clicking the
Guardrails tab first before asserting.
* Litellm ishaan march23 - MCP Toolsets + GCP Caching fix (#25146)
* feat(mcp): MCP Toolsets — curated tool subsets from one or more MCP servers (#24335)
* feat(mcp): add LiteLLM_MCPToolsetTable and mcp_toolsets to ObjectPermissionTable
* feat(mcp): add prisma migration for MCPToolset table
* feat(mcp): add MCPToolset Python types
* feat(mcp): add toolset_db.py with CRUD helpers for MCPToolset
* feat(mcp): add toolset CRUD endpoints to mcp_management_endpoints
* fix(mcp): skip allow_all_keys servers when explicit mcp_servers permission is set (toolset scope fix)
* feat(mcp): add _apply_toolset_scope and toolset route handling in server.py
* fix(mcp): resolve toolset names in responses API before fetching tools
* feat(mcp): add mcp_toolsets field to LiteLLM_ObjectPermissionTable type
* feat(mcp): register LiteLLM_MCPToolsetTable in prisma client initialization
* feat(mcp): validate mcp_toolsets in key-vs-team permission check
* feat(mcp): register toolset routes in proxy_server.py
* feat(mcp): add MCPToolset and MCPToolsetTool TypeScript types
* feat(mcp): add fetchMCPToolsets, createMCPToolset, updateMCPToolset, deleteMCPToolset API functions
* feat(mcp): add useMCPToolsets React Query hook
* feat(mcp): add toolsets (purple) as third option type in MCPServerSelector
* feat(mcp): extract toolsets from combined MCP field in key form
* feat(mcp): extract toolsets from combined MCP field in team form
* feat(mcp): show toolsets section in MCPServerPermissions read view
* feat(mcp): pass mcp_toolsets through object_permissions_view
* feat(mcp): add MCPToolsetsTab component for creating and managing toolsets
* feat(mcp): add Toolsets tab to mcp_servers.tsx
* feat(mcp): pass mcpToolsets to playground chat and responses API calls
* feat(mcp): generate correct server_url for toolsets in playground API calls
* docs(mcp): add MCP Toolsets documentation
* docs(mcp): add mcp_toolsets to sidebar
* fix(mcp): replace x-mcp-toolset-id header with ContextVar to prevent client forgery
* fix(mcp): use ContextVar + StreamingResponse for toolset MCP routes (fixes SSE streaming)
* fix(mcp): cache toolset permission lookups to avoid per-request DB calls
* test(mcp): add tests for toolset scope enforcement, ContextVar isolation, and access control
* fix(mcp): cache toolset name lookups in MCPServerManager to avoid per-request DB calls
* fix(mcp): prevent body_iter deadlock + use cached toolset lookup in responses API
- _stream_mcp_asgi_response: add done callback to handler_task that puts
the EOF sentinel on body_queue when the task exits, preventing body_iter
from hanging forever if the handler raises after headers are sent.
- litellm_proxy_mcp_handler: replace raw get_mcp_toolset_by_name() DB call
with global_mcp_server_manager.get_toolset_by_name_cached() so toolset
resolution uses the 60s TTL cache added for this purpose instead of
hitting the DB on every responses-API request.
* fix(mcp): toolset access control, asyncio fix, and real unit tests
- server.py: _apply_toolset_scope now enforces that non-admin keys must
have the requested toolset_id in their mcp_toolsets grant list;
admin keys always bypass the check.
- mcp_management_endpoints.py: three access-control fixes:
* fetch_mcp_toolsets: non-admin keys with mcp_toolsets=None now
return [] instead of all toolsets (only admins get 'all' when
the field is absent)
* fetch_mcp_toolset: non-admin keys that haven't been granted the
requested toolset_id now get 403 instead of the full result
* add_mcp_toolset: duplicate toolset_name now returns 409 Conflict
instead of an opaque 500
- proxy_server.py: use asyncio.get_running_loop() instead of
get_event_loop() inside an already-running coroutine (Python 3.10+).
- test_mcp_toolset_scope.py: replace four hollow tests that only
asserted local variable properties with real tests that call the
production fetch_mcp_toolsets() and handle_streamable_http_mcp()
functions with mocked dependencies.
* fix(mcp): add mcp_toolsets to ObjectPermissionBase, fix multi-toolset overwrite, fix delete 404, allow standalone key toolsets
* fix(mcp): add auth check on toolset resolution in responses API; union mcp_servers in _merge_toolset_permissions
* fix(mcp): handle RecordNotFoundError in update_mcp_toolset; union direct servers with toolset servers
* fix(mcp): use _user_has_admin_view; deny None mcp_toolsets for non-admin; use direct RecordNotFoundError import; fix docstring
* fix(mcp): add @default(now()) to MCPToolsetTable.updated_at; fix test for non-admin toolset access
* fix: use UniqueViolationError import; guard _ensure_eof for error/cancel only
* fix(mcp): preserve mcp_access_groups in toolset scope, use shared Redis cache for toolset perms
- Remove mcp_access_groups=[] from _apply_toolset_scope (server.py) and the
responses API toolset path (litellm_proxy_mcp_handler.py). A key's access-group
grants remain valid even when the request is scoped to a single toolset; clearing
them silently revoked legitimate entitlements.
- Switch resolve_toolset_tool_permissions and get_toolset_by_name_cached to use
user_api_key_cache (Redis-backed DualCache in production) instead of per-instance
in-memory dicts. Cache entries are now shared across workers, eliminating the
per-worker stale-toolset-permission window flagged as a P1 by Greptile.
- Use union merge (set union of tool names per server) when applying toolset
permissions in the responses API path so direct-server tool restrictions are not
overwritten by toolset permissions.
* fix(mcp): return 404 when edit_mcp_toolset target does not exist
* fix(mcp): align mcp_toolsets default to None in LiteLLM_ObjectPermissionTable
* fix(mcp): admin toolset visibility, in-place tool name mutation, test helper coercion
* fix(mcp): treat None/[] team mcp_toolsets as no restriction in key validation
* fix(mcp): allow_all_keys backward compat, blocked_tools API write-path, efficient startup query
* fix(mcp): use _mcp_active_toolset_id ContextVar to detect toolset scope, avoiding DB-default false-positive
* fix(mcp): remove dead toolset cache stubs, log invalidation failures, align schema updated_at defaults
* fix(mcp): deserialise MCPToolset from Redis cache hit, replace fastapi import in test
* fix(mcp): evict name-cache on toolset mutation, 409 on rename conflict, warning-level list errors
* fix(redis): regenerate GCP IAM token per connection for async cluster (#24426)
* fix(redis): regenerate GCP IAM token per connection for async cluster clients
Async RedisCluster was generating the IAM token once at startup and
storing it as a static password. After the 1-hour GCP token TTL, any
new connection (including to newly-discovered cluster nodes) would fail
to authenticate.
Fix: introduce GCPIAMCredentialProvider that implements redis-py's
CredentialProvider protocol. It calls _generate_gcp_iam_access_token()
on every new connection, matching what the sync redis_connect_func
already does. async_redis.RedisCluster accepts a credential_provider
kwarg which is invoked per-connection.
* refactor(redis): move GCPIAMCredentialProvider to its own file
Extract GCPIAMCredentialProvider and _generate_gcp_iam_access_token
into litellm/_redis_credential_provider.py. _redis.py imports them
from there, keeping the public API unchanged.
* fix: address Greptile review issues
- GCPIAMCredentialProvider now inherits from redis.credentials.CredentialProvider
so redis-py's async path calls get_credentials_async() properly
- move _redis_credential_provider import to top of _redis.py (PEP 8)
- remove dead else-branch that silently no-oped (gcp_service_account from
redis_kwargs.get() was always None since it's popped by _get_redis_client_logic)
- remove mid-function 'from litellm import get_secret_str' inline import
- remove unused 'call' import from test_redis.py
* chore: retrigger CI/review
* chore: sync schema.prisma copies from root
* chore: sync schema.prisma copies from root
* fix(proxy_server): use bounded asyncio.Queue with maxsize to prevent unbounded growth
* fix(a2a/pydantic_ai): make api_base Optional to match base class signature
* fix(a2a/pydantic_ai): make api_base Optional in handler and guard against None
* fix(mcp): remove unused get_all_mcp_servers import
* fix(mcp): remove unused MCPToolset import
* refactor(mcp): extract toolset permission logic to reduce statement count below PLR0915 limit
* fix(tests): update reload_servers_from_database tests to mock prisma directly
---------
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* fix(toolset_db): lazy-import prisma to avoid ImportError when prisma not installed
* fix(tests): update UI tests for toolset tab and updated empty state text
* fix(tests): add get_mcp_server_by_name to fake_manager stub
---------
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
