Raise vitest testTimeout from 10s to 30s and drop per-test timeout overrides
across UI unit tests. Group CreateUserButton and TeamInfo tests under nested
describe blocks to make the most flaky suites easier to scan.
calculateNewExpiryTime only handled s/h/d, but the grace-period
validation and backend accept m, w, and mo as well. Entering any of
those in the Expire Key field caused the function to return null,
which then propagated as expires: null in the onKeyUpdate payload —
the parent UI would then render the expiry as "Never" even though
the backend had correctly applied the new expiry.
Extend the suffix check to cover s/m/h/d/w/mo, matching "mo" before
"m" so "1mo" isn't misread as minutes. Also nullish-coalesce the
call site so an unparseable duration falls back to the previous
expiry instead of null. Add parametric tests for each supported
suffix plus a regression test for the null fallback.
The regenerate endpoint returns a GenerateKeyResponse that inherits
max_budget/tpm_limit/rpm_limit from KeyRequestBase, so the API echoes
the existing values back. The previous updatedKeyData layout spread
...response *after* the explicit formValues assignments, which meant
the user's just-submitted edits were silently overwritten by the API
echo before being propagated to the parent via onKeyUpdate.
Reorder so the response spread comes first and the formValues-derived
fields override it, and add a regression test that mocks a response
with stale limits to lock the behavior in. Also drop the two leftover
debug console.log statements.
- Label the key block with a small "Virtual Key" caption so the gray
box is clearly the key container.
- Move the Copy Key action to the modal footer as a primary button
with icon; inline copy icon next to the key is removed.
- Swap the button to "Copied" with a check icon on success instead of
firing a notification — less noisy and keeps feedback in place.
- Disable clicking outside the modal to close (maskClosable=false) so
users must explicitly dismiss via Close or X.
- Enlarge the key text and let its container span the full modal
width.
- Tests updated accordingly, including a new test for the copied-state
swap and the "Virtual Key" label.
Use Flex, Typography.Paragraph (with copyable), and Typography.Text
instead of raw divs + code block + CopyToClipboard wrapper. Drops the
direct react-copy-to-clipboard dependency in this component in favor
of antd's native copyable support.
Also fixes two test issues surfaced when running the e2e locally:
- RegenerateKeyModal.test.tsx no longer mocks react-copy-to-clipboard
(the component no longer imports it), removing the CJS require()
inside an ESM mock factory flagged by Greptile.
- keys.spec.ts scopes the Regenerate and Copy lookups to the modal.
The Regenerate button has an icon whose aria-label ("sync") is
concatenated into the button's accessible name, so an exact-match
lookup on "Regenerate" failed; and the new Paragraph copyable
renders a generic "Copy" button that collided with the other
copyable fields on the key info view.
* Add PromptGuard guardrail integration
Add PromptGuard as a first-class guardrail vendor in LiteLLM's proxy,
supporting prompt injection detection, PII redaction, topic filtering,
entity blocklists, and hallucination detection via PromptGuard's
/api/v1/guard API endpoint.
Backend:
- Add PROMPTGUARD to SupportedGuardrailIntegrations enum
- Implement PromptGuardGuardrail (CustomGuardrail subclass) with
apply_guardrail handling allow/block/redact decisions
- Add Pydantic config model with api_key, api_base, ui_friendly_name
- Auto-discovered via guardrail_hooks/promptguard/__init__.py registries
Frontend:
- Add PromptGuard partner card to Guardrail Garden with eval scores
- Add preset configuration for quick setup
- Add logo to guardrailLogoMap
Tests:
- 30 unit tests covering configuration, allow/block/redact actions,
request payload construction, error handling, config model, and
registry wiring
* Fix redact path and init ordering per review feedback
- P1: Update structured_messages (not just texts) when PromptGuard
returns a redact decision, so PII redaction is effective for the
primary LLM message path
- P2: Validate credentials before allocating the HTTPX client so
resources aren't acquired if PromptGuardMissingCredentials is raised
- Add tests for structured_messages redaction and texts-only redaction
* Harden PromptGuard integration: fail-open, event hooks, images, docs
- Add block_on_error config (default fail-closed, configurable fail-open)
- Declare supported_event_hooks (pre_call, post_call) like other vendors
- Forward images from GenericGuardrailAPIInputs to PromptGuard API
- Wrap API call in try/except for resilient error handling
- Add comprehensive documentation page with config examples
- Register docs page in sidebar alongside other guardrail providers
- Expand test suite from 32 to 40 tests covering new functionality
* Fix dict[str, Any] -> Dict[str, Any] for Python 3.8 compat
* Address remaining Greptile feedback: timeout, redact guard
- Add explicit 10s timeout to async_handler.post() to prevent
indefinite hangs when PromptGuard API is unresponsive
- Guard redact path: only update inputs["texts"] when the key
was originally present, avoiding phantom key injection
- Add test: redact with structured_messages only does not create
texts key (41 tests total)
* Fix CI lint: black formatting, add PromptGuardConfigModel to LitellmParams
- Reformat promptguard.py to match CI black version (parenthesization)
- Add PromptGuardConfigModel as base class of LitellmParams for proper
Pydantic schema validation, consistent with all other guardrail vendors
- Use litellm_params.block_on_error directly (now a typed field)
* Address Greptile review: redact path, null decision, error context
- P1: Filter _extract_texts_from_messages to user-role messages only,
preventing system/assistant content from being injected into texts
- P1: Strengthen test_redact_updates_structured_messages assertion from
weak `in` check to strict equality, catching the injection bug
- P2: Use `result.get("decision") or "allow"` to handle explicit null
decision values (not just absent keys)
- P2: Wrap bare exception re-raise in GuardrailRaisedException so the
caller knows which guardrail failed (block_on_error=True path)
- P2: Add static Promptguard entry in guardrail_provider_map so the
preset works before populateGuardrailProviderMap is called
- Add test for explicit null decision treated as allow
* Fix black formatting: collapse f-string in error message
Adopt a React Query mutation for policy attachment deletion and add a pending-state test on the policies index panel. This removes local delete-loading state and keeps modal loading tied to mutation status.
Made-with: Cursor
Replace Tremor components in the regenerate key modal with Ant Design
equivalents and move the component to a new PascalCase file. The form
layout now uses Row/Col to place Max Budget, TPM Limit, and RPM Limit
on one row and Expire Key with Grace Period on another, reducing the
vertical footprint. The success view shows an Alert banner, the key
alias as secondary context, and the regenerated key in a monospace
block with an inline primary Copy button.
Also adds unit tests for the new component and updates the existing
Playwright spec to match the new banner and button text.
enable_tag_filtering: false (the default) passed the hasValues guard,
causing every team save to send router_settings even for teams that
never had them configured. This could silently overwrite backend-set
values. Now exclude false and empty arrays from the check.
Remove the silent try/catch from setSecureItem so OAuth hooks can
surface actionable "enable storage" guidance instead of a cryptic
"state lost" error after the round-trip. Add a local try/catch in
ChatUI where the storage write is non-critical.
Remove the onChange/value feedback loop between TeamInfo and
RouterSettingsAccordion. The parent passes initial value once and reads
current values at save time via ref.getValue(). This eliminates the
circular data flow that caused stale state and fallback tab resets.
When a fallback model was selected, the onChange effect fired, parent
re-rendered with new value prop, and the useEffect re-initialized the
groups with new IDs from fallbacksToGroups. This caused the active
fallback group tab to reset to Group 1. Now internal updates always
skip re-initialization regardless of value key changes.
The uncontrolled TabGroup reset to the first tab whenever the component
re-rendered due to onChange state updates. Using controlled index/onIndexChange
keeps the user on the current tab during fallback selection.
When a user clears a numeric field in the RouterSettingsAccordion, the empty
input should result in null, not fall through to the previous value from state.
The RouterSettingsAccordion uses uncontrolled DOM inputs for numeric fields.
The onChange effect only fires when formValue or fallbacks change, not when
the user types into uncontrolled inputs. Using the ref's getValue() method
at save time ensures we read fresh values from the DOM.
Add RouterSettingsAccordion to the team edit form and a read-only
display of router settings (routing strategy, retries, fallbacks,
cooldown, timeout, tag filtering) in the Settings tab.
- Dockerfile.health_check: HEALTHCHECK now verifies the script is intact
instead of unconditionally exiting 0
- secureStorage.ts: replace deprecated escape/unescape with
encodeURIComponent/decodeURIComponent; don't delete legacy values on
decode failure so in-flight flows can time out naturally
- OAuth callback: add same-origin check before redirecting to stored
return URL
Port security fixes from litellm_v1.82.3.dev.6:
- Use secureStorage (sessionStorage wrapper) instead of raw storage for tokens
- Add URL validation for stored worker URLs to prevent open redirects
- Add same-origin checks before redirecting to stored return URLs
- Harden Dockerfile.health_check with non-root user and exec-form HEALTHCHECK
- Add constants.ts with all required exports (key aliases, team IDs)
- Add fixtures/users.ts with all role definitions and storage paths
- Add fixtures/seed.sql for deterministic test database seeding
- Remove Firefox project from playwright config (only Chromium installed)
- Remove unused variable in teams.spec.ts
- Rename CircleCI job to e2e_ui_testing
Add Playwright E2E tests covering proxy admin team and key management
workflows, with a self-contained test runner and CircleCI integration.
Tests cover: create team, invite user, edit/delete team members, create
key in team, regenerate key, update TPM/RPM limits, delete key, and
verify internal user keys are visible.
Infrastructure: run_e2e.sh builds the UI from source before starting
the proxy, ensuring tests always run against the latest UI changes.
Added data-testid attributes to key UI components for reliable selectors.
Replace static Modal.confirm with DeleteResourceModal so attachment delete reliably triggers the API call. Add a regression test covering the confirm->delete flow.
Made-with: Cursor
* feat: multiple concurrent budget windows per API key and team (#24883)
* feat(proxy): add BudgetLimitEntry type and wire budget_limits into key/team models
* feat(schema): add budget_limits Json column to VerificationToken and TeamTable
* feat(migrations): add migration for budget_limits column on keys and teams
* feat(keys): initialize budget_limits windows with reset_at on key create/update
* feat(teams): initialize budget_limits windows with reset_at on team create/update
* feat(auth): add _virtual_key_multi_budget_check and _team_multi_budget_check
* feat(auth): call multi-budget checks from common_checks for keys and teams
* feat(proxy): increment per-window Redis spend counters after each request
* feat(budget): reset individual budget windows on schedule via reset_budget_job
* feat(ui): add hourly option to BudgetDurationDropdown
* feat(ui): add budget_limits field to KeyResponse type
* feat(ui): add Budget Windows editor to key edit view
* feat(ui): add Budget Windows editor to create key form
* fix(proxy): strip budget_limits=None before Prisma upsert to fix login 500
Prisma rejects nullable JSON fields (Json? without @default) when passed as
Python None — it needs the field omitted entirely so the DB stores NULL via
the column's nullable constraint. This was breaking /v2/login because the UI
session key creation path hit the upsert with budget_limits=None.
* ui(key-edit): use antd InputNumber+Button for budget windows, add reset hints
* ui(create-key): use antd InputNumber+Button for budget windows, add reset hints
* docs(users): add multiple budget windows section with API + dashboard walkthrough
* fix: BudgetExceededError returns HTTP 429 instead of 400
- Add status_code=429 to BudgetExceededError class
- auth_exception_handler hardcoded code=400 → code=429
* fix: no-op else branch in multi-budget auth checks causes KeyError
- BudgetLimitEntry objects must be coerced via model_dump() not left as-is
- Move _virtual_key_multi_budget_check into common_checks (was asymmetric
with _team_multi_budget_check which already lived there)
* fix: len() on JSON string returns char count not window count
Guard with isinstance check + json.loads() before iterating per-window
Redis counters in increment_spend_counters
* fix: silent except:pass hides Redis reset failures in reset_budget_windows
Log Redis counter reset failures as warnings so they are observable
* test: add unit tests for multi-budget window enforcement
5 tests covering: no budget_limits passes, under budget passes,
over hourly window raises 429, over monthly window raises 429,
BudgetLimitEntry objects coerced without KeyError
* fix: key per-window counters stable across reorders (duration key, not index)
* fix: team+key per-window spend increments use duration key, not index
* fix: budget window reset uses duration key; log failures instead of swallowing
* refactor: extract BudgetWindowsEditor to shared component
* refactor: key_edit_view imports BudgetWindowsEditor from shared component
* refactor: create_key_button imports BudgetWindowsEditor from shared component
---------
Co-authored-by: Ishaan Jaffer <ishaanjaffer0324@gmail.com>
* fix(reset_budget_job): extract _reset_expired_window helper to fix PLR0915 too many statements
* feat(skills): Skills Registry & Hub — register skills, browse in AI Hub, public skill hub (#25118)
* feat(skills): add domain and namespace fields to plugin types
* feat(skills): store and return domain/namespace inside manifest_json
* feat(skills): add /public/skill_hub endpoint for unauthenticated access
* feat(skills): whitelist /public/skill_hub from auth requirements
* feat(skills): add domain, namespace to Plugin and RegisterPluginRequest types
* feat(skills): smart URL parser — paste github URL, auto-detect source type and name
* feat(skills): replace enable toggle with Public badge, make rows clickable
* feat(skills): add skill detail view with Overview and How to Use tabs
* feat(skills): add MakeSkillPublicForm modal for publishing skills to the hub
* feat(skills): rename panel to Skills, wire in skill detail view on row click
* feat(skills): add skill hub table columns — name, description, domain, source, status
* feat(skills): add SkillHubDashboard with stats row, domain dropdown filter, and table
* feat(skills): add Skill Hub tab to AI Hub with Select Skills to Make Public button
* feat(skills): move Skills to top-level nav item directly under MCP Servers
* feat(skills): add skillHubPublicCall and NEXT_PUBLIC_BASE_URL support
* feat(skills): add Skill Hub tab to public AI Hub page
* feat(skills): add skills page routing in main app router
* feat(skills): add /skills page route
* chore: update package-lock after npm install
* docs(skills): add Skills Gateway doc page with mermaid architecture diagram
* docs(skills): add Skills Gateway to sidebar under Agent & MCP Gateway
* docs(skills): add loom walkthrough video to Skills Gateway doc
* chore: fixes
---------
Co-authored-by: Ishaan Jaffer <ishaanjaffer0324@gmail.com>
Co-authored-by: Yuneng Jiang <yuneng@berri.ai>
* fix(bedrock): strip [1m]/[200k] context window suffixes before cost lookup
* test(bedrock): add test for [1m] context window suffix stripping in cost lookup
* schema: add allowed_models to BudgetTable, default_team_member_models to TeamTable
* migration: add allowed_models and default_team_member_models columns
* types: add allowed_models to TeamMemberAddRequest, TeamMemberUpdateRequest, UpdateTeamRequest
* utils: add allowed_models param to add_new_member, persist to budget table
* common_utils: add allowed_models to _upsert_budget_and_membership
* team endpoints: seed allowed_models on member_add, persist on member_update and team/update
* auth: enforce per-member allowed_models at request time
* networking: add allowed_models to Member type and teamMemberUpdateCall
* TeamMemberTab: add Model Scope column showing per-member allowed_models
* EditMembership: add Allowed Models multi-select field
* TeamInfo: add default_team_member_models field in Settings tab
* chore: sync schema.prisma copies from root
* fix(team_member_update): update existing budget in-place instead of creating new one
When a member already has a budget_id, patch only the fields the caller
provided rather than always creating a fresh budget record. The old
code ignored existing_budget_id entirely, so updating only allowed_models
silently dropped the stored max_budget / tpm_limit / rpm_limit values.
* fix(auth): pass llm_router to _check_team_member_model_access
Without the router, _can_object_call_model cannot resolve wildcard model
names (e.g. openai/*) or access-group names in allowed_models, causing
legitimate requests to be denied. Thread the existing llm_router from
_run_common_checks through to the new member-scope check.
* feat(ui): add Team Member Settings accordion to Create Team modal
Groups default_team_member_models, member budget/key duration, and
tpm/rpm defaults into a single collapsible section. The model picker
is filtered to only show the models selected for the team, and the
copy distinguishes it from the team-level Models field.
* feat(ui): consolidate Team Member Settings into accordion in edit team form
Moves default_team_member_models + per-member budget/key/tpm/rpm fields
into a collapsible "Team Member Settings" panel. Keeps the top-level
form focused on team-wide settings (team models, team budget, tpm/rpm).
* fix(ui): use tremor Accordion for Team Member Settings in edit team form
* fix(ui): move Team Member Settings accordion above budget fields in Create Team
* chore: fixes
---------
Co-authored-by: Ishaan Jaffer <ishaanjaffer0324@gmail.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Yuneng Jiang <yuneng@berri.ai>
Unit Tests: Proxy DB Operations / proxy-db (auth-checks, tests/proxy_unit_tests/test_auth_checks.py tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-db (remaining, tests/proxy_unit_tests --ignore=tests/proxy_unit_tests/test_key_generate_prisma.py --ignore=tests/proxy_unit_tests/test_auth_checks.py --ignore=tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Has been cancelled
* fix(ui): resolve login redirect loop when reverse proxy adds HttpOnly to cookies
When LiteLLM is behind nginx-ingress or similar with security-hardened
configs, the reverse proxy adds HttpOnly to all Set-Cookie headers. This
makes the JWT token unreadable by JavaScript, causing an infinite login
redirect loop. Fix by returning the JWT token in the /v2/login response
body so the frontend can set a JS-accessible cookie directly.
Fixes#19663
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: address Greptile review feedback
- Add window guard to setTokenCookie for SSR consistency with clearTokenCookies
- Add SSR test for window undefined case
- Add code comment explaining why JWT is included in response body
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: address second round of Greptile review feedback
- Add loginCall integration tests verifying setTokenCookie is called with
token and skipped when absent (backward-compatibility path)
- Use encodeURIComponent/decodeURIComponent in setTokenCookie/getCookie
for defense-in-depth against non-standard token formats
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Update ui/litellm-dashboard/src/utils/cookieUtils.ts
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* Update ui/litellm-dashboard/src/utils/cookieUtils.ts
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* fix(ui): use sessionStorage instead of cookie for login token storage
Replace setTokenCookie (which is a no-op when reverse proxy adds HttpOnly)
with storeLoginToken using sessionStorage. Add sessionStorage fallback to
getCookie so the token is found even when the cookie is HttpOnly. Also handle
'=' in cookie values with .slice(1).join("=") and clear sessionStorage on
logout.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(ui): use shared getCookie in page.tsx and user_dashboard.tsx
Replace local getCookie functions in page.tsx and user_dashboard.tsx
with the shared one from cookieUtils that has the sessionStorage
fallback. Without this, the HttpOnly cookie fix was incomplete —
page.tsx (the dashboard entry point) could not read the token,
causing the redirect loop to persist.
Also scope the sessionStorage fallback to the "token" key only,
and clear sessionStorage in page.tsx deleteCookie.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(ui): scope deleteCookie sessionStorage cleanup to token key only
Also document the sessionStorage cross-tab trade-off: per-tab scope
means users behind an HttpOnly proxy must log in once per tab, but
this is intentional to avoid localStorage XSS exposure.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Update ui/litellm-dashboard/src/utils/cookieUtils.ts
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* Apply suggestion from @greptile-apps[bot]
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* style: remove stray double blank line in user_dashboard.tsx
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(ui): guard storeLoginToken against empty/whitespace-only tokens
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(ui): preserve sessionStorage token across beforeunload clear
The existing beforeunload handler calls sessionStorage.clear() to
flush cached UI data on page refresh. This also wiped the token
stored by storeLoginToken, re-introducing the redirect loop after
any page refresh in the HttpOnly proxy scenario. Now the token is
saved and restored across the clear.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(ui): set JS-accessible cookie at /ui path as HttpOnly workaround
sessionStorage alone is unreliable. Also set the token via
document.cookie at path=/ui — nginx only adds HttpOnly to server-set
Set-Cookie headers, so a JS-set cookie is always readable.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(ui): use dynamic cookie path based on server_root_path
Hardcoded path=/ui breaks when LiteLLM is deployed with a custom
server_root_path. Now derives the cookie path from serverRootPath
so it works at /ui, /myapp/ui, etc.
Also reuse clearTokenCookies() in deleteCookie() to avoid duplication.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor(ui): remove circular dependency in cookieUtils.ts
Derive the UI cookie path from window.location.pathname instead of
importing serverRootPath from networking.tsx. This breaks the
cookieUtils → networking → cookieUtils cycle that could cause
serverRootPath to be undefined under certain bundler configurations.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(ui): harden getUiCookiePath regex and add missing tests
- Use regex /\/ui(?=\/|$)/ to match "/ui" only as a full path segment,
preventing false matches on paths like "/my-ui-tool/login".
- Add unit tests for storeLoginToken empty/whitespace guard and
cookie-at-/ui-path behavior.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* style: fix Black formatting in audit_logs.py
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix CI: formatting, test params, remove token from login JSON
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: reformat with Black 23.x to match CI
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: keep token in login JSON body for UI storeLoginToken flow
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: use storeLoginToken in exchangeLoginCode, add credentials include
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* revert: remove unrelated changes from HttpOnly cookie fix branch
Reset files not related to the login cookie fix back to main:
- prometheus.py, bedrock converse, guardrail handler
- auth_checks.py, reset_budget_job.py, audit_logs.py
- test_user_api_key_auth.py
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Revert "revert: remove unrelated changes from HttpOnly cookie fix branch"
This reverts commit 0684a1e275.
* Revert "fix: use storeLoginToken in exchangeLoginCode, add credentials include"
This reverts commit 866405f443.
* Revert "fix: keep token in login JSON body for UI storeLoginToken flow"
This reverts commit 086c41640c.
* Revert "fix: reformat with Black 23.x to match CI"
This reverts commit b2c3334c88.
* Revert "fix CI: formatting, test params, remove token from login JSON"
This reverts commit 2905d47bd4.
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Unit Tests: Proxy DB Operations / proxy-db (auth-checks, tests/proxy_unit_tests/test_auth_checks.py tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Waiting to run
Unit Tests: Proxy DB Operations / proxy-db (remaining, tests/proxy_unit_tests --ignore=tests/proxy_unit_tests/test_key_generate_prisma.py --ignore=tests/proxy_unit_tests/test_auth_checks.py --ignore=tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Waiting to run
* docs(blog): add cosign Docker image verification instructions
Add steps for verifying Docker images with cosign to three security blog posts:
CI/CD v2, Security Townhall, and Security Update.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs(proxy): add cosign verification to Docker/Helm/Terraform deploy page
Add image signature verification steps to the main deployment doc so
users pulling Docker images know how to verify them with cosign.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: fixes
* Update index.md
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* [Docs] Scope cosign signing docs to GHCR and specify starting version
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* [Docs] Add starting version callout to ci_cd_v2 blog post
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Krrish Dholakia <krrish+github@berri.ai>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Unit Tests: Proxy DB Operations / proxy-db (auth-checks, tests/proxy_unit_tests/test_auth_checks.py tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-db (remaining, tests/proxy_unit_tests --ignore=tests/proxy_unit_tests/test_key_generate_prisma.py --ignore=tests/proxy_unit_tests/test_auth_checks.py --ignore=tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-db (auth-checks, tests/proxy_unit_tests/test_auth_checks.py tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-db (remaining, tests/proxy_unit_tests --ignore=tests/proxy_unit_tests/test_key_generate_prisma.py --ignore=tests/proxy_unit_tests/test_auth_checks.py --ignore=tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Has been cancelled
The test was hitting "No QueryClient set" because TeamGuardrailsTab
(which pulls in useRegisterGuardrail) was not mocked alongside the
other tab children. Added a mock.
Also: the "+ Add New Guardrail" assertion was silently relying on
Tremor Tabs rendering all panels at once. antd Tabs only renders
the active tab's content, and defaultActiveKey is "submitted", so
the button in the "Guardrails" tab wasn't in the DOM. Clicking the
Guardrails tab first before asserting.
* Litellm ishaan march23 - MCP Toolsets + GCP Caching fix (#25146)
* feat(mcp): MCP Toolsets — curated tool subsets from one or more MCP servers (#24335)
* feat(mcp): add LiteLLM_MCPToolsetTable and mcp_toolsets to ObjectPermissionTable
* feat(mcp): add prisma migration for MCPToolset table
* feat(mcp): add MCPToolset Python types
* feat(mcp): add toolset_db.py with CRUD helpers for MCPToolset
* feat(mcp): add toolset CRUD endpoints to mcp_management_endpoints
* fix(mcp): skip allow_all_keys servers when explicit mcp_servers permission is set (toolset scope fix)
* feat(mcp): add _apply_toolset_scope and toolset route handling in server.py
* fix(mcp): resolve toolset names in responses API before fetching tools
* feat(mcp): add mcp_toolsets field to LiteLLM_ObjectPermissionTable type
* feat(mcp): register LiteLLM_MCPToolsetTable in prisma client initialization
* feat(mcp): validate mcp_toolsets in key-vs-team permission check
* feat(mcp): register toolset routes in proxy_server.py
* feat(mcp): add MCPToolset and MCPToolsetTool TypeScript types
* feat(mcp): add fetchMCPToolsets, createMCPToolset, updateMCPToolset, deleteMCPToolset API functions
* feat(mcp): add useMCPToolsets React Query hook
* feat(mcp): add toolsets (purple) as third option type in MCPServerSelector
* feat(mcp): extract toolsets from combined MCP field in key form
* feat(mcp): extract toolsets from combined MCP field in team form
* feat(mcp): show toolsets section in MCPServerPermissions read view
* feat(mcp): pass mcp_toolsets through object_permissions_view
* feat(mcp): add MCPToolsetsTab component for creating and managing toolsets
* feat(mcp): add Toolsets tab to mcp_servers.tsx
* feat(mcp): pass mcpToolsets to playground chat and responses API calls
* feat(mcp): generate correct server_url for toolsets in playground API calls
* docs(mcp): add MCP Toolsets documentation
* docs(mcp): add mcp_toolsets to sidebar
* fix(mcp): replace x-mcp-toolset-id header with ContextVar to prevent client forgery
* fix(mcp): use ContextVar + StreamingResponse for toolset MCP routes (fixes SSE streaming)
* fix(mcp): cache toolset permission lookups to avoid per-request DB calls
* test(mcp): add tests for toolset scope enforcement, ContextVar isolation, and access control
* fix(mcp): cache toolset name lookups in MCPServerManager to avoid per-request DB calls
* fix(mcp): prevent body_iter deadlock + use cached toolset lookup in responses API
- _stream_mcp_asgi_response: add done callback to handler_task that puts
the EOF sentinel on body_queue when the task exits, preventing body_iter
from hanging forever if the handler raises after headers are sent.
- litellm_proxy_mcp_handler: replace raw get_mcp_toolset_by_name() DB call
with global_mcp_server_manager.get_toolset_by_name_cached() so toolset
resolution uses the 60s TTL cache added for this purpose instead of
hitting the DB on every responses-API request.
* fix(mcp): toolset access control, asyncio fix, and real unit tests
- server.py: _apply_toolset_scope now enforces that non-admin keys must
have the requested toolset_id in their mcp_toolsets grant list;
admin keys always bypass the check.
- mcp_management_endpoints.py: three access-control fixes:
* fetch_mcp_toolsets: non-admin keys with mcp_toolsets=None now
return [] instead of all toolsets (only admins get 'all' when
the field is absent)
* fetch_mcp_toolset: non-admin keys that haven't been granted the
requested toolset_id now get 403 instead of the full result
* add_mcp_toolset: duplicate toolset_name now returns 409 Conflict
instead of an opaque 500
- proxy_server.py: use asyncio.get_running_loop() instead of
get_event_loop() inside an already-running coroutine (Python 3.10+).
- test_mcp_toolset_scope.py: replace four hollow tests that only
asserted local variable properties with real tests that call the
production fetch_mcp_toolsets() and handle_streamable_http_mcp()
functions with mocked dependencies.
* fix(mcp): add mcp_toolsets to ObjectPermissionBase, fix multi-toolset overwrite, fix delete 404, allow standalone key toolsets
* fix(mcp): add auth check on toolset resolution in responses API; union mcp_servers in _merge_toolset_permissions
* fix(mcp): handle RecordNotFoundError in update_mcp_toolset; union direct servers with toolset servers
* fix(mcp): use _user_has_admin_view; deny None mcp_toolsets for non-admin; use direct RecordNotFoundError import; fix docstring
* fix(mcp): add @default(now()) to MCPToolsetTable.updated_at; fix test for non-admin toolset access
* fix: use UniqueViolationError import; guard _ensure_eof for error/cancel only
* fix(mcp): preserve mcp_access_groups in toolset scope, use shared Redis cache for toolset perms
- Remove mcp_access_groups=[] from _apply_toolset_scope (server.py) and the
responses API toolset path (litellm_proxy_mcp_handler.py). A key's access-group
grants remain valid even when the request is scoped to a single toolset; clearing
them silently revoked legitimate entitlements.
- Switch resolve_toolset_tool_permissions and get_toolset_by_name_cached to use
user_api_key_cache (Redis-backed DualCache in production) instead of per-instance
in-memory dicts. Cache entries are now shared across workers, eliminating the
per-worker stale-toolset-permission window flagged as a P1 by Greptile.
- Use union merge (set union of tool names per server) when applying toolset
permissions in the responses API path so direct-server tool restrictions are not
overwritten by toolset permissions.
* fix(mcp): return 404 when edit_mcp_toolset target does not exist
* fix(mcp): align mcp_toolsets default to None in LiteLLM_ObjectPermissionTable
* fix(mcp): admin toolset visibility, in-place tool name mutation, test helper coercion
* fix(mcp): treat None/[] team mcp_toolsets as no restriction in key validation
* fix(mcp): allow_all_keys backward compat, blocked_tools API write-path, efficient startup query
* fix(mcp): use _mcp_active_toolset_id ContextVar to detect toolset scope, avoiding DB-default false-positive
* fix(mcp): remove dead toolset cache stubs, log invalidation failures, align schema updated_at defaults
* fix(mcp): deserialise MCPToolset from Redis cache hit, replace fastapi import in test
* fix(mcp): evict name-cache on toolset mutation, 409 on rename conflict, warning-level list errors
* fix(redis): regenerate GCP IAM token per connection for async cluster (#24426)
* fix(redis): regenerate GCP IAM token per connection for async cluster clients
Async RedisCluster was generating the IAM token once at startup and
storing it as a static password. After the 1-hour GCP token TTL, any
new connection (including to newly-discovered cluster nodes) would fail
to authenticate.
Fix: introduce GCPIAMCredentialProvider that implements redis-py's
CredentialProvider protocol. It calls _generate_gcp_iam_access_token()
on every new connection, matching what the sync redis_connect_func
already does. async_redis.RedisCluster accepts a credential_provider
kwarg which is invoked per-connection.
* refactor(redis): move GCPIAMCredentialProvider to its own file
Extract GCPIAMCredentialProvider and _generate_gcp_iam_access_token
into litellm/_redis_credential_provider.py. _redis.py imports them
from there, keeping the public API unchanged.
* fix: address Greptile review issues
- GCPIAMCredentialProvider now inherits from redis.credentials.CredentialProvider
so redis-py's async path calls get_credentials_async() properly
- move _redis_credential_provider import to top of _redis.py (PEP 8)
- remove dead else-branch that silently no-oped (gcp_service_account from
redis_kwargs.get() was always None since it's popped by _get_redis_client_logic)
- remove mid-function 'from litellm import get_secret_str' inline import
- remove unused 'call' import from test_redis.py
* chore: retrigger CI/review
* chore: sync schema.prisma copies from root
* chore: sync schema.prisma copies from root
* fix(proxy_server): use bounded asyncio.Queue with maxsize to prevent unbounded growth
* fix(a2a/pydantic_ai): make api_base Optional to match base class signature
* fix(a2a/pydantic_ai): make api_base Optional in handler and guard against None
* fix(mcp): remove unused get_all_mcp_servers import
* fix(mcp): remove unused MCPToolset import
* refactor(mcp): extract toolset permission logic to reduce statement count below PLR0915 limit
* fix(tests): update reload_servers_from_database tests to mock prisma directly
---------
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* fix(toolset_db): lazy-import prisma to avoid ImportError when prisma not installed
* fix(tests): update UI tests for toolset tab and updated empty state text
* fix(tests): add get_mcp_server_by_name to fake_manager stub
---------
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
