Backend:
- list_guardrail_submissions no longer 403s non-admins; it returns only
submissions whose team_id matches one of the caller's teams (via
get_user_object.teams). Admins still see all.
- Filtering by a team the caller is not in returns 403.
- Users with no team memberships get an empty list (no DB query).
- get_guardrail_submission applies the same scoping to single-item GETs.
Frontend:
- Remove admin-only bail-out in TeamGuardrailsTab.fetchSubmissions so
internal users actually load their team's submissions.
- Finish antd migration in guardrails.tsx: drop the last Tremor Button.
- Remove guardrailsList.length === 0 gate on the Test Playground tab;
the playground already renders a "No guardrails available" inline
empty state, which is more discoverable than a disabled tab.
Tests:
- Cover non-admin scoped access, empty teams, cross-team filter 403,
and per-submission GET scoping.
* fix(pricing): add unversioned vertex_ai/claude-haiku-4-5 entry
Missing unversioned entry causes cost tracking to return $0.00 for
all requests using vertex_ai/claude-haiku-4-5. All other Vertex AI
Claude models have both versioned and unversioned entries.
* fix(router): skip misleading tags error when no candidates (e.g. cooldown)
Return early from get_deployments_for_tag when healthy_deployments is empty so
tag-based routing does not raise no_deployments_with_tag_routing after cooldown
filters all deployments. Adds regression test.
Made-with: Cursor
* feat(oci): add embedding support and update model catalog
- Add OCIEmbeddingConfig for OCI GenAI embedding models
- Add 16 new chat models (Cohere, Meta Llama, xAI Grok, Google Gemini)
- Add 8 embedding models (Cohere embed v3.0, v4.0)
- Update documentation with embedding examples
- Update pricing for all new models
* test(oci): add unit tests for OCI embedding support
- 17 unit tests covering OCIEmbeddingConfig
- Tests for URL generation, param mapping, request/response transform
- Tests for model pricing JSON completeness
* style(oci): format with black and ruff
* fix(oci): correct embedding request body format
OCI embedText API expects inputs, truncate, and inputType at the
top level of the request body, not nested under embedTextDetails.
Fixed transformation and updated tests accordingly.
Verified with real OCI API: 3/3 embedding models working.
* docs: clarify tag routing early return and test intent
Made-with: Cursor
* fix(oci): address code review findings from Greptile
- P1: Fix signing URL mismatch with custom api_base by accepting
api_base parameter in transform_embedding_request
- P2: Remove encoding_format from supported params (OCI does not
support it, was silently dropped)
- P2: Raise ValueError for token-array inputs instead of silently
converting to string representation
- Add test for token-list rejection
* fix(mcp): add STS AssumeRole support for MCP SigV4 authentication
MCPSigV4Auth only supported static AWS credentials or the boto3 default
credential chain. Production Kubernetes environments typically authenticate
via IAM role assumption (sts:AssumeRole), which was not possible.
Add aws_role_name and aws_session_name parameters to the MCP SigV4 auth
stack. When aws_role_name is provided, MCPSigV4Auth calls sts:AssumeRole
to obtain temporary credentials before signing requests. Explicit keys,
if also provided, are used as the source identity for the STS call;
otherwise ambient credentials (pod role, instance profile) are used.
* fix: stop logging credential values and add missing redaction patterns
Replaces raw credential values in debug/error log messages with
boolean presence checks or type names. Adds PEM block, GCP token,
JWT, SAS token, and service-account blob patterns to the redaction
filter. Fixes private_key pattern to capture full PEM blocks instead
of stopping at the first whitespace.
Addresses: Vertex AI credential JSON (including RSA private key)
being logged to stderr on health check failures.
* fix: log only field names for UserAPIKeyAuth, not full object
* style: apply black formatting to experimental_mcp_client/client.py
* style: fix black/isort formatting and mypy error in proxy_server.py
- Fix black formatting in experimental_mcp_client/client.py (done in prev commit)
- Fix black/isort formatting in key_management_endpoints.py, proxy_server.py, transformation.py
- Fix mypy: iterate over optional list safely (access_group_ids or []) in proxy_server.py
* fix(test): patch check_migration.verbose_logger directly to fix xdist ordering issue
When test_proxy_cli.py tests run before test_check_migration.py in the same
xdist worker, litellm.proxy.db.check_migration is already in sys.modules.
Patching litellm._logging.verbose_logger has no effect on the already-bound
reference. Patch the correct target (check_migration.verbose_logger) and
import the module before patching so the order doesn't matter.
* fix(mypy): make api_base Optional in PydanticAIProviderConfig to match base class signature
---------
Co-authored-by: Ihsan Soydemir <soydemir.ihsan@gmail.com>
Co-authored-by: Milan <milan@berri.ai>
Co-authored-by: Daniel Gandolfi <danielgandolfi@gmail.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: michelligabriele <gabriele.michelli@icloud.com>
Co-authored-by: user <70670632+stuxf@users.noreply.github.com>
Co-authored-by: Ishaan Jaffer <ishaanjaffer0324@gmail.com>
The previous fix (9dca431989) switched the export display logic to use the
entity key directly, but left the unit tests asserting the old behavior
where id/alias were extracted from api_key_breakdown metadata.
Rename mock entity keys from entity1/entity2 to "team-1"/"team-2" to
reflect the real shape of team-export payloads (breakdown.entities is
keyed by the entity identifier). Replace three tests whose assertions
documented the removed behavior:
- "should use key alias when available" → entity key is the alias when
no team alias map is supplied
- "should use team alias map when key alias is not available" → team
alias map resolves from the entity key
- "should use dash when team id is not available" → entity key itself
is the fallback label (illustrated with a tag export)
The export utility always extracted team_id from api_key_breakdown
metadata to populate the entity label/ID columns. This worked for
team exports (where entity key = team_id) but was wrong for every
other entity type — tags, orgs, customers, agents, users all
showed the API key's team name (e.g. "admins") instead of the
actual entity value.
Replace extractTeamIdFromApiKeyBreakdown with resolveEntityDisplay
which uses the entity key directly. For teams the teamAliasMap
still resolves a human-readable alias; for all other types the
entity key itself is the correct label.
Previously, a row with a model selected but both limits blank was
silently dropped on save (neither model_tpm_limit nor model_rpm_limit
got the key), so the row disappeared on reload with no feedback.
Now the TPM field's validator blocks submission with "Set at least
one of TPM or RPM" when a row has a model but neither limit filled.
Exposes the backend's existing model_tpm_limit/model_rpm_limit fields
(which lived in team.metadata) through a new "Model-Specific Rate Limits"
form section on the team Settings tab. Limits round-trip through the
team-update API and render on the Overview card and Settings view.
Model picker is scoped to the team's currently-selected models (unfurls
wildcards, falls back to userModels for all-proxy-models / all-team-models).
Follow-up to the previous commit. The initial fix correctly prevented
injecting vector_store_ids: [] when the user never set any, but broke
the inverse case: a user who had ["vs_abc"] set and cleared the
selector would have their change silently ignored, because the handler
deleted the key from the PATCH payload and the backend's merge kept
the old value.
Distinguish "never touched" from "explicitly cleared" by initializing
the form field to undefined (not []) when the model has no stores, and
adding a middle branch in the submit handler that sends [] when the
form value is [] (user cleared) versus deleting the key when it's
undefined (user never touched).
Verifies that saving an edit to a model without vector stores does not
inject vector_store_ids into the PATCH payload. The test fails against
the pre-fix handler (payload contains vector_store_ids: []) and passes
with the length-guard fix.
The edit-model form initialized vector_store_ids to [] and then
unconditionally included it in the update payload. Editing any unrelated
field (e.g. temperature) on a model without vector stores would inject
vector_store_ids: [] into the model's litellm_params via the PATCH-merge
backend, which then propagated to inference requests and broke Anthropic
calls.
Mirror the cache_control_injection_points pattern directly below:
only include the field when length > 0, otherwise delete the key from
the payload so PATCH leaves the stored value untouched.
AntD <Select allowClear> returns undefined when the user clears the
selection. Coalescing to "" caused the team-update payload to carry
organization_id: "" instead of null, relying on the backend to coerce
it. Send null directly so the intent is explicit at the source.
- useKeyAliases, PaginatedKeyAliasSelect: add trailing `undefined` to
spy matchers for the new `team_id` param on `useInfiniteKeyAliases`
and `keyAliasesCall`.
- EntityUsage: mock new `TeamMultiSelect` child so QueryClientProvider
is not required for team-entity tests.
- ModelsCell: replace the overflow-accordion test with one that
verifies the new collapse-on-`all-proxy-models` behavior (no
accordion, single badge).
Add a self-contained Playwright E2E test suite that runs against a local
PostgreSQL database instead of Neon. Tests cover role-based access for all
5 user roles (proxy admin, admin viewer, internal user, internal viewer,
team admin) and authentication flows.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The TeamData interface was missing access_group_models,
access_group_mcp_server_ids, and access_group_agent_ids fields,
causing a TypeScript build failure.
The Key Alias dropdown on the Virtual Keys page was showing aliases from
all teams regardless of which team was selected. The team_id was never
passed through the frontend chain to the backend /key/aliases endpoint.
- Backend: add optional team_id query param to /key/aliases endpoint
- networking.tsx: add team_id param to keyAliasesCall
- useKeyAliases: accept and forward team_id to API call and query key
- filter.tsx: pass allFilters context to custom filter components
- PaginatedKeyAliasSelect: read Team ID from allFilters and pass to hook
* Litellm ishaan april1 (#25103)
* fix(proxy): enforce upperbound key params on key/update and add custom_key_update hook
The /key/update endpoint did not enforce upperbound_key_generate_params,
allowing users to bypass configured limits (tpm_limit, rpm_limit,
max_budget, duration, budget_duration) by updating an existing key
instead of generating a new one.
Extract the upperbound enforcement logic from _common_key_generation_helper()
into a standalone _enforce_upperbound_key_params() function and call it from
both the generate and update paths. For updates, None values are skipped
(not filled with defaults) since they mean "don't change this field".
Also adds a custom_key_update config option and user_custom_key_update global,
mirroring the existing custom_key_generate pattern, so custom key validation
logic can fire during key updates as well.
* fix(proxy): invoke custom_key_update hook in bulk update path
The user_custom_key_update hook was only called in update_key_fn
(single key update) but not in _process_single_key_update (bulk
update path), allowing custom validation to be bypassed via the
/key/update/bulk endpoint. Mirror the hook invocation in both paths.
* fix(proxy): pass UpdateKeyRequest to hook in bulk path, not BulkUpdateKeyRequestItem
Move the custom_key_update hook invocation to after UpdateKeyRequest
is constructed so the hook receives the same type in both single and
bulk update paths. Previously the bulk path passed
BulkUpdateKeyRequestItem (5 fields only), which would cause
AttributeError for hooks accessing fields like tpm_limit or models.
* fix(bedrock): promote cache usage to message_delta for Claude Code (#24850)
Ensure Bedrock/Anthropic-compatible streaming exposes cache usage where Claude Code reads it by promoting message_stop usage onto message_delta and preserving usage fields in fake-streamed message_delta events.
Made-with: Cursor
* fix(search): Support self-hosted Firecrawl response format in search transform (#24866)
The `transform_search_response` method only handled Firecrawl Cloud (v2)
response format where `data` is a dict with `web`/`news` keys. Self-hosted
Firecrawl (v1) returns `data` as a flat list of result objects, causing an
`AttributeError: 'list' object has no attribute 'get'`.
Detect the response format by checking if `data` is a list (self-hosted)
or dict (cloud) and handle both cases.
Cloud format: {"data": {"web": [...], "news": [...]}}
Self-hosted: {"success": true, "data": [{"url": "...", "title": "...", ...}]}
Co-authored-by: Synergy <synergyoclaw@gmail.com>
* feat: add environment and user tracking to prompt management (#24855)
* feat: add environment and user tracking to prompt management
- Add environment (development/staging/production) and created_by columns to LiteLLM_PromptTable
- Update unique constraint to [prompt_id, version, environment]
- All CRUD endpoints support environment filtering and user tracking
- Redesigned prompt detail page with environment tabs and version history
- UI: environment filter on list page, environment selector in editor
- 8 new tests for environment and user tracking
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: Black formatting and add environments to PromptInfoResponse TypeScript type
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: address Greptile review findings
- P1: delete_prompt scopes in-memory cleanup to environment when provided
- P2: dotprompt_content parsed directly regardless of environment flag
- P2: use distinct for environments query
- P2: fix double-fetch on initial mount in prompt_info.tsx
- fix: remove unsupported select kwarg from find_many
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: address remaining Greptile review comments
- Remove unused useCallback import (index.tsx)
- Remove unused ENV_COLORS variable (prompt_info.tsx)
- P1: in-memory fallback in get_prompt_versions now respects environment filter
- P1: reset selectedEnv when promptId changes to avoid stale state
- Cyclic imports are pre-existing pattern, not introduced by this PR
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: scope patch_prompt to environment using primary key
- Add environment query param to patch_prompt endpoint
- Look up target row by composite key (prompt_id + version + environment)
- Update by primary key (id) to target exactly one row
- Fixes Greptile finding: patch with multiple environments no longer ambiguous
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: use actual start_time for failed request spend logs (#24906)
async_post_call_failure_hook set both start_time and end_time to
datetime.now(), making all failed requests show duration=0. Use the
actual start_time from litellm_logging_obj instead, so spend logs
reflect the real request duration on timeout and other failures.
Fixes#24888
* feat(bedrock): add nova canvas image edit support (#24869)
* feat(bedrock): add nova canvas image edit support
* fix(bedrock): support PathLike inputs for nova image edit
* chore: sync schema.prisma copies from root
* fix(mypy): correct type-ignore code for delta_usage arg-type
* fix(mypy): cast status_code to str, suppress intentional str yield
* fix(lint): extract _create_content_block_chunks to fix PLR0915
* fix(lint): extract helpers to fix PLR0915 in prompt endpoints
---------
Co-authored-by: michelligabriele <gabriele.michelli@icloud.com>
Co-authored-by: Sameer Kankute <sameer@berri.ai>
Co-authored-by: redhelix <amin.lalji@gmail.com>
Co-authored-by: Synergy <synergyoclaw@gmail.com>
Co-authored-by: Talha Anwar <37379131+talhaanwarch@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: madhu19991 <madhu@thunkai.com>
Co-authored-by: Srikanth @adobe <devarakondasrikanth@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* fix(test): update model armor streaming test to handle string or int error code
---------
Co-authored-by: michelligabriele <gabriele.michelli@icloud.com>
Co-authored-by: Sameer Kankute <sameer@berri.ai>
Co-authored-by: redhelix <amin.lalji@gmail.com>
Co-authored-by: Synergy <synergyoclaw@gmail.com>
Co-authored-by: Talha Anwar <37379131+talhaanwarch@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: madhu19991 <madhu@thunkai.com>
Co-authored-by: Srikanth @adobe <devarakondasrikanth@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Replace the static team dropdown on the usage page with a new
TeamMultiSelect component that uses the paginated v2/team/list
endpoint with debounced server-side search and infinite scroll.
- Replace Tremor TabGroup with antd Tabs (key-based matching) to fix
blank content when conditional tabs are hidden for non-admins
- Skip /guardrails/submissions fetch for non-admin users via
useAuthorized() hook (avoids 401 error)
- Move get_team_membership to inline import in register endpoint
- Non-admins default to Submitted Guardrails tab with Add Guardrail
button visible
- Make team_id optional again, fall back to API key's team_id
(preserves backwards compatibility)
- Add PROXY_ADMIN bypass for team membership check
(admins can register guardrails for any team)
- Move get_team_membership import to module level
- Prevent extra_litellm_params spread from overwriting controlled
fields (guardrail, mode, api_base) by spreading extras first
Backend:
- Add required team_id to RegisterGuardrailRequest body
- Validate team membership before allowing submission
Frontend:
- Add useRegisterGuardrail React Query mutation hook
- Wire form onFinish to mutation, pass team_id from dropdown
- Remove admin-only restriction on guardrails nav item
- Internal users see only Test Playground + Submitted Guardrails tabs
- Admins continue to see all 4 tabs
Backend:
- Add required team_id to RegisterGuardrailRequest body
- Validate team membership before allowing submission
Frontend:
- Add useRegisterGuardrail React Query mutation hook
- Wire form onFinish to mutation, pass team_id from dropdown
- Show notification and refresh submissions on success
Wire the placeholder "Add Guardrail" button to open an antd Modal+Form
with team selector, guardrail name, mode, API base URL, optional extra
litellm_params JSON, and optional guardrail_info JSON. Backend call is
stubbed with a TODO for now.
- Remove get_access_object from module-level import in team_endpoints.py
and use a lazy _get_access_object wrapper to avoid cyclic dependency
- Add _prisma_client is None early-exit guard in _resolve_access_group_resources
- Short-circuit UI to show "All Proxy Models" when team.models is empty
or contains "all-proxy-models", skipping access group model resolution
Add access_group_models, access_group_mcp_server_ids, and
access_group_agent_ids to /team/info and /v2/team/list responses.
These fields contain resources inherited from access groups, kept
separate from direct assignments so the UI can distinguish the source.
Backend: _resolve_access_group_resources() helper resolves access
group resources via existing _get_*_from_access_groups() functions.
UI: Teams table and detail view show direct models as blue badges
and access-group-sourced models as green badges.
Remove @neondatabase/api-client and neonctl to address CVE-2026-25639
(axios supply chain vulnerability). Pin all JS dependencies to exact
versions across all package.json files to prevent future supply chain
attacks via semver range resolution.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Source fixes:
- page.tsx: add explicit isValidReturnUrl() check at redirect site
- public_model_hub.tsx: replace() → replaceAll() for all wildcard occurrences
- CodeSnippets.tsx: escape backslashes before quotes in generated Python
- TeamGuardrailsTab.tsx: escape backslashes before quotes in generated YAML
CodeQL suppressions for false positives:
- ChatUI.tsx: sessionStorage for apiKey/apiKeySource (sessionStorage is
correct per project policy — scoped to tab, cleared on close)
- ChatUI.tsx: setInputMessage(prompt) where prompt is a hardcoded literal
- mcp_server_edit.tsx, create_mcp_server.tsx: sessionStorage for OAuth state
- useMcpOAuthFlow.tsx, useUserMcpOAuthFlow.tsx: sessionStorage wrappers
- LoginPage.tsx: localStorage.getItem for worker URL in SSO flow
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: harden npm supply chain — pin overrides, enforce npm ci, add ignore-scripts
Replace open-ended >= version overrides with exact pins matching lockfile
versions across all 6 package.json files. Remove dead overrides for packages
not present in lockfiles. Switch CI and devcontainer from npm install to
npm ci for deterministic lockfile-based installs.
Add .npmrc to all 7 JS project directories with ignore-scripts=true (blocks
postinstall RAT vectors like the axios@1.14.1 supply chain attack) and
min-release-age=3d (refuses packages published <3 days ago, requires npm
>=11.10). Remove Yarn-only resolutions field from docs/my-website.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: bump sharp to 0.33.5 in docs, add docs .npmrc
sharp 0.32.x uses postinstall to download native binaries, which breaks
with ignore-scripts=true. sharp 0.33+ distributes via optionalDependencies
instead, making it compatible with the new .npmrc hardening.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: remove docs .npmrc to fix Vercel deploy
Vercel's build for docs/my-website uses npm install which needs
sharp 0.32.6's postinstall script. Since we don't control Vercel's
build process, remove the .npmrc from docs rather than fight it.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: Dockerfile npm ci + nvm checksum verification
- Replace npm install with npm ci in Dockerfile.non_root,
Dockerfile.custom_ui, and spend-logs/Dockerfile for deterministic
lockfile-based installs
- Replace curl-pipe-bash nvm install with download-then-verify pattern
in build_admin_ui.sh, build_ui.sh, and build_ui_custom_path.sh
- Update nvm from v0.38.0 (2021) to v0.40.4 (Jan 2026) with SHA256
checksum verification before execution
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: macOS sha256sum compat + clarify min-release-age scope
- Use shasum -a 256 fallback on macOS where sha256sum is unavailable
- Clarify in .npmrc comments that min-release-age only protects local
npm install, not npm ci (used in CI)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace Tremor Text with native <span> elements (with block class where
stacking is needed) and Tremor Badge with antd Tag. All Tailwind classes
preserved. Gray badge default mapped to undefined (antd Tag default).
Add setCurrentPage(1) to handleFilterReset in the hook so page resets
regardless of which component consumes it. Replace hardcoded page_size: 50
in empty-state fallbacks with the pageSize parameter for consistency.
- Set backendFilteredLogs to empty response on performSearch error so the
UI shows "0 results" instead of appearing stuck in a loading state
- Cancel debounced search on filter reset instead of firing a request
whose result would be ignored (hasBackendFilters is false after reset)
- Clear backendFilteredLogs immediately on filter change to prevent
stale results from previous filter showing during debounce window
- Normalize response.data with nullish coalescing to handle missing data
- Use `not team_object.models` for None-safe emptiness check
The backendFilteredLogs state initialized to { data: [], ... } which was
indistinguishable from "API returned empty results". When backend filters
were active, the filteredLogs memo fell back to showing unfiltered `logs`
because it couldn't tell whether a search had completed or not.
Fix: use null as initial state so we can distinguish "not yet searched"
(null → show empty placeholder) from "search returned empty" ({ data: [] }
→ show empty results). This prevents the fallback to unfiltered logs that
caused the model filter to display mismatched data.
- Migrate budget CRUD from manual state to React Query hooks (useBudgets, useCreateBudget, useUpdateBudget, useDeleteBudget)
- Fix crash when budget list contains null entries by filtering in query hook
- Fix max_budget type from string to number to match DB schema (double precision)
- Disable budget_id field in edit modal to prevent accidental changes
- Use budget_id as React key instead of array index
- Update tests to mock hooks instead of networking functions
