- Add _verify_org_access to deprecated POST /organization/info endpoint
- Move get_user_object to module-level import in organization_endpoints.py
- Add tests for _verify_team_access 403 denial path
* build: migrate packaging metadata to uv
* ci: move automation and local tooling to uv
* docker: migrate image builds and runtime setup to uv
* docs: update install and deployment guidance for uv
* chore: align auxiliary scripts and tests with uv
* test: harden test_litellm isolation
* fix: keep release and health check images self-contained
* build: pin uv tooling and health check deps
* test: isolate bedrock image request formatting from suite state
* test: cover sandbox executor requirements flow
* ci: fix circleci no-op command steps
* ci: fix circleci publish workflow parsing
* fix: stabilize remaining uv migration CI checks
* ci: increase matrix test timeout headroom
* fix: restore published docker and license coverage
* fix: restore proxy runtime build parity
* fix: restore proxy extras parity and venv migrations
* ci: persist uv path across circleci steps
* fix: keep psycopg binary in default test env
* docker: preserve prisma cache across stages
* test: run local proxy checks through uv python
* build: restore runtime deps moved into ci
* build: refresh uv lock after upstream merge
* fix: restore module import in test_check_migration after merge
The conflict resolution imported only the function but the test body
references check_migration as a module throughout.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: revert dependency promotions, remove nodejs-wheel-binaries, fix Docker layer caching
- Move google-generativeai, Pillow, tenacity back to ci group (they are
lazily imported and bloat the base SDK install needlessly)
- Remove nodejs-wheel-binaries from extra_proxy and proxy-dev (redundant
in Docker where system Node.js is already installed via apk)
- Remove all nodejs-wheel node replacement and venv npm patching blocks
from Dockerfiles since the wheel is no longer installed
- Add --no-default-groups to CodSpeed benchmark workflow so the benchmark
environment matches the old minimal pip install footprint
- Apply standard uv two-phase Docker pattern: copy metadata first, install
deps (cached layer), then copy source and install project
- Replace CircleCI enterprise no-op with proper uv sync command
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: regenerate uv.lock after removing nodejs-wheel-binaries
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(ci): use cache/restore instead of cache to prevent cache poisoning
The old workflow used actions/cache/restore (read-only). The uv migration
changed it to actions/cache (read-write), which zizmor flags as a cache
poisoning risk. Restore the safer read-only variant.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(ci): disable setup-uv built-in cache to silence cache-poisoning alert
The setup-uv action enables caching by default, which zizmor flags as a
cache poisoning risk. Disable it since we already use a read-only
cache/restore step.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(ci): disable setup-uv cache in publish workflow
Silences zizmor cache-poisoning alert. Publishing workflow runs
infrequently on protected branches so caching adds no real benefit.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(test): remove duplicate verbose_logger mock in test_check_migration
The logger was patched twice — first via mocker.patch() then via
mocker.patch.object(autospec=True). The second call fails because
autospec cannot inspect an already-mocked attribute. Remove the
redundant first patch.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(ci): free disk space before Docker build in test-server-root-path
The Dockerfile.non_root build ran out of disk on the CI runner. Remove
Android SDK, .NET, Boost, and GHC toolchains (~12GB) to free space.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Adds a new endpoint to bulk-update team_member_permissions across
teams. Supports apply_to_all_teams (with cursor-based pagination)
or a specific list of team_ids. Merges new permissions into each
team's existing set rather than overwriting.
Also fixes test isolation bug in test_get_prompt_info_by_base_id
where leaked prisma_client state from other tests caused a
TypeError on await.
Tests cover: matching deployments, wrong team_id filtering, string-encoded
model_info, empty results, invalid model_info, and mixed deployment filtering.
Also updates MockPrismaClient.find_many to support the new startswith query.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace per-ID _resolve_access_group_resources loop with a single
find_many call that deduplicates IDs across all teams. Removes the
N+1 query pattern on cold cache for the team list endpoint.
Three tests were patching the non-existent `get_access_object` instead
of `_get_access_object` (the lazy-import wrapper), causing AttributeError.
Also added missing `prisma_client` mock so tests get past the early-exit
guard and actually exercise the resolution logic.
* Litellm ishaan april1 (#25103)
* fix(proxy): enforce upperbound key params on key/update and add custom_key_update hook
The /key/update endpoint did not enforce upperbound_key_generate_params,
allowing users to bypass configured limits (tpm_limit, rpm_limit,
max_budget, duration, budget_duration) by updating an existing key
instead of generating a new one.
Extract the upperbound enforcement logic from _common_key_generation_helper()
into a standalone _enforce_upperbound_key_params() function and call it from
both the generate and update paths. For updates, None values are skipped
(not filled with defaults) since they mean "don't change this field".
Also adds a custom_key_update config option and user_custom_key_update global,
mirroring the existing custom_key_generate pattern, so custom key validation
logic can fire during key updates as well.
* fix(proxy): invoke custom_key_update hook in bulk update path
The user_custom_key_update hook was only called in update_key_fn
(single key update) but not in _process_single_key_update (bulk
update path), allowing custom validation to be bypassed via the
/key/update/bulk endpoint. Mirror the hook invocation in both paths.
* fix(proxy): pass UpdateKeyRequest to hook in bulk path, not BulkUpdateKeyRequestItem
Move the custom_key_update hook invocation to after UpdateKeyRequest
is constructed so the hook receives the same type in both single and
bulk update paths. Previously the bulk path passed
BulkUpdateKeyRequestItem (5 fields only), which would cause
AttributeError for hooks accessing fields like tpm_limit or models.
* fix(bedrock): promote cache usage to message_delta for Claude Code (#24850)
Ensure Bedrock/Anthropic-compatible streaming exposes cache usage where Claude Code reads it by promoting message_stop usage onto message_delta and preserving usage fields in fake-streamed message_delta events.
Made-with: Cursor
* fix(search): Support self-hosted Firecrawl response format in search transform (#24866)
The `transform_search_response` method only handled Firecrawl Cloud (v2)
response format where `data` is a dict with `web`/`news` keys. Self-hosted
Firecrawl (v1) returns `data` as a flat list of result objects, causing an
`AttributeError: 'list' object has no attribute 'get'`.
Detect the response format by checking if `data` is a list (self-hosted)
or dict (cloud) and handle both cases.
Cloud format: {"data": {"web": [...], "news": [...]}}
Self-hosted: {"success": true, "data": [{"url": "...", "title": "...", ...}]}
Co-authored-by: Synergy <synergyoclaw@gmail.com>
* feat: add environment and user tracking to prompt management (#24855)
* feat: add environment and user tracking to prompt management
- Add environment (development/staging/production) and created_by columns to LiteLLM_PromptTable
- Update unique constraint to [prompt_id, version, environment]
- All CRUD endpoints support environment filtering and user tracking
- Redesigned prompt detail page with environment tabs and version history
- UI: environment filter on list page, environment selector in editor
- 8 new tests for environment and user tracking
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: Black formatting and add environments to PromptInfoResponse TypeScript type
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: address Greptile review findings
- P1: delete_prompt scopes in-memory cleanup to environment when provided
- P2: dotprompt_content parsed directly regardless of environment flag
- P2: use distinct for environments query
- P2: fix double-fetch on initial mount in prompt_info.tsx
- fix: remove unsupported select kwarg from find_many
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: address remaining Greptile review comments
- Remove unused useCallback import (index.tsx)
- Remove unused ENV_COLORS variable (prompt_info.tsx)
- P1: in-memory fallback in get_prompt_versions now respects environment filter
- P1: reset selectedEnv when promptId changes to avoid stale state
- Cyclic imports are pre-existing pattern, not introduced by this PR
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: scope patch_prompt to environment using primary key
- Add environment query param to patch_prompt endpoint
- Look up target row by composite key (prompt_id + version + environment)
- Update by primary key (id) to target exactly one row
- Fixes Greptile finding: patch with multiple environments no longer ambiguous
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: use actual start_time for failed request spend logs (#24906)
async_post_call_failure_hook set both start_time and end_time to
datetime.now(), making all failed requests show duration=0. Use the
actual start_time from litellm_logging_obj instead, so spend logs
reflect the real request duration on timeout and other failures.
Fixes#24888
* feat(bedrock): add nova canvas image edit support (#24869)
* feat(bedrock): add nova canvas image edit support
* fix(bedrock): support PathLike inputs for nova image edit
* chore: sync schema.prisma copies from root
* fix(mypy): correct type-ignore code for delta_usage arg-type
* fix(mypy): cast status_code to str, suppress intentional str yield
* fix(lint): extract _create_content_block_chunks to fix PLR0915
* fix(lint): extract helpers to fix PLR0915 in prompt endpoints
---------
Co-authored-by: michelligabriele <gabriele.michelli@icloud.com>
Co-authored-by: Sameer Kankute <sameer@berri.ai>
Co-authored-by: redhelix <amin.lalji@gmail.com>
Co-authored-by: Synergy <synergyoclaw@gmail.com>
Co-authored-by: Talha Anwar <37379131+talhaanwarch@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: madhu19991 <madhu@thunkai.com>
Co-authored-by: Srikanth @adobe <devarakondasrikanth@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* fix(test): update model armor streaming test to handle string or int error code
---------
Co-authored-by: michelligabriele <gabriele.michelli@icloud.com>
Co-authored-by: Sameer Kankute <sameer@berri.ai>
Co-authored-by: redhelix <amin.lalji@gmail.com>
Co-authored-by: Synergy <synergyoclaw@gmail.com>
Co-authored-by: Talha Anwar <37379131+talhaanwarch@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: madhu19991 <madhu@thunkai.com>
Co-authored-by: Srikanth @adobe <devarakondasrikanth@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
- Fetch each access group object once and extract all 3 resource fields
in a single pass instead of 3 separate calls (3N → N lookups)
- Use asyncio.gather to resolve access groups across teams concurrently
in list_team_v2 instead of sequential awaits
- Add 5 unit tests for _resolve_access_group_resources
* update bedrock models in tests
* updated more tests and model_prices_and_context_window
* fix model id and pricing
* replace more sonnet models
* update tests
* git push
* update pricing
* flaky total cost
* monkey patch
* relax the cost change
* fix and revert some changes
* revert the pricing
* chore: move cost/pricing changes to bedrock-cost-fixes branch
* chore: split Bedrock file-api beta stripping to separate branch
Removes strip_unsupported_file_api_betas_for_bedrock_invoke from this branch;
see litellm_bedrock_invoke_strip_file_api_betas for that fix.
Made-with: Cursor
CI fails because LITELLM_MASTER_KEY is not set, causing
encrypt_value_helper to error on NoneType.encode(). Patch it
to pass through values unencrypted in the test.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Skip the integration test that requires a live proxy and OPENAI_API_KEY
(removed from CI/CD). Add deterministic mock test covering the same
add → delete → double-delete-fails lifecycle.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Substring matching for user_id and key_alias on /key/list is now gated
by caller role (admin only) rather than by whether user_id was auto-set.
This closes the gap where a non-admin explicitly passing their own
user_id still got contains-based filtering, which could surface keys
belonging to other users with overlapping IDs.
Adds three unit tests covering the substring, exact-match, and default
code paths in _build_key_filter_conditions. Restores the existing
member-team test to assert exact matching as a security regression guard.
Substring matching for user_id and key_alias now applies unconditionally.
The admin-only flag was unnecessary since user_ids are typically UUIDs/emails
where substring collisions are not a practical concern. Updated existing test
to assert the new filter structure.
During SSO login, bearer tokens are stripped from the OAuth response
before role mapping runs. Custom role claims encoded inside the JWT
access token are lost, so map_jwt_role_to_litellm_role() returns None
and the user falls back to internal_user_viewer.
process_sso_jwt_access_token() now returns the decoded JWT payload, and
a new _sync_user_role_from_jwt_role_map() receives it so
jwt_litellm_role_map works correctly during SSO login.
Add input validation to get_user_id_from_request (length limit, control char rejection) and a blocklist of dangerous environment variable keys in _load_environment_variables to prevent PATH/LD_PRELOAD/PYTHONPATH override via config.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Cache LITELLM_ENABLE_TEAM_STALE_ALIAS_BYPASS at module level to avoid hot-path secret lookups
- Add clarifying comments for should_include_deployment team isolation logic
- Add negative assertion for update_team.assert_not_called() in test
- Add docstring clarification for _get_team_deployments helper pattern
- Add explicit assertion message in test_get_model_list_alias_optimization
Made-with: Cursor
- Use O(1) team index lookup instead of map_team_model in alias guard
- Fix MockPrismaClient to validate where clause filters
- Add comment explaining DB query trade-off for team deployments
Made-with: Cursor
- Skip model_aliases rewrite if model resolves to team deployments
- Add test coverage for sibling-preservation branch
- Update MockPrismaClient to support sibling deployment scenarios
Made-with: Cursor
- Add clarifying comments to test assertions
- Query prisma DB instead of in-memory router to avoid stale state
- Prevents incorrect deletion of old public name when siblings exist
Made-with: Cursor
Remove team model_alias rewrites and resolve team deployments by team_public_model_name with team_id so sibling deployments stay in the routing candidate pool, with explicit logs showing candidate selection before load balancing.
Made-with: Cursor
Use a deterministic internal model_name for team-scoped deployments so sibling deployments with the same public model share a routing group. This makes team alias writes idempotent and preserves multi-deployment failover/load balancing behavior.
Made-with: Cursor
Instead of returning a 400 error when return_to is passed without
control_plane_url configured, silently ignore it and proceed with
the normal same-origin SSO flow.
Adds a control plane capability that enables a central admin instance
to manage multiple regional worker proxies from a single UI.
Backend:
- Worker registry loaded from YAML config (worker_id, name, url)
- /.well-known/litellm-ui-config exposes is_control_plane and workers list
- /v3/login + /v3/login/exchange: opaque code exchange for cross-origin
username/password auth (JWT never in URL/logs, single-use 60s TTL)
- SSO cookie handoff with return_to → opaque code → exchange
- _validate_return_to: full origin validation (scheme+hostname+port)
- Startup warning when control_plane_url set without Redis
- Both /v3 endpoints gated behind control_plane_url config
Frontend:
- Worker selector dropdown on login page (gated behind is_control_plane)
- Cross-origin SSO code exchange handling on callback
- switchToWorkerUrl: localStorage-persisted worker URL for API calls
- useWorker hook: shared worker state management
- WorkerDropdown in navbar for switching workers
- Logout/switch clears worker state from localStorage
Tests:
- 7 tests for /v3/login + /v3/login/exchange
- 10 tests for _validate_return_to
- 2 tests for control plane discovery endpoint
Tests were outdated after _get_and_validate_existing_key was refactored
to use prisma_client.db.litellm_verificationtoken.find_unique() and
ProxyException. Also add ProxyException handling in bulk_update_keys
error extractor so error messages aren't empty.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Tests were outdated after _get_and_validate_existing_key was refactored
to use prisma_client.db.litellm_verificationtoken.find_unique() instead
of prisma_client.get_data(), and to raise ProxyException instead of
HTTPException. Also fix bulk_update_keys error handler to extract
ProxyException.message (str(ProxyException) returns empty string).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* add DD Tracing (#24033)
* feat(models): add Azure GPT-5.4 mini and nano variants (#24045)
Add `azure/gpt-5.4-mini` and `azure/gpt-5.4-nano` to the model
database with official pricing from Azure OpenAI:
- GPT-5.4 mini: $0.75/M input, $0.075/M cached, $4.5/M output
- GPT-5.4 nano: $0.20/M input, $0.02/M cached, $1.25/M output
Both models support:
- 1.05M input / 128K output context window
- Chat, batch, and responses endpoints
- Function calling, tools, vision, reasoning
- Prompt caching with automatic tiered pricing
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* Add new model pricing details for volcengine Doubao-Seed-2.0 series (#23871)
Add entries for volcengine Doubao-Seed-2.0 series
* fix(mcp): support refresh_token grant type in OAuth token endpoint (#23701)
* fix(mcp): support refresh_token grant type in OAuth token endpoint (#23700)
The .well-known/oauth-authorization-server metadata advertises
refresh_token as a supported grant type, but the token endpoint
rejected it with HTTP 400. This adds refresh_token grant support
so MCP clients can refresh expired tokens without re-authenticating.
* test(mcp): add tests for refresh_token grant type in OAuth token endpoint
* fix(mcp): move code_verifier guard into authorization_code branch
code_verifier is only relevant for authorization_code grants (PKCE).
Move it inside the else branch so it doesn't apply to refresh_token.
* fix(mcp): guard None client_secret and forward scope in token exchange
- Conditionally include client_secret in form data to prevent httpx
from sending the literal string "None" (applies to both
authorization_code and refresh_token branches)
- Forward optional scope parameter per RFC 6749 §6, allowing clients
to request a subset of originally-granted scopes on refresh
* fix(mcp): validate code param in authorization_code grant
Guard against None code being form-encoded as literal string "None"
by httpx, symmetric with the existing refresh_token guard.
* docs: add incident report for guardrail logging secret exposure (#24059)
Add blog post documenting the guardrail logging path exposing internal
request data (e.g. Authorization headers) in spend logs and OTEL traces.
Fix available in LiteLLM 1.82.3+.
Made-with: Cursor
* [Fix] Datadog LLM Observability tags format (env, service, version missing) (#23673)
* tag fix
* greptile comment
* fix(ci): stabilize 6 failing CI jobs
1. mypy: remove duplicate type annotation for token_data in discoverable_endpoints.py
2. integrations tests: add parameterized to CI test deps
3. doc quality: document OTEL_IGNORE_CONTEXT_PROPAGATION env key
4. security: allowlist CVE-2026-2673, CVE-2026-3644, CVE-2026-4224 (no fix available)
5. proxy_store_model_in_db: fix missing x-litellm-call-id header on error responses
6. google tests: add --retries 3 for transient Vertex AI rate limits
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* fix(streaming): handle RuntimeError during model_copy in streaming handler
The race condition occurs when model_copy(deep=True) tries to deepcopy
_hidden_params dict while it's being concurrently modified by logging
callbacks. Fall back to shallow copy if the deep copy fails.
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* fix(cost): handle non-string traffic_type in cost calculator + add retries
1. Fix AttributeError in _map_traffic_type_to_service_tier when traffic_type
is an integer (cast to str before calling .upper()). This was causing
pass-through vertex spend logging to fail silently.
2. Add --retries to llm_translation_testing for flaky external API calls.
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
---------
Co-authored-by: Emerson Gomes <emerson.gomes@thalesgroup.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: ExMatics HydrogenC <33123710+HydrogenC@users.noreply.github.com>
Co-authored-by: Jack Venberg <jack.venberg@rover.com>
Co-authored-by: milan-berri <milan@berri.ai>
Co-authored-by: Shivam Rawat <161387515+shivamrawat1@users.noreply.github.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
- Deduplicate: update_key_fn now delegates to _get_and_validate_existing_key()
instead of inlining its own copy of the lookup logic
- Use _hash_token_if_needed (already imported at module level) instead of
inline `from proxy_server import hash_token` + manual conditional
- Fix stale docstring: _get_and_validate_existing_key raises ProxyException,
not HTTPException
- Add unit test: test_update_key_nonexistent_key_returns_404
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Incorporate new _check_key_admin_access() calls from the base branch
into block_key/unblock_key alongside our existence-check fix.
Update test mocks: replace references to removed get_key_object and
_cache_key_object with _delete_cache_key_object in both the shared
_setup_block_unblock_mocks helper and individual test functions.
Co-authored-by: yuneng-jiang <yuneng-jiang@users.noreply.github.com>
These were only used in block_key/unblock_key for cache refresh,
which now uses _delete_cache_key_object instead.
Co-authored-by: yuneng-jiang <yuneng-jiang@users.noreply.github.com>
- test_block_key_nonexistent_key_returns_404: verifies block_key returns
404 (not misleading 401) when the key doesn't exist in the DB
- test_unblock_key_nonexistent_key_returns_404: same for unblock_key
- test_block_key_existing_key_succeeds: verifies block_key succeeds and
invalidates cache for existing keys
- Update test_unblock_key_supports_both_sk_and_hashed_tokens to reflect
the new cache invalidation pattern (_delete_cache_key_object instead
of get_key_object + _cache_key_object)
Co-authored-by: yuneng-jiang <yuneng-jiang@users.noreply.github.com>
