OAuth2 role mapping is fail-closed by default; unknown IdP roles raise
ValueError instead of silently granting INTERNAL_USER access. Operators
whose IdP returns custom role names not in LitellmUserRoles can set
LITELLM_OAUTH2_UNKNOWN_ROLE_DEFAULT=<role> to map unknown roles to that
fallback. An invalid env value raises on first use so misconfiguration
is loud, not silently downgraded.
Resolves the reviewer split: Veria flagged the original INTERNAL_USER
default as fail-open; Greptile flagged the pure-raise as breaking
deployments with custom IdP role names. The opt-in flag preserves the
secure default and gives operators an explicit escape hatch.
Function-local imports in the new identity module were cargo-culted to
dodge a circular import that does not exist. Move them to module-level.
The one exception is the proxy_server import inside get_identity_cache
which IS a real cycle (proxy_server -> user_api_key_auth -> identity)
and stays lazy.
Test patches that pinned the imported name in the source module move to
the binding in the consuming module, which is correct for module-level
imports anyway.
* feat(azure_ai): add MAI-Image-2.5 image generation support
Route azure_ai MAI models to /mai/v1/images/generations and map OpenAI size to width/height for the serverless API.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(azure_ai): address MAI image generation review feedback
Validate unsupported size values, default width/height independently, add MAI-Image-2.5 pricing, and expand test coverage.
@greptileai
Co-authored-by: Cursor <cursoragent@cursor.com>
* feat(azure_ai): add MAI image edit and expand model cost map
Add MAI image edit support with usage normalization for Azure response format,
and register MAI-Image-2.5-Flash and MAI-Image-2e pricing in the model map.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(azure_ai): validate MAI edit size by consuming map iterator
Greptile: lazy map() never evaluated int() so values like 1024xabc passed through.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(azure_ai): normalize MAI usage in generation response handler
Apply normalize_mai_image_usage before building ImageResponse so token-based
cost calculation works when Azure returns num_output_tokens fields.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(azure_ai): narrow MAI edit size param type for mypy
Co-authored-by: Cursor <cursoragent@cursor.com>
* Fix Azure MAI image response handling
* Fix MAI image generation base model routing
* fix(azure_ai): preserve zero num_output_tokens in MAI usage normalization
* fix(azure_ai): wrap MAI generation response JSON parsing in error handling
* fix(azure_ai): build MAI image edit URL correctly for /mai/ root bases
* fix(azure_ai): build MAI image generation URL correctly for /mai/ root bases
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
delete_verification_tokens now awaits async_delete_cache (via
invalidate_identity_for_token). Tests that patched user_api_key_cache as a
bare MagicMock raised 'MagicMock can't be used in await expression'. Set
async_delete_cache to AsyncMock in the affected key-deletion tests.
delete_verification_tokens (the /key/delete path) cleared the legacy key
cache but never the identity cache, so a revoked key kept authenticating
from its identity:v1:<hash> entry until the TTL expired. Wire
invalidate_identity_for_token into the deletion loop. The other deletion
sites already route through _delete_cache_key_object, which invalidates;
this was the one path that bypassed it (Veria review finding).
An IdP returning an unrecognized role claim was silently mapped to
INTERNAL_USER, granting default access on OAuth2-protected routes.
Raise a ValueError instead so the caller in oauth2_check rejects the
token. The original behavior before this PR also rejected unknown
roles (via Pydantic ValidationError); the fail-closed posture is now
explicit and intentional.
_prepare_introspection_request returns a pre-encoded form body (Authlib
ClientAuth may append client auth to it). The call site passed it through
client.post(data=<str>), which httpx deprecates for strings (it warns
'Use content=<...> to upload raw bytes/text content') and would re-encode
a mapping. Send it as raw content with the form Content-Type already set.
Adds a regression test asserting the introspection POST uses content=
carrying the urlencoded token (Greptile review finding).
Add mutation-quality regression tests asserting that key deletion
(_delete_cache_key_object) invalidates the token identity entry, and
that update_organization and delete_organization bump the org identity
generation counter. These endpoint call sites previously had no test
asserting they invoke invalidate_identity_*, and they were the only
uncovered invalidation lines in the diff (team and user paths are
already covered by existing endpoint tests).
Capture user_id and extra_info from metadata or litellm_metadata. The single-bag read dropped identity whenever a request carried a present litellm_metadata field (null or a user-supplied dict), since /chat/completions routes the authenticated identity into metadata while the guardrail read litellm_metadata first
store.py lives outside proxy/ but is re-exported by litellm/identity/__init__,
so importing the identity package pulled fastapi at module load and broke
SDK-only installs that omit the proxy extras. Use http.HTTPStatus for the
401 code instead; the value is unchanged (Greptile review finding).
Add focused regression tests for the branches the existing unit tests
missed: the object-permission fetch fallback and its swallow path, the
no-DB-connected guard, legacy-cache population and its swallow path, and
bundled-user rehydration failure in store; the non-dict-metadata,
no-generations, unscoped-snapshot and proxy-cache-fallback branches in
cache; resolve_user_api_key_auth hashing in resolver; the missing-peer
path in client; and the attrs/span-attribute helpers in the traced
decorator. Also drop the unused _generation_attr_key helper. Identity
module is now fully covered by its own unit tests.
tests/test_litellm/identity was not in any CI test-path, so the identity
unit tests never ran in CI and their coverage was never recorded. Codecov
therefore reported the identity source as uncovered even though the tests
exist and exercise it. Wire the dir into the misc shard so it runs and
reports coverage.
* feat(vantage): include organization metadata in FOCUS Tags export
Join LiteLLM_OrganizationTable when building Vantage/FOCUS export rows so
organization_id and organization_alias appear in Tags for org-level filtering.
Co-authored-by: Cursor <cursoragent@cursor.com>
* test(focus): include api_requests in organization Tags tests
FocusTransformer now requires api_requests after staging merge; add the
column to test fixtures so integrations CI can run the Tags assertions.
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
A team's BYOK models (rows in LiteLLM_ProxyModelTable with model_info.team_id set)
were left orphaned when the team was deleted; they lingered in the database and kept
showing on the Models + Endpoints page. delete_team now removes them via a new
delete_team_models helper that deletes the rows in one transaction and syncs the
in-memory router only after that transaction commits, run before the team rows are
deleted so a mid-flight failure never leaves the team gone with its models orphaned
_refresh_cached_team now bumps the identity generation counter via
async_increment_cache after every team write. The update_team and
team_model_add/delete unit tests patch user_api_key_cache with a bare
MagicMock, so the new await raised 'MagicMock can't be used in await
expression'. Mirror the existing async_set_cache wiring and make
async_increment_cache an AsyncMock in the affected tests.
Raise the PyJWT floor in pyproject (>=2.13.0,<3.0) and re-resolve uv.lock so
the proxy installs 2.13.0 instead of 2.12.0. Bump the ws transitive-version
override in the dashboard from 8.19.0 to 8.20.1 and regenerate package-lock;
jsdom and openai both dedupe onto the single 8.20.1 copy.
Both are routine dependency maintenance bumps to keep pinned versions current.
* fix(vertex): propagate Vertex AI metadata in streaming success callbacks
Streaming calls assembled via stream_chunk_builder were missing
vertex_ai_grounding_metadata and vertex_ai_url_context_metadata in
standard_logging_object.response. Merge metadata from chunks into the
assembled response and mirror non-streaming hidden_params on Gemini chunks.
Co-authored-by: Cursor <cursoragent@cursor.com>
* refactor(vertex): move streaming metadata merge into provider config hook
Address review feedback by delegating assembled-stream metadata propagation
to VertexGeminiConfig via BaseConfig.apply_assembled_streaming_response_metadata,
and only write chunk hidden_params when metadata is non-empty.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(redaction): scrub Vertex provider metadata when message logging is off
Clear vertex_ai_grounding_metadata and related fields from standard
logging responses and assembled streaming ModelResponse objects so
turn_off_message_logging cannot leak prompt-derived web search queries.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Use assembled model for streaming metadata hook
* Fix Vertex metadata redaction bypass in logging callbacks.
Scrub Vertex provider fields from litellm_params.metadata.hidden_params during perform_redaction so streaming success_handler merges do not leak prompt-derived metadata when message logging is disabled.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Fix Vertex streaming metadata from hidden params
* fix(vertex): mirror vertex_ai_safety_results on assembled streaming responses
The non-streaming transform_response stores safety data under
vertex_ai_safety_results, but the streaming path only wrote
vertex_ai_safety_ratings. Assembled streaming responses therefore never
carried vertex_ai_safety_results, so any consumer reading that field saw
a silent difference between streaming and non-streaming calls.
Set vertex_ai_safety_results alongside vertex_ai_safety_ratings in the
shared stream metadata setter and add it to the assembled metadata field
list so it propagates through stream_chunk_builder.
* fix(streaming): log provider streaming metadata hook failures instead of swallowing them
* refactor(vertex): share single Vertex metadata field tuple across redaction and streaming
* refactor(vertex): move Vertex metadata redaction helpers into llms/vertex_ai
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
glm-5p1 supports native tools on Fireworks; explicit false flags caused
drop_params to strip tools and tool_choice before the provider request.
Co-authored-by: Cursor <cursoragent@cursor.com>
On /team/update for a standalone (no-org) team, _check_user_team_limits()
compared the request max_budget against the caller's personal max_budget
whenever max_budget was present in the payload. A team admin whose personal
budget is lower than the team's budget could not edit any field (tpm_limit,
team name, etc.) because the UI re-sends the unchanged max_budget on every
update, tripping the personal-budget check.
Pass the team's current max_budget into _check_user_team_limits() and skip the
personal-budget comparison when the incoming value is unchanged or lower than
the team's current budget. Only genuine increases above the team's current
budget are still validated against the caller's personal limit, so no
over-relaxation. Proxy admins and the org-scoped path are unaffected.
Adds two regression tests for the standalone update path (unchanged budget +
tpm_limit change, and lowering the budget), both for a caller whose personal
budget is below the team budget.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(jwt-auth): defer to single-team DB fallback on claim mismatch
Extends the single-team DB fallback introduced in #26418 to two more
cases where it previously could not run:
* `find_and_validate_specific_team_id`: when `team_id_jwt_field` is
configured and a claim value is present in the token but the team
does not exist in the LiteLLM DB (HTTPException 404 from
`get_team_object`), return `(None, None)` instead of raising — the
auth_builder fallback then attributes the request to the user's
single DB team. Only HTTPException is caught; other errors (e.g.
"No DB Connected") still propagate.
* `find_team_with_model_access`: when none of the `team_ids_jwt_field`
groups resolve to a real LiteLLM team, return `(None, None)` instead
of raising 403 so the same fallback path runs. If at least one group
DID resolve to a team but none granted the requested model, the
original 403 is preserved (legitimate access denial — not a claim
mismatch). Tracked via the new `any_claim_team_resolved` flag.
The strict `is_required_team_id` raise and `enforce_team_based_model_access`
raise remain unchanged. Unit tests cover both new soft-fail paths and
guard each preserved path (strict required, enforce_team_based, the
preserved 403, and the non-HTTPException propagation).
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(jwt-auth): narrow HTTPException catch to 404 (greptile review)
Address Greptile review comments on #28913:
* `find_and_validate_specific_team_id`: re-raise HTTPException when
`status_code != 404`, pinning the catch to the "team doesn't exist
in db" path documented for `get_team_object`. A future change that
introduces a different status code (e.g. 403 for a blocked team)
will now propagate instead of silently falling through to the
single-team DB fallback.
* Add `test_find_and_validate_specific_team_id_non_404_http_exception_propagates`
parametrised over 400 / 403 / 500 to lock in the contract.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(jwt-auth): gate claim-mismatch fallback behind opt-in flag
The unresolved-team-claim fallback added in the previous commit
weakened the strict claim-based authorization contract by default —
an authenticated user whose JWT carries a stale or invalid team
claim could still consume their single DB team's models/quota via
the fallback.
Gate both soft-fail paths in `find_and_validate_specific_team_id`
and `find_team_with_model_access` behind a new opt-in flag
`team_claim_fallback` on `LiteLLM_JWTAuth` (default False).
Default-off preserves the pre-existing strict behavior. Operators
who intentionally treat IdP team claims as advisory (e.g. machine
tokens whose group claims live in a separate namespace from
LiteLLM team_ids) opt in via config.
Adds two regression tests guarding the default-off behavior.
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(model-management): allow deleting a BYOK model after its team is deleted
A team BYOK model (model_info.team_id set) became undeletable once its team
was deleted: POST /model/delete ran can_user_make_model_call, which looked the
team up and raised 400 "Team id=... does not exist in db" before the delete
could run, so the model lingered on the Models + Endpoints page with no way to
remove it.
Drop the team-existence prerequisite from the delete path. When the model's
team still exists the normal auth check runs unchanged; when it is gone a proxy
admin may delete the orphan and any other caller gets a 403. The check is
fail-closed, so a missing or errored team lookup can only block the delete or
require an admin, never grant a non-admin access. Add/update/health keep their
team-existence validation.
* refactor(model-management): drop redundant team lookup on model delete
Move the orphaned-team handling into can_user_make_model_call behind an
allow_missing_team flag instead of pre-checking team existence in delete_model.
The endpoint no longer issues its own litellm_teamtable lookup, so deleting a
model whose team still exists hits the team table once instead of twice. The
auth behavior is unchanged: a proxy admin can delete a model whose team was
deleted, any other caller gets a 403, and add/update/health keep the strict
"team must exist" validation.
* feat(galileo): add health check support for UI callback test
Register galileo in /health/services so the proxy UI callback connection test works.
Co-authored-by: Cursor <cursoragent@cursor.com>
* feat(galileo): verify API key via /current_user health check
Call Galileo's current_user endpoint so the UI callback test validates credentials against the provider.
Co-authored-by: Cursor <cursoragent@cursor.com>
* chore(ui): regenerate schema.d.ts for galileo health service
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(galileo): return IntegrationHealthCheckStatus from async_health_check
Fixes mypy assignment error in health_services_endpoint where response was
narrowed to IntegrationHealthCheckStatus from earlier branches.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Fix Galileo logging to match Langfuse across all endpoint types.
Stop skipping ingest when output is empty and log embeddings with a placeholder so embedding, speech, and other non-text responses are recorded like Langfuse.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(galileo): remove unreachable health-check guard and None output sentinel
The use_v2_api flag is derived from bool(api_key), so the inner
GALILEO_API_KEY check inside the v2 branch could never run; collapse the
credential validation into the username/password path with a combined
message. _serialize_galileo_output now returns an empty string for None,
so _get_galileo_input_output_content always yields a str and the
post-call None coalescing guard is no longer needed.
* test(galileo): cover async_health_check failure paths and empty model response
Add regression tests for the Galileo health check unhealthy branches
(missing project id, missing base url, missing credentials, auth
failure, and request exception) and for logging a model response with
no choices, which now queues an empty output instead of being skipped.
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(bedrock_mantle): add SigV4/IAM auth to Responses API route (fixes#29665) (#29788)
* feat(responses): add default no-op sign_request to BaseResponsesAPIConfig
* feat(responses): call sign_request after body is final, send signed bytes when signed
* feat(bedrock_mantle): add SigV4 sign_request via composed BaseAWSLLM (bearer path)
* test(bedrock_mantle): cover SigV4 access-key, AssumeRole, body bytes, region/auth consistency
* feat(bedrock_mantle): defer auth to sign_request; validate_environment no longer requires bearer
* docs(bedrock_mantle): document SigV4 + Bearer auth on Responses route
* test(responses): cover fake-stream signing order and mantle bearer arg/env precedence
* fix(bedrock_mantle): wrap all botocore credential errors with both-paths guidance
* fix(bedrock_mantle): catch specific credential errors, not all BotoCoreError, so STS transport failures are not masked
* fix(bedrock_mantle): sign the compact Responses route too, not just create
* fix(github-copilot): route per-model on /v1/responses based on model info (#29747)
* feat(focus): add GCS destination for FOCUS export (#29751)
* test: add failing tests for FocusGCSDestination
* feat: add FocusGCSDestination reusing GCSBucketBase auth
* feat: register FocusGCSDestination in factory; export from __init__
* fix(focus): preserve GCS_PATH_SERVICE_ACCOUNT when service_account_json not in config
* style: apply Black formatting to gcs_destination and tests
* style: apply Black formatting to factory.py
* fix(bedrock): omit empty additionalModelRequestFields and system from Converse API payload (#29565)
Amazon Nova Pro (and other strict Bedrock models) return 400 Malformed input
request when additionalModelRequestFields: {} or system: [] are present in the
payload. Both fields are optional in CommonRequestObject (total=False) and must
be omitted rather than sent as empty structures.
Co-authored-by: shin-berri <shin-laptop@berri.ai>
Co-authored-by: yuneng-jiang <yuneng@berri.ai>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(proxy): recognize *.cognitiveservices.azure.com as OpenAI-compatible in pass-through cost tracking (#29730)
* fix(proxy): recognize *.cognitiveservices.azure.com as OpenAI-compatible
Azure OpenAI resources created via the newer "Azure AI Foundry" /
Cognitive Services pathway live on `*.cognitiveservices.azure.com`
subdomains, not the older `openai.azure.com`. Both are valid Azure
OpenAI surfaces in production today.
The OpenAI pass-through cost-tracking handler hard-codes only the older
hostname in five places (four `is_openai_*_route` methods on
OpenAIPassthroughLoggingHandler, plus is_openai_route on
PassThroughEndpointLogging). As a result, calls from newer Azure
deployments are silently classified as "not an OpenAI route", the
dispatch into the cost-tracking handler is skipped, and tokens/cost
never get extracted into LiteLLM_SpendLogs — the row gets written with
prompt_tokens=0, completion_tokens=0, spend=0, model='unknown'.
Reproduced 2026-06-04 against a real Azure OpenAI deployment on
`*.cognitiveservices.azure.com` proxied through LiteLLM v1.88.0.
Fix: factor the hostname check into a single helper
`_is_openai_compatible_host` listing all three recognized surfaces
(api.openai.com, openai.azure.com, cognitiveservices.azure.com), and
have all five call sites delegate to it. Purely additive — never
weakens recognition for the originally-supported hostnames.
Adds a test
`test_is_openai_route_recognizes_cognitiveservices_azure_com` that
exercises all four `is_openai_*_route` static methods against
`*.cognitiveservices.azure.com` URLs (positive cases per route + a
small cross-route negative to confirm route-specific path matching
still works on the new hostname).
Out of scope for this PR (separate followup):
- `openai_passthrough_handler` calls chat/completions
`transform_response` on Responses API payloads (`output:` not
`choices:`), which throws inside the dispatch and drops the
SpendLogs row entirely. Recognized + tracked separately.
* ci: trigger fresh run
Empty commit to re-run checks. The previous auth-and-jwt failure was
a transient HuggingFace Hub 429 rate-limit hitting tokenizer downloads
in tests/proxy_unit_tests/test_custom_tokenizer_bug.py — unrelated to
this PR's scope (hostname recognition in pass-through cost tracking).
No code change.
---------
Co-authored-by: shin-berri <shin-laptop@berri.ai>
Co-authored-by: yuneng-jiang <yuneng@berri.ai>
* fix(responses): preserve forced-function tool_choice name in Responses to Chat transform (#29812)
The Responses API forces a specific function with a top-level name
({"type": "function", "name": "X"}), but _transform_tool_choice only handled the
nested Chat Completions shape and fell through to returning "required" for the flat
form, silently dropping the function name and degrading a forced function call to
force-any-tool. Map the flat Responses shape to the nested Chat shape, keeping the
"required" fallback when no name is present.
* Preserve x-anthropic-billing-header system blocks for first-party Anthropic (#29584)
* Preserve x-anthropic-billing-header system blocks for first-party Anthropic
PR #20951 strips system blocks beginning with "x-anthropic-billing-header:" for
every Anthropic target. That block is how the first-party Anthropic API recognizes
Claude Code subscription (OAuth) traffic, so dropping it makes requests that carry
only that block, such as the auto-mode tool-safety classifier, fail with a
misleading 429 rate_limit_error; normal turns still work because they also carry
the "You are Claude Code" identity block.
Gate the strip behind should_strip_billing_metadata(), defaulting to False on the
first-party AnthropicConfig and AnthropicMessagesConfig so the block is kept, and
overridden to True on the providers that reach these transforms and reject the
block (Bedrock platform, Vertex, Azure for the chat path; Minimax, Azure, DeepSeek
for the messages path). Behavior for those providers is unchanged.
* Strip billing header on Bedrock invoke and Vertex messages pass-through
Two more subclasses reach the gated strip but inherited keep-by-default.
AmazonAnthropicClaudeConfig (Bedrock invoke) calls AnthropicConfig.transform_request,
which calls translate_system_message, and VertexAIPartnerModelsAnthropicMessagesConfig
(Vertex messages pass-through) calls super().transform_anthropic_messages_request.
Override should_strip_billing_metadata() to True on both.
Add a parametrized test asserting the flag for every first-party base (False) and
provider subclass (True), covering all overrides, plus a translate_system_message
regression test for the Bedrock invoke path.
* fix(cache): log hashed cache keys (#29890)
* fix(ui): save routing groups as list (#29889)
* Revert "fix(ui): save routing groups as list (#29889)" (#29928)
This reverts commit 9b1f78ffa7.
* feat(parasail): add Parasail as a JSON-configured OpenAI-compatible provider (#29842)
* feat(parasail): add Parasail as a JSON-configured OpenAI-compatible provider
Registers parasail in the openai_like JSON provider loader with both
/v1/chat/completions and /v1/responses support. Parasail's Responses API
rejects store:true and any request that omits store, so the loader gains a
force_store_false special_handling flag; the parasail entry sets it and
the generated Responses config overrides store=false on every call. This
keeps callers from hitting "State storage not supported" and matches what
Parasail's docs require.
Adds the PARASAIL enum value, listing under openai_compatible_providers,
provider documentation at docs/my-website/docs/providers/parasail.md, and
a focused unit test file under tests/test_litellm/llms/parasail/ that
covers JSON registration, chat URL construction, Responses URL
construction with PARASAIL_API_BASE override, and the force_store_false
regression in both the caller-sent-store=true and caller-omitted cases.
* fix(parasail): register in provider_endpoints_support, drop in-repo docs
Greptile review feedback. The provider doc belongs in the litellm-docs
repo, not this one's docs/my-website tree; removing it here. Adds the
parasail entry to provider_endpoints_support.json so the
check_provider_folders_documented.py CI check passes (chat_completions
and responses true; others false).
* fix: normalize Anthropic passthrough server tool usage (#29827)
* test(anthropic): cover server_tool_use dict cost tracking
* fix: normalize Anthropic server tool usage
(cherry picked from commit 982f726bed)
* fix: keep server tool usage subscriptable
(cherry picked from commit 70280b9b27)
---------
Co-authored-by: Genmin <joey@joeyroth.com>
* fix(proxy): fix typo generic_role_mappoings -> generic_role_mappings in ui_sso.py (#29753)
Co-authored-by: shin-berri <shin-laptop@berri.ai>
Co-authored-by: yuneng-jiang <yuneng@berri.ai>
* feat(proxy): add disable_budget_reservation general setting (#27639) (#29493)
* feat(proxy): add disable_budget_reservation general setting (#27639)
* feat(proxy): register disable_budget_reservation in ConfigGeneralSettings (#27639)
* docs(proxy): document disable_budget_reservation concurrency tradeoff (#27639)
* ci: re-trigger flaky docker build (prisma generate ECONNRESET)
* fix(proxy): warn and document budget enforcement tradeoff when disable_budget_reservation is set (#27639)
* feat(gemini_tts): adding support to Gemini TTS languageCode parameters (#29623)
* Adding support to Gemini TTS Language Code parameters
* Mapping Gemini TTS languageCode param in Docstring
* Use snake_case for language_code input keyMapping Gemini TTS languageCode param in Docstring
* Restoring files modified under enterprise/litellm_enterprise due to lint/formatting checks
---------
Co-authored-by: João Garrido <joaogarrido@google.com>
* feat(guardrails): capture user and model metadata in CrowdStrike AIDR (#29517)
* fix(proxy): require OpenAI path segment for shared Azure Cognitive Services domains
Address Greptile review: the `*.cognitiveservices.azure.com` /
`*.openai.azure.com` domains are shared by every Azure Cognitive Service
(Speech, Vision, Language, ...), so a hostname-only substring match
misclassified non-OpenAI Azure traffic as OpenAI routes.
- Replace the substring host test with suffix matching (rejects look-alike
domains like cognitiveservices.azure.com.attacker.example).
- Add `_is_openai_compatible_url` that requires an OpenAI-style path marker
(`/openai/` or `/v1/`) on the shared Azure domains, and use it in
PassThroughEndpointLogging.is_openai_route (previously hostname-only).
- Add negative tests for Azure Speech/Vision paths and look-alike domains.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix: support Responses input in Redis semantic cache (#29581)
* fix: support responses input in redis semantic cache
* test: cover redis semantic prompt extraction
* test: handle blank redis semantic text fallbacks
* chore: remove async cache dead statement
* test: cover redis semantic cache miss paths
* fix: filter sensitive cache lookup kwargs
* chore: rerun ci after huggingface rate limit
* chore(ui): regenerate dashboard API types (npm run gen:api)
Sync src/lib/http/schema.d.ts with the proxy OpenAPI spec: adds the
disable_budget_reservation general-settings field and picks up the
RateLimitError docstring reindent. Fixes the gen:api CI drift check.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* test(bedrock): assert empty additionalModelRequestFields is omitted
The Converse transformer now drops an empty additionalModelRequestFields
block instead of sending it as `{}`. Update test_bedrock_top_k_param so
models without top_k support (llama3) assert the key is absent rather than
equal to an empty dict.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Kent <72616338+kingdoooo@users.noreply.github.com>
Co-authored-by: codgician <15964984+codgician@users.noreply.github.com>
Co-authored-by: Praveen Ghuge <95286176+pghuge-cloudwiz@users.noreply.github.com>
Co-authored-by: Roi <roytev@gmail.com>
Co-authored-by: shin-berri <shin-laptop@berri.ai>
Co-authored-by: yuneng-jiang <yuneng@berri.ai>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Liam Scott <liam@uilliam.com>
Co-authored-by: abhay23-AI <abhaytrivedi22@gmail.com>
Co-authored-by: Ceder Dens <cederdens@gmail.com>
Co-authored-by: 冯基魁 <56265583+fengjikui@users.noreply.github.com>
Co-authored-by: Kai Huang <kaihuang724@gmail.com>
Co-authored-by: rinto <54238243+ririnto@users.noreply.github.com>
Co-authored-by: Genmin <joey@joeyroth.com>
Co-authored-by: Arnav Bhilwariya <arnavbhilwariya0408@gmail.com>
Co-authored-by: Armaan Sandhu <74664101+Ar-maan05@users.noreply.github.com>
Co-authored-by: João Garrido <48538534+johngarrido@users.noreply.github.com>
Co-authored-by: João Garrido <joaogarrido@google.com>
Co-authored-by: Kenan Yildirim <kenan@kenany.me>
Co-authored-by: Dávid Balatoni <balcsida@gmail.com>
* fix(ui): let non-creator users OAuth into OBO-mode MCP servers from the Tools page
* fix(ui): clear OBO Tools-tab one-shot on navigate-back and gate on credential-status errors
* fix(ui): unify migrated-route URLs and cut the API Reference page over to path routing
Route all migrated-page navigation through one /ui-prefixed, serverRootPath-aware
builder in migratedPages.ts (migratedHref/legacyPageHref/legacyKeyForPathname),
replacing the three divergent base-URL constructions that lived in the dashboard
layout's withBase, leftnav, and the page.tsx redirect. The previous migratedHref
read NEXT_PUBLIC_BASE_URL, which no build sets, so it produced URLs without the
/ui prefix the app is served under; every other internal link hardcodes /ui and
this now matches that convention.
Remove the sidebar's own pushState navigation so the parent (legacy root page or
dashboard layout) is the single owner of navigation, fixing the double-navigate
that fired when moving between a path route and a legacy ?page= route.
Cut API Reference over to its path route: add api_ref -> api-reference to
MIGRATED_PAGES and delete its arm from the legacy switch. Visiting
/ui/?page=api_ref redirects to /ui/api-reference, the sidebar links to and
highlights it, and navigating away returns to the legacy switch.
* fix(ui): address review on migrated-page routing
Keep the legacy hyphenated ?page=api-reference form working by mapping it to the
api-reference route alongside api_ref; the old switch matched both, so a bookmark
using the hyphen would otherwise fall through to the Usage default.
Add legacyKeyForPathname coverage: a migrated path (with and without trailing
slash) resolves to the api_ref sidebar key rather than the alias, a non-migrated
path returns null, and a non-root serverRootPath prefix is stripped before matching.
* fix(ui): populate serverRootPath from getUiConfig so migrated nav links keep the root path
getUiConfig updated proxyBaseUrl but never called updateServerRootPath, so the
module-level serverRootPath stayed at its "/" default. Under a custom
server_root_path the unified migratedHref/legacyPageHref builders then dropped
the prefix and the sidebar produced /ui/api-reference (404) instead of
/<root>/ui/api-reference. Adds the missing updateServerRootPath call plus a
regression test asserting getUiConfig sets serverRootPath and that migratedHref
carries the prefix
The key edit page showed the default key type (no allowed_routes restriction)
as "Default", while the key creation form already labels the same value
"Full Access". Align the edit page to the create form so the two surfaces agree
on both the label and its description.
The create guardrail modal used antd's default maskClosable, so clicking
outside it dismissed the modal and reset every field the user had entered.
Setting maskClosable={false} keeps the modal open; it now closes only via
the explicit close button or Cancel, matching the other form modals in the
dashboard
