* add DD Tracing (#24033)
* feat(models): add Azure GPT-5.4 mini and nano variants (#24045)
Add `azure/gpt-5.4-mini` and `azure/gpt-5.4-nano` to the model
database with official pricing from Azure OpenAI:
- GPT-5.4 mini: $0.75/M input, $0.075/M cached, $4.5/M output
- GPT-5.4 nano: $0.20/M input, $0.02/M cached, $1.25/M output
Both models support:
- 1.05M input / 128K output context window
- Chat, batch, and responses endpoints
- Function calling, tools, vision, reasoning
- Prompt caching with automatic tiered pricing
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* Add new model pricing details for volcengine Doubao-Seed-2.0 series (#23871)
Add entries for volcengine Doubao-Seed-2.0 series
* fix(mcp): support refresh_token grant type in OAuth token endpoint (#23701)
* fix(mcp): support refresh_token grant type in OAuth token endpoint (#23700)
The .well-known/oauth-authorization-server metadata advertises
refresh_token as a supported grant type, but the token endpoint
rejected it with HTTP 400. This adds refresh_token grant support
so MCP clients can refresh expired tokens without re-authenticating.
* test(mcp): add tests for refresh_token grant type in OAuth token endpoint
* fix(mcp): move code_verifier guard into authorization_code branch
code_verifier is only relevant for authorization_code grants (PKCE).
Move it inside the else branch so it doesn't apply to refresh_token.
* fix(mcp): guard None client_secret and forward scope in token exchange
- Conditionally include client_secret in form data to prevent httpx
from sending the literal string "None" (applies to both
authorization_code and refresh_token branches)
- Forward optional scope parameter per RFC 6749 §6, allowing clients
to request a subset of originally-granted scopes on refresh
* fix(mcp): validate code param in authorization_code grant
Guard against None code being form-encoded as literal string "None"
by httpx, symmetric with the existing refresh_token guard.
* docs: add incident report for guardrail logging secret exposure (#24059)
Add blog post documenting the guardrail logging path exposing internal
request data (e.g. Authorization headers) in spend logs and OTEL traces.
Fix available in LiteLLM 1.82.3+.
Made-with: Cursor
* [Fix] Datadog LLM Observability tags format (env, service, version missing) (#23673)
* tag fix
* greptile comment
* fix(ci): stabilize 6 failing CI jobs
1. mypy: remove duplicate type annotation for token_data in discoverable_endpoints.py
2. integrations tests: add parameterized to CI test deps
3. doc quality: document OTEL_IGNORE_CONTEXT_PROPAGATION env key
4. security: allowlist CVE-2026-2673, CVE-2026-3644, CVE-2026-4224 (no fix available)
5. proxy_store_model_in_db: fix missing x-litellm-call-id header on error responses
6. google tests: add --retries 3 for transient Vertex AI rate limits
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* fix(streaming): handle RuntimeError during model_copy in streaming handler
The race condition occurs when model_copy(deep=True) tries to deepcopy
_hidden_params dict while it's being concurrently modified by logging
callbacks. Fall back to shallow copy if the deep copy fails.
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* fix(cost): handle non-string traffic_type in cost calculator + add retries
1. Fix AttributeError in _map_traffic_type_to_service_tier when traffic_type
is an integer (cast to str before calling .upper()). This was causing
pass-through vertex spend logging to fail silently.
2. Add --retries to llm_translation_testing for flaky external API calls.
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
---------
Co-authored-by: Emerson Gomes <emerson.gomes@thalesgroup.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: ExMatics HydrogenC <33123710+HydrogenC@users.noreply.github.com>
Co-authored-by: Jack Venberg <jack.venberg@rover.com>
Co-authored-by: milan-berri <milan@berri.ai>
Co-authored-by: Shivam Rawat <161387515+shivamrawat1@users.noreply.github.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
On /key/block, /key/unblock, and /key/update, the request body 'key'
field could contaminate the api_key Security dependency, causing the
auth layer to authenticate against the target key instead of the
caller's bearer token. This returned 401 for a nonexistent body key
even when the Authorization header contained a valid master key.
Added a guard in user_api_key_auth that re-reads the Authorization
header directly from the request, ensuring the header is always the
authoritative source for authentication.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Document intentional legacy-matching behavior: when user_id is
provided to an org admin, no org filter is applied (returns all of
that user's teams across all orgs, same as legacy endpoint)
- Fix two existing security tests to properly patch user_api_key_cache,
proxy_logging_obj, and get_user_object instead of relying on
incidental error handling
- Add three new org admin test cases:
- Org admin sees org-scoped teams (200 with correct where clause)
- Org admin rejected when filtering by other org (403)
- Org admin with user_id filter returns target user's teams
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Replace raw find_unique with get_user_object in
_build_team_list_where_conditions for cache/metrics consistency
- Remove over-complex OR clause for org admin + user_id: when user_id
is provided, filter by that user's direct team memberships (same as
regular users) since the access control gate already verified the
org admin's authority
- Preserve caller-supplied organization_id instead of overwriting with
org_admin_org_ids
- Update test mock to match get_user_object call path
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add disable_custom_api_keys UI setting that prevents users from specifying
custom key values during key generation and regeneration. When enabled, all
keys must be auto-generated, eliminating the risk of key hash collisions
in multi-tenant environments.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When updating or regenerating a key without changing its key_alias, the
existing alias was being re-validated against current format rules. This
caused keys with legacy aliases (created before stricter validation) to
become uneditable. Now validation only runs when the alias actually changes.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move the non-admin team validation into the existing get_team_object call
site to avoid an extra DB round-trip. The existing call already fetches
the team for limits checking — we now add the LIT-1884 guard there when
team_obj is None for non-admin callers.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Internal users could exploit key/generate and key/update to create unbound
keys (no user_id, no budget) or attach keys to non-existent teams. This
adds validation for non-admin callers: auto-assign user_id on generate,
reject invalid team_ids, and prevent removing user_id on update.
Closes LIT-1884
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Non-admin users (INTERNAL_USER) could call /key/block and /key/unblock on
arbitrary keys, and modify max_budget on their own keys via /key/update.
These endpoints are now restricted to proxy admins, team admins, or org admins.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When scope_user_search_to_org flag is ON, team admins (non-org-admins) were
getting 403 because the code only checked for ORG_ADMIN role in org memberships.
Now checks all org memberships (any role) and falls back to the API key's team_id
to resolve the org.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Rewrite Default Team Settings UI from Tremor to antd with hardcoded fields,
fix default team params not applying during team creation or persisting
across proxy restarts, remove dead code, and add comprehensive tests.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Use `min`/`max` instead of `_min`/`_max` for Prisma group_by input
parameters. The underscore-prefixed names are output keys, not input
kwargs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Verifies that validate_key_team_change does not call can_team_access_model
for the "all-team-models" sentinel, allowing keys with that value to be
moved between teams without model validation failures.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The self-exclusion filter compared raw key strings against SHA-256
hashed tokens from the DB, so keys were never excluded and
double-counting persisted. Now hash data.key before comparison.
Also add tpm_limit_type/rpm_limit_type to _throughput_fields_changed
guard, fall back to existing_key_row.team_id for team limit checks
(matching the org pattern), and add team self-exclusion test.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Only run org validation (get_org_object + _check_org_key_limits) when
the update actually touches throughput-related fields (tpm_limit,
rpm_limit, or organization_id). Previously, any update to a key
belonging to an org would trigger the check, which would fail with a
400 if the org had been deleted — blocking unrelated field changes.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When updating a key, _check_org_key_limits and _check_team_key_limits
would include the key being updated in the find_many results, causing
its current limits to be counted twice (once from the DB query, once
from the new requested limits). This caused false 400 errors on valid
limit adjustments.
Fix: exclude the key being updated (by matching token) from the
allocated totals before checking limits.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The previous fix (124b44ec) only updated 3 tests but missed 10 more
that still patched the old `ui_sso.httpx.AsyncClient` path. Also
updated credential assertions to check Authorization header instead
of httpx.BasicAuth kwargs, matching the production code change.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The /key/update endpoint was missing support for organization_id, which
was already available on /key/generate. This adds the field to
UpdateKeyRequest and validates org key limits during updates.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The test_chat_completion_low_budget test was flaky because async spend
tracking couldn't reliably catch up within 50 calls with 0.5s sleeps.
Increased to 200 calls with 0.1s sleeps (same total time budget) to
give more opportunities for budget enforcement to trigger.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The recent commit 2a997993d4 replaced httpx.AsyncClient() with
get_async_httpx_client() in ui_sso.py, but the PKCE tests still
patched the old httpx.AsyncClient path. Updated all 10 affected
tests to mock get_async_httpx_client and removed unnecessary
context manager setup since AsyncHTTPHandler is returned directly.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Vertex AI batch cost tests: replace removed gemini-1.5-flash-001 model
with gemini-2.0-flash-001 in pricing lookups
- MCP test_executes_tool_when_allowed: add server_id and auth_type attrs
to StubServer to match new _resolve_allowed_mcp_servers_with_ip_filter
- MCP M2M tests: infer oauth2_flow='client_credentials' in
_execute_with_mcp_client when client_id/client_secret/token_url present
(NewMCPServerRequest lacks oauth2_flow field)
- Team list test: update mock find_many to filter by team_id per the
current per-team query pattern in list_team
- Azure DALL-E 3 health check: skip test due to 410 ModelDeprecated
Co-authored-by: yuneng-jiang <yuneng-jiang@users.noreply.github.com>
Apply the same fix: move litellm.public_model_groups_links assignment
to after get_config()/save_config() so it is not overwritten by the
stale DB value read.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
get_config() internally calls _update_config_from_db which overwrites
litellm.public_model_groups with the stale DB value. Moving the
in-memory assignment to after get_config()/save_config() ensures the
new value persists.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(sso): add direct PKCE token exchange and Redis cache wiring for multi-instance SSO
When PKCE is enabled, bypass fastapi-sso and perform direct token exchange so
code_verifier is correctly included. Store PKCE verifiers as dict in cache
for proper JSON serialization in Redis. Wire user_api_key_cache to Redis when
available so PKCE verifiers are shared across ECS tasks/pods.
Also adds clearer error messages when PKCE is required but not configured.
* refactor(sso): extract PKCE token exchange into SSOAuthenticationHandler methods
- Move import httpx/jwt to module level (top of file, not inside function)
- Extract inline PKCE token exchange + userinfo logic into two static methods:
_pkce_token_exchange() and _get_pkce_userinfo()
- get_generic_sso_response PKCE path is now a single method call
- Fix double-logging in except block for non-PKCE errors
- Use %-style log formatting (no f-strings in log calls)
* fix: address greptile review feedback
- Fix access_token missing in PKCE path: read from combined_response directly
instead of generic_sso.access_token (which is only set by verify_and_process)
- Fix PKCE error hint firing when PKCE is already enabled: only show
'set GENERIC_CLIENT_USE_PKCE=true' advice when code_verifier was absent
- Fix unguarded KeyError on access_token: check for error field in HTTP 200
responses before accessing token_response['access_token']
- Fix silent empty userinfo: raise ProxyException when both userinfo endpoint
and id_token fallback produce no user data
- Fix backward-incompatible Redis wiring: only attach Redis to user_api_key_cache
when GENERIC_CLIENT_USE_PKCE=true, preserving existing in-memory behaviour
* fix: address second round of greptile review feedback
- Fix PKCE error hint: check env var directly (not code_verifier presence) to
distinguish 'PKCE not configured' from 'PKCE enabled but cache miss'
- Fix misleading Redis TTL comment in proxy_server.py
* fix: address third round of greptile review feedback
- Fix CRITICAL log firing on every non-PKCE callback: only log when PKCE is enabled
- Remove unused pkce_env_value intermediate variable
- Prefer reusing redis_usage_cache over creating separate RedisCache instance
(avoids losing advanced connection options like SSL, timeouts, db)
* fix: address fourth round of greptile review feedback
- Strip OAuth token credentials from response_convertor input to prevent
access_token/id_token appearing in restricted-group error messages
- Reuse single httpx.AsyncClient for both token exchange and userinfo requests
to avoid a second TCP/TLS handshake per SSO callback
- Revert Redis wiring to user_api_key_cache: PKCE code already uses
redis_usage_cache directly; wiring would route all API-key lookups through
Redis unnecessarily. Add startup warning instead when PKCE+Redis mismatch.
- Move _OAUTH_TOKEN_FIELDS to module level
* fix remaining PKCE test assertion for dict-format verifier storage
* sanitize PKCE cache log to not expose verifier content
* address greptile review feedback (greploop iteration 3)
* address greptile review feedback (greploop iteration 4)
* address greptile review feedback (greploop iteration 5)
* simplify _get_pkce_userinfo: remove shared-client complexity, use async with directly
* address greptile review feedback (greploop iteration 6)
* address greptile review feedback (greploop iteration 7)
* address greptile review feedback (greploop iteration 8)
* address greptile review feedback (greploop iteration 9)
* address greptile review feedback (greploop iteration 10)
* address greptile review feedback (greploop iteration 11)
* address greptile review feedback (greploop iteration 12)
* fix misleading comment on user_api_key_cache TTL line
* address greptile review feedback (greploop iteration 13)
* address greptile review feedback (greploop iteration 14)
* address greptile review feedback (greploop iteration 15)
* address greptile review feedback (greploop iteration 16)
* address greptile review feedback (greploop iteration 17)
* address greptile review feedback (greploop iteration 18)
* address greptile review feedback (greploop iteration 19)
* address greptile review feedback (greploop iteration 20)
* address greptile review feedback (greploop iteration 21)
* address greptile review feedback (greploop iteration 22)
* address greptile review feedback (greploop iteration 23)
* address greptile review feedback (greploop iteration 24)
* address greptile review feedback (greploop iteration 25)
* address greptile review feedback (greploop iteration 26)
* address greptile review feedback (greploop iteration 27)
* address greptile review feedback (greploop iteration 28)
* address greptile review feedback (greploop iteration 29)
* address greptile review feedback (greploop iteration 30)
* address greptile review feedback (greploop iteration 31)
* address greptile review feedback (greploop iteration 32)
* address greptile review feedback (greploop iteration 33)
* address greptile review feedback (greploop iteration 34)
* address greptile review feedback (greploop iteration 35)
* address greptile review feedback (greploop iteration 37)
- read GENERIC_CLIENT_USE_PKCE env var once in prepare_token_exchange_parameters
- include actual decode error in jwt.decode failure exception message
- add GENERIC_CLIENT_USE_PKCE=true to no-state regression test
* defer PKCE verifier deletion until after all downstream processing
Move _delete_pkce_verifier to after response_convertor and
process_sso_jwt_access_token complete. If JWT processing raises,
the verifier stays in cache so the user can retry without restarting
the full OAuth flow.
* address greptile review feedback (greploop iteration 38)
- fix strict-mode cache miss error message to differentiate
cross-instance routing failures (Redis configured) from single-instance
issues (TTL expiry, pod restart) when only in-memory cache is available
- add comment above _get_pkce_userinfo call explaining that bearer
credentials are always sourced from token_response in the merge step
* fix null JSON response body in _pkce_token_exchange
- Guard against HTTP 200 with body null: response.json() returns None
for JSON null, and calling .get() on None raises AttributeError.
Now raises a clean ProxyException with a clear error message.
- Fix misleading userinfo warning: was always saying "empty dict" but
also fires for JSON null responses; updated to say "empty or null".
- Add HTTP status code assertion to cache miss test.
* address greptile review feedback (greploop iteration 39)
- fix credential leakage: directly assign received_response from
combined_response instead of relying on nonlocal mutation; Pyright
was flagging the old guard as unreachable, meaning credential stripping
might not execute — now it always runs unconditionally
- add test for legacy plain-string cache format backward compat branch
- add test for HTTP 200 with no error field and no access_token (else branch)
- add test for HTTP 200 with JSON null body (new AttributeError guard)
* fix _OAUTH_TOKEN_FIELDS merge loop to preserve userinfo values on absent fields
When the token endpoint omits a bearer-credential field entirely (field
absent from token_response), the previous code deleted it from merged even
if userinfo provided a valid value. Now:
- non-null in token_response → restore authoritative token endpoint value
- explicit null in token_response → remove key from merged (clean absence)
- field absent from token_response → leave userinfo value unchanged
* use HTTP 401 for PKCE missing config errors
GENERIC_CLIENT_ID and GENERIC_TOKEN_ENDPOINT missing when PKCE is
enabled are auth-flow failures, not server errors. Use 401 instead
of 500 to avoid triggering false-positive server error alerts in
monitoring systems.
* address greptile review feedback (greploop iteration 40)
- fix duplicate error logging: demote first format-error log to DEBUG
so the detailed ERROR in strict-mode branch is not duplicated
- add HTTP status code assertions to all PKCE ProxyException tests
for better regression protection against accidental code changes
* add credential absence assertions to test_pkce_token_exchange_basic_auth
Verify that client_id and client_secret are NOT double-sent in the POST
body when Basic Auth is used (include_client_id=False with client_secret).
Catches regressions where credentials leak into both Auth header and body.
* address greptile review feedback (greploop iteration 41)
- add Bearer token header assertion to test_pkce_token_exchange_credentials_in_body
- add cache query assertions to both non-strict mode tests to confirm
the cache was accessed before the warning path triggers
* address greptile review feedback (greploop iteration 42)
- assert null id_token is absent from merged result in basic auth test
- add test for HTTP 200 empty/null userinfo body with no id_token fallback
* use caplog to verify warning logs in non-strict cache miss tests
The two non-strict mode tests now use pytest's caplog fixture to assert
that a warning is actually emitted, not just that the code continues
without raising. This catches regressions where the warning silently
disappears.
* remove dead-code response=None guard in _pkce_token_exchange
* clean up stale pkce verifier cache entries in non-strict mode
* fix test: configure async_delete_cache as AsyncMock and assert cleanup called
* add sentinel guard so pkce-no-redis warning only fires once across hot-reloads
* fix misleading comments: code_verifier init and bearer-credential merge docs
* add best-effort cleanup in strict-mode for corrupt/empty cache entries
* add redirect_uri assertion, userinfo body in non-200 log, sentinel comment
* fix(test): null AWS SigV4 fields on MagicMock in test_inherit_credentials_from_existing_server
* fix(test): null AWS SigV4 fields on MagicMock in test_add_session_mcp_server_caches_and_redacts_credentials
* fixed mcp api
* added non-admin test
* resolved greptile comemnt
* fix: add IP filtering to get_mcp_server_by_id path in fetch_mcp_server
Apply _is_server_accessible_from_ip check after get_mcp_server_by_id lookup
to prevent external callers from accessing MCP servers configured with
available_on_public_internet=False when they know the server_id.
Made-with: Cursor
* fix(mcp): fix OpenAPI OAuth flow — transport mapping, error messages, and discovery bypass
Three bugs fixed to make the end-to-end OAuth flow work for OpenAPI MCP servers:
1. **Transport mapping in getTemporaryPayload**: `TRANSPORT.OPENAPI` is a UI-only concept;
the backend only accepts `"http"`, `"sse"`, or `"stdio"`. The pre-OAuth temp-session
call was sending `transport: "openapi"` and getting a 422. Fixed by mapping to `"http"`.
2. **deriveErrorMessage handles FastAPI 422 arrays**: FastAPI validation errors return
`detail` as an array of `{loc, msg, type}` objects. The shared error extractor was
returning the array directly, causing `Error: [object Object]`. Fixed to map each
item to its `.msg` field.
3. **Skip OAuth discovery when authorization_url already provided**: `build_mcp_server_from_table`
was unconditionally calling `_descovery_metadata(server_url)` for OAuth servers. For
OpenAPI servers the url is the spec JSON file, not the API base — this caused a timeout
fetching e.g. the GitHub spec (2 MB). Fixed by skipping discovery when `authorization_url`
is already set.
Also: collapsible auth section in MCP server form, "Create OAuth App →" link next to
Client ID when a docs URL is available (e.g. GitHub OAuth App creation page), and
`extractErrorMessage` helper in `useMcpOAuthFlow` for cleaner error display.
* refactor(mcp): extract needs_discovery flag and reduceStaticHeaders helper
* feat(mcp): user OAuth connect flow — OAuthConnectModal, MCPCredentialsTab, useUserMcpOAuthFlow
Adds the user-facing MCP OAuth2 PKCE connect flow:
- OAuthConnectModal: modal that launches the PKCE flow for a user to connect to an MCP server
- MCPCredentialsTab: credentials management tab in the MCP apps panel
- useUserMcpOAuthFlow: hook that handles the full PKCE auth code exchange for user-level connections
- MCPAppsPanel: wires up the new credentials tab and connect modal
- ChatPage: further cleanup after responses-API revert
- db.py / mcp_management_endpoints.py / _types.py: backend support for storing user MCP credentials
* fix(mcp): make client_id optional in /authorize — use server's stored client_id when not provided
* address greptile review feedback
* fix(mcp): narrow bare except to RecordNotFoundError in BYOK credential delete
* refactor(mcp): move inline imports to module level in db.py
* docs(claude): add MCP OAuth, transport mapping, and browser storage patterns
* fix(security): remove accessToken from sessionStorage in OAuth flow state
The LiteLLM API key was being serialised into sessionStorage as part of
StoredFlowState. After the OAuth redirect the component re-mounts with the
same accessToken prop, so it never needed to be stored. Read it from props
in resumeOAuthFlow instead.
* fix(ui): remove duplicate extractErrorMessage, sessionStorage-only in admin OAuth hook, call delete API on disconnect
* fix(ui): guard resumeOAuthFlow against wrong hook instance consuming OAuth result
* fix(ui): separate OAuth result keys per flow, sessionStorage-only, surface revoke errors
* fix(ui): remove dead OAuthConnectModal, revert tsconfig jsx mode to preserve
* fix(mcp): guard BYOK overwrite in oauth credential store, raise clear error when client_id absent
* fix: forward OAuth error params in callback, fix BYOK guard exception handling in db.py
* fix: use group_by instead of find_many(distinct) in /tag/list to avoid full table scan
The list_tags endpoint used Prisma's find_many(distinct=["tag"]) to discover
dynamic tags from LiteLLM_DailyTagSpend. Prisma's distinct is a client-side
post-processing filter that fetches all rows with all columns before
deduplicating, causing a full table scan of 3M+ rows on every usage page load.
Replace with Prisma's group_by() which generates proper GROUP BY SQL, letting
the database handle deduplication efficiently. NULL tags are filtered out via
where={"tag": {"not": None}}, and created_at/updated_at are preserved via
_min/_max aggregates.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: address PR review feedback
- Remove unused LiteLLM_DailyTagSpendTable class and datetime import
- Simplify created_at/updated_at access (values guaranteed by spend system)
- Remove DB call assertions from tests, keep response-only assertions
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: RheagalFire <arishalam121@gmail.com>
* fix: batch key queries in list_team to eliminate N+1
The list_team endpoint issued one DB query per team to fetch keys,
causing N+1 query overhead. Replace with a single batched IN query.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: address PR review feedback on team list test
- Rename test to include "v1" for consistency with other test names
- Remove unnecessary DB call assertion, keep behavioral assertions
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: RheagalFire <arishalam121@gmail.com>
* fix(anthropic): enforce type:'object' on tool input schemas
Anthropic's API requires all tool input_schema to have type:'object'
at the root level. When OpenAI-format tools have parameters with a
missing or non-'object' type field (common with MCP tool servers),
the schema was passed through unchanged, causing Anthropic to reject
with: 'tools.N.custom.input_schema.type: Input should be object'.
The existing default handles the case where parameters is entirely
missing, but does not normalize schemas that ARE provided with a
wrong or absent type field.
Fix: After extracting _input_schema in _map_tool_helper(), ensure
type is set to 'object' and properties exists. This matches the
normalization already done implicitly by the Bedrock handler.
Added 4 unit tests covering: missing type, wrong type, valid schema
(no-op), and entirely missing parameters.
Related issues: #12020, #64, #1671
* fix(anthropic): deduplicate tool_result messages by tool_call_id
Anthropic requires exactly one tool_result per tool_use. When
conversation history (e.g. from session resume/checkpoint restore)
contains duplicate tool result messages with the same tool_call_id,
the API rejects with: 'each tool_use must have a single result.
Found multiple tool_result blocks with id: <id>'.
This is already handled for Bedrock via _deduplicate_bedrock_tool_content()
but was missing from the Anthropic direct and Vertex AI partner paths,
which share sanitize_messages_for_tool_calling().
Fix: Add Case D to sanitize_messages_for_tool_calling() — after the
existing orphan detection passes, scan for duplicate tool_call_ids
and keep only the last occurrence (most complete result).
Added 3 unit tests: dedup with duplicates, no-op with unique IDs,
and behavior when modify_params=False.
Related issues: #11804, #11029, #6836, #1782, #151
* fix: shallow copy input_schema to avoid caller mutation + add mutation guard test
Addresses Greptile review:
- dict(_input_schema) before mutation prevents cross-provider state leakage
- Test asserts original tool parameters dict is unchanged after call
* feat: add qwen3.5 series for openrouter
* fix: typo on max_output_tokens and max_tokens from qwen3.5 series
* chore: fix
* chore: fix
* [Test] UI - Logs: Add unit tests for 5 untested view_logs components
Add vitest tests for TypeBadges, ErrorViewer, ConfigInfoMessage, TimeCell, and TruncatedValue covering rendering, user interactions, and edge cases.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Rename 'Team-Based Guardrails' to 'Team Bring-Your-Own Guardrails' (#23307)
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
* feat(chat-ui): responses API + MCP tool execution in /chat (#23297)
* feat(ui): add Chat UI v0 — standalone LiteLLM-branded chat window
Adds a full chat UI accessible from the sidebar Chat link (opens in new tab).
- Standalone route at /chat (outside dashboard layout — no Navbar/Sidebar chrome)
- Claude.ai-style layout: model selector top-left, LiteLLM logo center, settings top-right
- Greeting with time-of-day, centered input card, suggestion chips (Write/Learn/Code/Brainstorm)
- Sliding conversation history sidebar with Cmd+K search, rename, delete, date grouping
- localStorage-backed conversation persistence (litellm_chat_history_v1)
- Streaming completions via makeOpenAIChatCompletionRequest with AbortController stop support
- MCP server picker (toggle servers on/off per conversation)
- LiteLLM aesthetic: white/light-gray background, Ant Design blue (#1677ff) primary, system font
- Sidebar2: Chat menu item opens in new tab via window.open
* feat(chat-ui): responses API + MCP tool execution display
- Switch /chat from chat completions to responses API (previous_response_id session chaining)
- Add MCP server picker with search filter in chat input bar
- Show MCP tool call events (list_tools + call_tool) inline in chat via MCPEventsDisplay
- Add tool chip strip showing available tools when MCP servers are selected
- Non-blocking MCP toggle: server added immediately, verification in background (works for no-auth MCPs like deepwiki)
- Add truncateAfterMessage to useChatHistory for edit/retry
- Sync activeConversationId on URL change (fixes stale conversation on new chat)
- Add "Open Chat" shortcut button to sidebar
* fix(chat-ui): switch to responses API, remove dead code, add tests
- Switch handleSend from makeOpenAIChatCompletionRequest to makeOpenAIResponsesRequest with previous_response_id session chaining
- Add responsesSessionId state; reset to null when starting a new conversation
- Remove unused ChatInputBar.tsx and ModelSelector.tsx (dead code)
- Add tests/test_litellm/test_chat_ui_responses_session.py covering previous_response_id forwarding and signature validation
* fix(chat-ui): address greptile review issues
- Reset responsesSessionId when activeConversationId changes (not just on new conversation)
- Wire onMCPEvent callback into makeOpenAIResponsesRequest; render MCPEventsDisplay below messages
- Clear mcpEvents on each new send
- Explicitly filter history to user/assistant roles only (no tool-role casting)
- Remove duplicate "Chat" menu item from sidebar (pinned button serves same purpose)
- Make Sider a flex column so "Open Chat" button actually pins to bottom
- Fix tests to intercept real HTTP requests and assert previous_response_id in body
* fix(chat-ui): address greptile review feedback (greploop iteration 1)
- Fix duplicate context: when responsesSessionId is set, only send the
new user message as input (prior context is already server-side via
session chaining). Full history is still sent on the first turn.
- Fix ephemeral MCP events: store events per-message in ChatMessage.mcpEvents
instead of ephemeral component state. Events now survive across turns
and render inline below each assistant response via MCPEventsDisplay.
- Remove stale mcpEvents useState and ephemeral panel at bottom of chat.
* fix(chat-ui): address greptile review feedback (greploop iteration 2)
- Fix stale session on edit/retry: derive previousResponseId as null when
historyOverride is set so edit/retry always starts a fresh Responses API
session rather than chaining off a now-invalid prior session
- Fix unsafe MCPEvent cast: import MCPEvent directly from MCPEventsDisplay
into types.ts and type ChatMessage.mcpEvents as MCPEvent[], eliminating
the bare 'as MCPEvent[]' cast in ChatMessages.tsx
* fix(chat-ui): fix MCPEvent layering, batch localStorage writes, module-level test imports
- Move MCPEvent interface definition into chat/types.ts (single source of truth)
- MCPEventsDisplay.tsx now imports MCPEvent from types.ts instead of defining it locally
- Batch MCP event localStorage writes: accumulate during stream, persist once in finally
- Move test imports to module level per PEP 8 convention
* fix(chat-ui): fix MCPEvent import path and rename truncateFromMessage
- responses_api.tsx now imports MCPEvent directly from chat/types (not via MCPEventsDisplay re-export)
- Remove the now-unnecessary MCPEvent re-export from MCPEventsDisplay.tsx
- Rename truncateAfterMessage → truncateFromMessage: the function removes the target message and all subsequent ones (not just what comes after), so the new name accurately describes the behavior
* fix(responses-api): fix whitespace token filter and MCP server URL construction
- Drop the delta.trim() whitespace filter that was silently swallowing spaces
and newlines during streaming, causing words to concatenate and paragraphs
to collapse. Only skip truly empty strings (delta.length > 0).
- Use proxyBaseUrl for MCP server_url construction instead of the hardcoded
relative path "litellm_proxy/mcp", so non-root deployments route correctly.
* fix(responses-api): use unique server_label per MCP server to prevent tool routing collisions
* fix(chat-ui): move MCPEvent to shared mcp_tools/types, skip partial events on abort
- Move MCPEvent interface to mcp_tools/types.tsx (shared with MCPServer/MCPTool),
eliminating the playground→chat cross-module dependency. chat/types.ts and
both playground components now import from mcp_tools/types.
- Only persist accumulated MCP events when the stream completes cleanly; aborted
or errored turns drop partial events to avoid showing incomplete tool calls.
* fix(responses-api): use server_name for MCP URL routing, fix test path
- Use server_name (not alias) as the URL path segment for MCP server_url;
alias is a display name that may differ from the registered proxy route.
URL-encode the path to handle names with spaces/special characters.
- Fix sys.path.insert in tests to use __file__-relative path so tests pass
regardless of which directory pytest is invoked from.
* fix(chat-ui): fix stale session after failed edit, clean MCP event persistence, unique server_label
- Eagerly call setResponsesSessionId(null) when historyOverride is set so a
failed/aborted edit does not leave a stale session contaminating the next turn
- Replace abort-signal check with streamCompletedCleanly flag to correctly skip
MCP event persistence on both abort and non-abort errors (network/API failures)
- Use server_name (unique) as server_label instead of alias to prevent silent
tool-routing failures when two MCP servers share the same display name
* [Feat] UI - Show logos on MCP Apps page (#23320)
* feat(ui): add MCP server logo support across admin and chat UIs
- New MCPLogoSelector component with grid of well-known logos (GitHub,
Slack, Notion, Linear, Jira, etc.) and custom URL input
- Create MCP Server form: logo picker with preview, OpenAPI presets
auto-fill logo from registry icon_url
- Edit MCP Server form: logo picker pre-populated from mcp_info.logo_url
- Admin table: logos rendered next to server name in Name column
- Chat MCPAppsPanel: logos on server cards (list + detail view) with
graceful fallback to letter avatars
- Chat MCPConnectPicker: logos next to server names in toggle list
- Fix pre-existing bug: setTools -> clearTools in create form cancel
- All 321 vitest files / 3211 tests pass
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* feat(ui): use local SVG logos for MCP services, fix Chat UI rendering
- Add 15 new MCP service logo SVGs (Slack, Notion, Linear, Jira, Figma,
Gmail, Stripe, Salesforce, Shopify, HubSpot, Twilio, Sentry, Zapier,
GitLab, Google Drive) to both source and pre-built directories
- Switch MCPLogoSelector from CDN URLs (cdn.simpleicons.org) to local
asset paths (/ui/assets/logos/) for reliable rendering
- Logos now served by the proxy itself, working from any page path
including /ui/chat/ (absolute paths resolve correctly everywhere)
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* fix(codeql): remove ruby from language matrix (#23227)
* Add team-scoped MCP server filtering for key creation and fix UnboundLocalError
When creating a key, the MCP server list now filters by the selected team's
allowed servers. Also fixes UnboundLocalError on `is_restricted_virtual_key`
when `team_id` query param was provided to GET /v1/mcp/server.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Fix cross-team MCP server info disclosure and restricted key bypass
The GET /v1/mcp/server endpoint allowed any authenticated user to pass
an arbitrary team_id and enumerate another team's MCP server config.
Restricted virtual keys could also use the team_id param to bypass
their access limitations. Add team membership check for non-admins
and block restricted keys from using the team_id filter.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Fix mcp_tool_permissions JSON string deserialization in _resolve_team_allowed_mcp_servers
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* [Feature] UI - MCP Servers: Add per-server health recheck
Allow users to recheck health for individual MCP servers by clicking
the health status badge. On hover the badge text changes to "Recheck"
with a refresh icon, and the check runs only for that server.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Fix Anthropic docs link for beta endpoint
Update the Anthropic /v1/messages beta endpoint docstring to point to
its current pass-through documentation.
This keeps the change scoped to the incorrect URL and avoids changing
unverified wording in the surrounding comment.
---------
Co-authored-by: netbrah <162479981+netbrah@users.noreply.github.com>
Co-authored-by: Yong woo Song <ywsong.dev@kakao.com>
Co-authored-by: yuneng-jiang <yuneng.jiang@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: Sameer Kankute <sameer@berri.ai>
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
Co-authored-by: Joe Reyna <joseph.reyna@gmail.com>
The GET /v1/mcp/server endpoint allowed any authenticated user to pass
an arbitrary team_id and enumerate another team's MCP server config.
Restricted virtual keys could also use the team_id param to bypass
their access limitations. Add team membership check for non-admins
and block restricted keys from using the team_id filter.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When creating a key, the MCP server list now filters by the selected team's
allowed servers. Also fixes UnboundLocalError on `is_restricted_virtual_key`
when `team_id` query param was provided to GET /v1/mcp/server.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat(mcp): add BYOM (Bring Your Own MCPs) submission + admin review workflow
Non-admins can now submit MCP servers for review via POST /v1/mcp/server/register.
Admins get a Submissions tab in the UI to approve or reject pending servers.
Approved servers enter the active runtime; rejected ones stay out with notes.
- DB: add approval_status, submitted_by, submitted_at, reviewed_at, review_notes
to LiteLLM_MCPServerTable with migration
- Backend: new endpoints register, submissions, approve, reject
- reload_servers_from_database now only loads approval_status=active servers
- UI: Submissions tab with stat cards, card list, confirm dialogs; non-admin
"Submit MCP Server" button wired to /register endpoint
- Fix get_mcp_submissions to filter by submitted_at IS NOT NULL (not submitted_by,
which can be null for team-scoped keys without an associated user)
* feat(mcp): rename nav item to Team MCPs + add New badge
* fix(mcp): revert nav label, rename Submissions tab to Team MCPs + New badge
* feat(mcp): add MCP Standards — required fields config + CI-style checks on submissions
Adds a "Standards" tab (admin-only) to MCP Servers where admins define which
server fields are required for a submission to pass. Each submission card in
Team MCPs then shows a green ✓ or red ✗ for each required field, with a
summary "N/M checks" badge in the header — like GitHub CI status rows.
Also adds a `source_url` field (GitHub / Source URL) to the MCP server schema
so non-admins can link to the source repo when submitting a server.
- schema.prisma: add `source_url String?` to LiteLLM_MCPServerTable
- migration: 20260309000001_add_mcp_source_url
- _types.py: source_url on NewMCPServerRequest, UpdateMCPServerRequest, LiteLLM_MCPServerTable
- types.tsx: source_url on MCPServer interface
- create_mcp_server.tsx: GitHub/Source URL form field
- MCPStandardsSettings.tsx: new — toggle which fields are required (stored in general settings as mcp_required_fields)
- mcp_servers.tsx: Standards tab (admin-only)
- MCPSubmissionsTab.tsx: load required fields + CI-style check pills on each card
* refactor(mcp): move submission rules into Team MCPs tab, grouped free-form UI
Folds the Standards tab into Team MCPs. Submission Rules panel now lives at the
top of the Team MCPs tab — collapsible, shows active rules as chips when closed,
expands to a grouped checkbox editor (Documentation / Source / Connection /
Security). Removes the separate Standards tab from the nav.
MCPStandardsSettings.tsx is now constants-only (FIELD_GROUPS, MCP_REQUIRED_FIELD_DEFS,
SETTINGS_KEY) — the UI lives in MCPSubmissionsTab.
* feat(mcp): add mcp_required_fields to ConfigGeneralSettings + config/list endpoint
Registers mcp_required_fields as a proper general_settings field so the UI
can read/write it via /config/list and /config/field/update without the
"Invalid field" error. Also fixes a pre-existing pyright None-check issue
in _sync_ui_settings_to_general_settings.
* ui(mcp): GitHub-style PR checks panel on submission cards
* ui: rename Team MCPs -> Submitted Tools, Team Guardrails -> Submitted Guardrails
* address greptile review feedback (greploop iteration 1)
* fix: inline import, add approval workflow tests, rename Submitted MCPs
* fix(mcp): allow re-approval of rejected MCP server submissions
* fix(mcp): evict rejected servers from runtime; enforce mcp_required_fields on /register
* fix(mcp): sort submissions newest-first; force active status on admin-created servers
* fix(mcp): add missing mock in test, show Approve for rejected, clear submission metadata, drop spurious Content-Type
* fix(mcp/ui): show Reject for active servers; show submit form to non-admins with team-key note
* fix(mcp): conditional reload on reject; view-only admin for submissions; block admin from /register
* fix(mcp): match auth_type required-field validation to UI compliance check (reject 'none')
* fix(mcp): block view-only admin from /register; log settings failure; warn on active server reject
* fix(mcp): allow view-only admin to use /register; add _validate_mcp_required_fields tests
* fix(mcp): validate field names in mcp_required_fields; surface backend error in submit UI
* fix(mcp): fix falsy field check; add field-name validation; add take limit; document server-managed fields; close dialog on error
