Commit graph

40341 commits

Author SHA1 Message Date
tin-berri
f19bf2c984
feat(mcp): migrate the token_exchange (OBO) arm to the v2 resolver (#31526)
* feat(mcp): v2-native RFC 8693 token exchanger for the token_exchange mode

Adds the pure Rfc8693TokenExchanger plus its composition root: the OBO exchange POSTs the
RFC 8693 grant through an injected HTTP edge and returns the upstream-bound token as a typed
Result, caching and single-flighting per (subject_token, server) so a repeated caller token
skips the IdP round-trip. The audience is carried on TokenExchangeConfig and sent only when the
operator set one, matching the spec default behavior. Errors are values: a missing endpoint or
client credential is misconfigured, an IdP that returns no usable token is upstream_unavailable.

* feat(mcp): migrate the token_exchange arm to the v2 resolve_credentials

Routes RFC 8693 OBO servers through the v2 resolver: the resolver arm reads the caller's
inbound token and swaps it via the injected TokenExchanger, to_server_spec maps a complete
oauth2_token_exchange server (endpoint plus client credentials) to TokenExchangeConfig, and the
egress wires the LazyTokenExchanger in. A token_exchange server with no caller token fails closed
with a plain 401 rather than v1's fall-through to client_credentials, so the call site now scopes
the per-server browser-OAuth challenge to authorization_code and lets other modes raise their own.

* fix(mcp): bind the token-exchange cache key to the exchange config

The exchanged-token cache was keyed only by (subject_token, server_id), so rotating a server's
audience, scope, endpoint, client_id, or secret kept serving a token minted for the old config
until TTL. The key now hashes the caller token together with the config that minted it, so a config
change forces a fresh exchange. Everything is hashed, so no secret is held in the key.

* refactor(mcp): build the token exchanger eagerly, dropping the lazy wrapper

The token exchanger reads no runtime global at build time (its httpx client is acquired per call),
unlike the per-user store, so it does not need lazy first-use construction. Building it once at
egress construction removes the first-use init path entirely and keeps the process-lifetime cache.

* fix(mcp): map non-object token-exchange JSON to a miss instead of a 500

The post adapter annotated the parsed body as a dict without checking it, so a valid-but-non-object
JSON response (list/string/number) was returned as-is and crashed the field parsing with an
AttributeError. It now validates the shape at the boundary and returns None for a non-object body,
so a malformed IdP response surfaces as a typed upstream_unavailable rather than a server error.

* fix(mcp): fail closed on a non-Bearer token_type in the OBO exchange

* feat(mcp): honor token_endpoint_auth_method (client_secret_basic) in the v2 OBO exchange

* feat(mcp): reject a non-access issued_token_type in the OBO exchange

* fix: include token endpoint auth method in exchange cache key

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2026-07-03 16:04:56 -07:00
Krrish Dholakia
7109b2f61c fix(ui): view-switcher navigation from chat route, add beta banner
"AI Gateway" in the topnav view switcher only called setMode(), which
is meaningful inside the dashboard SPA shell but a no-op on /chat,
which lives outside it (only "Chat" had a real navigation). Now
switching modes from the chat route does a real navigation back to
the dashboard root.

Also adds a persistent banner across all chat routes flagging it as a
pre-v0 feature not for production use, with a feedback link.
2026-07-03 15:54:44 -07:00
Krrish Dholakia
a58930e94e Merge remote-tracking branch 'origin/litellm_chat-keys-usage' into litellm_chat-keys-usage 2026-07-03 15:36:49 -07:00
Krrish Dholakia
afa739b423 fix(ui): design polish and per-tab routing for chat UI
Moves Chats/Integrations/Credentials/API Keys/Usage from client-side
tab state to real nested routes (/chat, /chat/integrations,
/chat/credentials, /chat/api-keys, /chat/usage) so each is bookmarkable
and survives a hard reload. Extracts the chat sidebar into ChatShell
and shared state (MCP server selection, conversation history) into
ChatShellContext, both consumed via the new app/chat/layout.tsx.

Along the way: fixes conversation URLs pointing at the wrong path
(/ui/chat instead of /chat in dev, which 404'd after sending the first
message) by reusing the existing migratedHref helper instead of a
one-off uiConfig-based path; fixes the topnav view-switcher always
showing "AI Gateway" as selected even while on the chat route; and
cleans up several shadcn/tailwind styling bugs introduced by the antd
migration (boxed tab outline instead of underline, model-selector
dropdown overflowing its popover, sidebar nav labels centered instead
of left-aligned, duplicate logo, dead non-interactive controls).
2026-07-03 15:35:52 -07:00
yuneng-jiang
c58e2266a2
Merge pull request #32072 from BerriAI/litellm_budget_fallbacks_ui
feat(ui): add budget fallbacks configuration to key create/edit forms
2026-07-03 15:32:10 -07:00
yuneng-jiang
55e3296e0c
Merge pull request #32026 from BerriAI/litellm_headroom_unreachable_fallback
feat(guardrails): add unreachable_fallback fail-open option to headroom guardrail
2026-07-03 15:24:34 -07:00
Krrish Dholakia
15d6a29c61 merge: resolve eslint-metrics.json conflict with litellm_internal_staging
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-07-03 22:08:32 +00:00
yuneng-jiang
36cbc3a24c
Merge pull request #32082 from BerriAI/litellm_/eloquent-swanson-d634ca
chore(ui): add no-console lint ratchet and strip console from prod builds
2026-07-03 15:07:48 -07:00
yucheng-berri
099bc97320
fix: prevent duplicate budget alert emails on concurrent threshold crossings (#32011)
* fix: prevent duplicate budget alert emails on concurrent threshold crossings

Budget alert emails were sent more than once for a single threshold crossing. The email dedup guard read the "already sent" marker, awaited the send, then wrote the marker, so concurrent requests crossing the same threshold within the send window all saw no marker and each sent. This affected the multi-threshold path (default_key_max_budget_alert_emails), the legacy single-threshold path (EMAIL_BUDGET_ALERT_MAX_SPEND_ALERT_PERCENTAGE), and the soft budget path, all in EmailBaseCallback.budget_alerts

All three branches now claim the send slot atomically before sending via async_increment_cache, which is atomic per event loop for the in-memory cache and across workers via Redis INCR; only the caller that observes a count of 1 sends. On send failure the marker is released with async_delete_cache so a transient failure does not suppress the alert for the full 24h TTL

* fix: harden budget alert claim release and skip-path event allocation

Addresses review feedback on the claim-before-send change. The claim release in each send-failure handler now logs the send error first and releases the claim best-effort through a shared helper, so a transient cache error during async_delete_cache cannot propagate out of the fire-and-forget budget_alerts task, drop the send-failure log, and leave the claim stuck for the full 24h TTL. In the multi-threshold branch the increment claim now runs before the WebhookEvent is built, so skipped concurrent crossings no longer construct and discard the event, matching the single-threshold and soft budget branches
2026-07-03 14:59:34 -07:00
Yuneng Jiang
68d52ac251
chore(ui): preserve console.warn in prod builds to match lint allow-list
The lint rule allows console.warn (allow: [warn, error]) but removeConsole
only excluded error, so approved console.warn calls were silently dropped
from production bundles. Add warn to the exclude list so the prod strip
and the lint allow-list agree; only console.log/debug/info are stripped
now, warn and error both survive (verified: warn 85 to 85, error 906 to
906, log 675 to 14).
2026-07-03 14:51:14 -07:00
Yuneng Jiang
5b7c73a573
chore(ui): sync no-console budget to 484 after staging merge
Merging litellm_internal_staging dropped 2 console.log calls (the
currentUser logs removed in #32079), so the no-console budget max and
metric move from 486 to 484 to match the current count.
2026-07-03 14:37:41 -07:00
Yuneng Jiang
2e44691f56
Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_/eloquent-swanson-d634ca 2026-07-03 14:35:30 -07:00
Yuneng Jiang
76a9f7b5f3
chore(ui): add no-console lint ratchet and strip console from prod builds
Introduce a gradual ratchet to remove raw console.* calls from the
dashboard, mirroring the existing no-explicit-any budget.

The no-console eslint rule is set to warn with allow: [warn, error] so
the 486 console.log/debug/info calls are tracked without force-deleting
the legitimate console.error/warn error reporting in catch blocks. The
count is grandfathered via eslint-budgets.json (max 486, target 0) and
eslint-metrics.json, so any newly added console.log fails the budget
check and follow-up PRs grind the max down toward zero.

Independently, next.config strips console output from production builds
via SWC removeConsole (exclude: [error]), gated on NODE_ENV=production so
dev keeps full console output. This gives an immediate prod-hygiene net
regardless of how long the source cleanup takes. Verified against a real
production build: app-code console.log dropped from 675 to 14 in the
bundle (remainder is node_modules, which the transform leaves alone),
console.warn app calls stripped, console.error preserved 906 to 906.
2026-07-03 14:35:23 -07:00
yuneng-jiang
8e4e37d662
Merge pull request #32079 from BerriAI/litellm_/dreamy-lovelace-4a90c8
test(ui): quiet vitest CI logs by silencing passing-test console output
2026-07-03 14:31:26 -07:00
Shivam Rawat
b723dfb93d fix(realtime): preserve nested transcription model and session-first model priority
_with_resolved_session_model was overwriting the nested
input_audio_transcription.model and audio.input.transcription.model with the
realtime conversation model, silently replacing a caller's transcription model
(e.g. whisper-1) since those are a different model than the realtime deployment.
It now only resolves the top-level session model.

Also restores session.model taking precedence over the top-level model in
acreate_realtime_client_secret, matching the proxy's own
_prepare_client_secret_session ordering and avoiding a backwards-incompatible flip.

Adds routing coverage for arealtime_calls (api_base resolution) and
acreate_realtime_transcription_session (api_key resolution) so all three realtime
HTTP endpoints have router credential-resolution tests, plus regression tests for
the two fixes above.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-03 14:21:22 -07:00
Krrish Dholakia
b42cd37a30 style: format key_edit_view.tsx with prettier
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-07-03 21:21:20 +00:00
Yuneng Jiang
fc17ea3409
Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_/dreamy-lovelace-4a90c8
# Conflicts:
#	ui/litellm-dashboard/vitest.config.ts
2026-07-03 14:19:48 -07:00
Yuneng Jiang
0115bfa523
test(ui): quiet vitest CI logs by silencing passing-test console output
The ui_unit_tests CircleCI job logged ~45k lines for a single run, most of
it React act() warnings, antd deprecation notices and component stack traces
emitted as console output by passing tests, which buried real failures.

Set silent: "passed-only" (Vitest 3.2+) gated on process.env.CI so console
output from passing tests is suppressed while a failing test still prints its
logs and full stack trace. Also drop two stray console.log calls in
UsagePageView that dumped the whole currentUser object on every render in
production, not just tests.

Verified by running the suite the way CI does
(CI=true npm run test -- --run --pool forks --poolOptions.forks.maxForks=6):
45,075 lines before, 981 after, all 4075 tests still passing. A throwaway
failing test confirms its console.log and assertion diff remain visible.
2026-07-03 14:19:30 -07:00
Krrish Dholakia
a55cc3aab3 fix: allow clearing budget_fallbacks in edit view when key had existing fallbacks
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-07-03 21:17:37 +00:00
Mateo Wang
5f4b9ad51c
chore: clarify the linear ticket instruction in pr template (#32076)
* chore: clarify the linear ticket instruction in pr template

* fix: make it more concise

* Update CLAUDE.md

lol

* chore: tell claude not to search for it if it doesn't have it

---------

Co-authored-by: ryan-crabbe-berri <ryan@berri.ai>
2026-07-03 14:14:40 -07:00
yuneng-jiang
23e83f86a6
Merge pull request #32078 from BerriAI/litellm_/distracted-banach-e3481f
fix(ci): stop ui_unit_tests vitest onTaskUpdate RPC timeout flake
2026-07-03 14:13:24 -07:00
Mateo Wang
2e1d8d2928
fix(anthropic): keep context_management working when drop_params is enabled (#32020)
* fix(anthropic): keep context_management working when drop_params is enabled

drop_params (proxy-wide or per-request) silently disabled the in-gateway
context_management polyfill on the /v1/messages -> chat completions adapter
path, even though context_management is a LiteLLM-supported param (native on
Anthropic, polyfilled elsewhere). Gate the polyfill on an explicit
additional_drop_params: ["context_management"] opt-out instead, which also
makes that escape hatch actually work on the adapter path.

* test(anthropic): cover sync adapter polyfill gate for global drop_params and additional_drop_params
2026-07-03 14:12:32 -07:00
Yuneng Jiang
682d55a37d
Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_/distracted-banach-e3481f 2026-07-03 14:03:27 -07:00
Yuneng Jiang
fff6a5396c
fix(ci): stop ui_unit_tests vitest onTaskUpdate RPC timeout flake
The ui_unit_tests job runs vitest with maxForks=8 on an 8-vCPU xlarge
container, leaving no headroom for the main vitest process that services
worker RPCs. Under full CPU saturation the coordinator misses the
onTaskUpdate ack, vitest raises "Timeout calling onTaskUpdate" as an
unhandled error, and the job exits 1 even though every test passes.

Lower maxForks to 6 so the coordinator, jsdom, and OS keep two cores, and
raise teardownTimeout to 60s for extra slack on heavy runs.
2026-07-03 14:00:43 -07:00
Yuneng Jiang
0d31088724
fix(cache): use builtin dict annotation to satisfy UP006 budget gate 2026-07-03 14:00:22 -07:00
ryan-crabbe-berri
57ca48a863
feat(mcp): add all-proxy-mcpservers sentinel to grant teams every MCP server (#32012)
* feat(mcp): add all-proxy-mcpservers sentinel to grant every MCP server

Teams can now be scoped to the all-proxy-mcpservers sentinel so they gain
access to every MCP server on the proxy without listing each id. The
sentinel expands to the live registry at request time, so a server added
later is picked up with no change to the team's stored permission. The team
ceiling that validates a key's MCP scope expands the sentinel too, so a key
can be scoped to any server (including one registered after the team) and
still pass subset validation

Expose the option in the team create and edit forms via a new exclusive
"All Proxy MCP Servers" choice in MCPServerSelector, mirroring the existing
"No MCP Servers" sentinel

* Update litellm/proxy/management_helpers/object_permission_utils.py

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* fix(mcp): honor all-proxy-mcpservers only on the team path, never per-key

The sentinel was expanded inside the shared expand_permission_list, which
also feeds the key, org, end_user and agent resolvers. A key whose stored
object_permission ever held all-proxy-mcpservers (a stale write, a
configured default, or a bug) would silently resolve to every MCP server at
runtime, and a teamless key had nothing to cap it, so all servers got
injected. Only write-time validation stripping the value stood between that
value and a full grant

Move the expansion out of expand_permission_list and into
_get_allowed_mcp_servers_for_team so the sentinel is honored only where it is
settable (a team). Anywhere else it now passes through as an inert literal
that matches no registered server and is denied downstream. Reserved-id
protection already blocks a real server from taking that id

* fix(mcp): require proxy admin to grant a team the all-proxy MCP sentinel

Granting a team every MCP server on the proxy is a proxy-wide authorization
decision, but team create/update let any caller who can manage a team set
object_permission.mcp_servers, with no ceiling check. Org admins reach
/team/update by default (org_admin_allowed_routes) and _verify_team_access
also admits team admins, so a non-proxy-admin could set all-proxy-mcpservers
and self-grant their team access to every MCP server on the proxy, including
servers never assigned to that team

Gate the grant in new_team and update_team: a non-proxy-admin cannot add the
all-proxy-mcpservers sentinel. The check is scoped to newly adding it, so a
team a proxy admin already scoped to all-proxy can still be edited by a team
admin without being forced to strip the sentinel. The UI only offers the
"All Proxy MCP Servers" option to proxy admins in the team create and edit
forms

* fix(ui): render friendly all-proxy MCP label for non-admins editing an all-proxy team

A team scoped to the all-proxy-mcpservers sentinel could be opened in the team
edit form by a team admin or org admin (canEditTeam admits them), but the
"All Proxy MCP Servers" option in MCPServerSelector was rendered only behind the
proxy-admin-gated allowAllProxyMcpServers flag. For a non-proxy-admin the stored
sentinel was hydrated into the selected value with no matching Select.Option, so
antd showed the raw all-proxy-mcpservers literal as a chip, and adding another
server could persist a mixed [all-proxy-mcpservers, <id>] value.

Render the option whenever the sentinel is present in the value, not only when
the caller may grant it, and drive the real-option disabling off presence too so
the selection stays exclusive. A non-proxy-admin now sees the friendly label
read-only and cannot build a mixed state; only a proxy admin can newly add it,
which the backend already enforces.

Adds regression tests: the selector shows the friendly option (not the raw
literal) when the sentinel is stored but the grant flag is off, plus exclusive
emit and disabled-real-options coverage, and MCPServerPermissions renders the
green "All" state instead of the raw sentinel string.

* fix(ui): drop redundant "All servers" hint from the all-proxy MCP chip

antd renders a Select option's children inside the selected tag, so the
all-proxy option showed both "All Proxy MCP Servers" and the green "All servers"
type-hint in the chip, which say the same thing. Collapse the option to a single
green "All Proxy MCP Servers" label so the dropdown row and the chip read cleanly
without the duplication.

* fix(ui): color the all-proxy MCP label blue to match server chips

Use the same blue (#1890ff) as regular MCP server entries for the
"All Proxy MCP Servers" option/chip instead of green.

* fix(ui): make the all-proxy MCP permissions display blue, not green

Match the blue used by the selector chip and regular server entries so the
"All Proxy MCP Servers" badge and row in MCPServerPermissions are consistent
across the team/key/org detail views. The red "Blocked" state for
no-mcp-servers is unchanged.

---------

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-07-03 13:59:28 -07:00
Yuneng Jiang
b86c01c624
fix(cache): mask inline url credentials and drop discrete username under url precedence
Two follow-ups from review of the url/db work.

A Redis/Valkey url can embed a password (redis://:secret@host:6379/1), but
_CACHE_SENSITIVE_FIELDS only masked the discrete password and sentinel_password,
so a stored password-bearing url came back in plaintext from every
GET /cache/settings. Add url to the masked set so it gets the same masked-on-read
treatment as password.

The url-precedence resolver dropped host/port/db/password but not username, even
though a url can encode a username too (redis://user:pass@host). Left in, the
discrete username rode along and could contradict the url. Add username to the
overridden set and update the Redis URL help text to list it among the fields url
takes precedence over.

Tests: GET masks a password-bearing url (secret never returned verbatim) while a
non-credential field is untouched, and the resolver drops a discrete username when
a url is present.
2026-07-03 13:55:45 -07:00
Krrish Dholakia
640ee9384b fix: prevent stale budget fallback entries after form reset and guard empty payload in edit view
Address Greptile P1 (stale state after reset): use key prop to force
BudgetFallbacksEditor remount when parent resets budgetFallbacks to {},
matching the existing routerSettingsKey pattern.

Address Greptile P2 (inconsistent empty payload): guard budget_fallbacks
in edit view to only include when non-empty, matching create form behavior.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-07-03 20:52:25 +00:00
Shivam Rawat
ef758e6a88 fix(proxy): route realtime HTTP endpoints through router for credential resolution
Realtime client_secrets, calls, and transcription_sessions were bypassing
the router and falling back to an empty OPENAI_API_KEY for wildcard, team-scoped,
and credential-name deployments.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-03 13:36:34 -07:00
Krrish Dholakia
0b3e327d02 fix(ui): use project cva config instead of class-variance-authority in badge and tabs
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-07-03 20:23:11 +00:00
Yuneng Jiang
611b8dee18
feat(cache): back the Redis URL and Database Index UI fields end-to-end
The typed cache-settings form already renders a Redis URL and a Database
Index field, but the backend never defined them, so GET /cache/settings
could not round-trip a saved value into the form and the "URL takes
precedence over Host/Port/Password/Database Index" help text the UI shows
was not actually enforced anywhere.

Add the url and db entries to CACHE_SETTINGS_FIELDS so the endpoint knows
about them, and add _resolve_cache_url_precedence: when a non-empty url is
present it wins and the discrete host/port/db/password fields are dropped
before the settings are tested or persisted, matching how litellm._redis
resolves the connection at runtime (redis.Redis.from_url ignores them).
Cluster mode is exempt because it authenticates via the discrete fields
rather than a url. Both test and save paths go through the resolver so the
stored config is unambiguous.

This finishes LIT-3996: operators can now isolate the cache into a logical
database (e.g. redis://host:6379/1) entirely from the Admin UI instead of
hardcoding REDIS_URL in the environment.
2026-07-03 13:21:46 -07:00
Krrish Dholakia
11314f4bae feat(ui): migrate chat UI from antd to shadcn/ui
Replace all Ant Design components (Table, Modal, Popover, Tooltip, Skeleton,
Select, Spin, Popconfirm, Switch) with shadcn/ui primitives and Lucide React
icons across all chat components:

- ChatPage: sidebar, model selector, input bar, comparison mode
- ConversationList: search dialog, delete confirmation, scroll area
- ChatMessages: message bubbles, tool cards, copy button
- MCPAppsPanel: list/detail views, OAuth2 flow, tabs
- MCPConnectPicker: server toggle switches
- MCPCredentialsTab: credentials table with delete
- KeysPanel: API key management with rotation dialog (enterprise)
- UsagePanel: spend/request stats with sparkline charts

Add design.md as the design specification guiding the migration.
Install 15 shadcn/ui components (dialog, popover, tooltip, table, etc.).
All existing functionality preserved; no backend changes.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-07-03 20:15:33 +00:00
yucheng-berriai
bd6ae9effa fix(proxy): stop the decrypt-failure debug log from leaking the raw value
decrypt_value_helper logged `Unable to decrypt value={value}` at DEBUG, which
printed the raw secret whenever decryption failed (for example after a salt or
master key change). This is the same environment_variables config path the
db-config redaction covers, so a DATABASE_URL connection string could still
leak here when the module regex scrubber is bypassed. Drop the value; the key
already identifies the failing pair.

Regression forces a decrypt failure with the redaction filter disabled and
asserts the raw value never reaches a log record while the key stays visible.
2026-07-03 13:03:45 -07:00
yucheng-berriai
89d3f2a7b8 fix: redact db environment variable debug logs 2026-07-03 13:03:45 -07:00
yucheng-berriai
64d6a15182 fix(proxy): redact secrets on the db-config and litellm_settings log paths too
Reuse the existing recursive `_redact_secret_values_in_obj` for the worker
config log instead of a hand-rolled top-level pass, so a credential nested
under general_settings is masked at any depth and depth overrun fails closed.
Route the `_update_config_from_db` param_value log (the store_model_in_db
path) and the litellm_settings apply-loop log through the same redactors, so
master_key, database_url, and secret-named settings such as api_key stop
leaking at DEBUG when the module regex scrubber is bypassed. A plain setting
like num_retries still logs its real value.

Regression tests disable _ENABLE_SECRET_REDACTION and cover the nested worker
config shape, the db-config path, and the litellm_settings loop in both
directions.
2026-07-03 13:03:45 -07:00
yucheng-berriai
680f15f1e3 fix(proxy): stop leaking master_key and database_url in startup DEBUG logs
Three startup log statements in litellm/proxy/proxy_server.py dumped
secret-bearing values in cleartext when the last-line-of-defense regex
scrubber was bypassed (LITELLM_DISABLE_REDACT_SECRETS=true, older versions
that predated the SecretRedactionFilter, or any downstream handler that
snapshots log records before the module filter runs)

ProxyConfig._load_alerting_settings logged the whole general_settings
dict under a label that only referred to the alerting callbacks; a
copy-paste bug that happened to leak master_key, database_url, and every
other secret sitting in general_settings. Now logs only the alerting
callback list

ProxyConfig.load_config logged the resolved DB URL after secret-manager
resolution. The line's stated purpose was to confirm the retrieval ran,
which does not need the value. Now logs a value-less breadcrumb

proxy_startup_event logged the raw WORKER_CONFIG blob, which docker/K8s
deployments hand the proxy as a JSON string containing master_key,
database_url, and provider API keys. Now routes through
_redact_worker_config_for_logging, which combines the segment-matching
SensitiveDataMasker (catches master_key, api_key, *_token) with an
explicit pass over _EXTRA_SECRET_GENERAL_SETTINGS_FIELDS (catches
database_url and other credential-URL fields the segment masker misses)

Regression tests disable the module-level SecretRedactionFilter so
assertions see the raw record; without the fix they would trip on the
secret substring, so a future refactor cannot silently reconstruct the
leaky string
2026-07-03 13:03:45 -07:00
Krrish Dholakia
ed51c96d3f feat(ui): add budget fallbacks configuration to key create/edit forms
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-07-03 19:50:36 +00:00
Krrish Dholakia
87137710c5 fix: restore schema.d.ts from base branch
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-07-03 19:44:41 +00:00
Krrish Dholakia
01dfbf7ebb fix(guardrails): address review comments on headroom fail_open
- Prevent fail-open from registering user-supplied hashes as valid for
  CCR retrieval; _call_compress now returns (messages, compressed_ok)
  so apply_guardrail skips hash extraction and tool injection when
  compression did not succeed
- Remove Optional wrapper from HeadroomGuardrailConfigModel.unreachable_fallback
  to match BaseLitellmParams typing
- Add fail_open tests for non-JSON response, missing messages key, and
  empty message list paths
- Add regression test verifying fail_open does not authorize attacker-planted
  hashes
- Regenerate dashboard API types

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-07-03 19:40:33 +00:00
Krrish Dholakia
e06adb5588
feat(ui): re-add chat UI, allow simple UI for MCP OBO auth (#31893)
Some checks are pending
GitHub Actions Security Analysis / zizmor (push) Waiting to run
2026-07-03 12:36:36 -07:00
Krrish Dholakia
28ddad271e
feat(proxy): add key-level budget_fallbacks to reroute requests when a per-model budget is exceeded (#31783) 2026-07-03 12:20:12 -07:00
mubashir1osmani
70785b9b6a
fix(e2e): define SpendTagsResponse/TagSpend so spend suite collects
fix(e2e): define SpendTagsResponse/TagSpend so spend suite collects
2026-07-03 19:15:47 +00:00
tin-berri
3235f4a499
fix(mcp): persist DCR client_id from on-create MCP OAuth Authorize & Fetch (#31920)
* fix(mcp): persist DCR client_id so interactive OAuth token refresh works

Interactive authorization_code MCP servers register an OAuth client via Dynamic
Client Registration (RFC 7591) during the authorize flow, but the minted
client_id and the discovered token_url were returned to the caller and never
written to the server row. The autonomous refresh_token grant reads client_id,
client_secret and token_url off the server, so an expired access token could not
be refreshed; the user was bounced back to re-authorize and tools/list returned
zero tools

Persist the DCR client_id (plus client_secret and token_endpoint_auth_method when
the registration returns them) and the discovered token_url onto the server row,
reusing the encrypt_credentials write that client_credentials and token exchange
already use, then refresh the in-memory registry so the value is live at refresh
time. Both the v1 refresher and the v2 AuthorizationCodeRefresher read those same
fields, so egress needs no change

* fix: reuse persisted MCP DCR clients

* fix(ui): persist DCR client_id from on-create MCP OAuth "Authorize & Fetch"

The interactive "Authorize & Fetch" flow on the create form registers an OAuth
client (RFC 7591) against a temporary server that has no DB row, then creates the
real server afterward. useMcpOAuthFlow captured the DCR client_id and client_secret
but passed only the token to onTokenReceived, so the create request dropped the
client identity and the created server could not refresh its access token; its row
had credentials={} and the refresh_token grant 401d at the upstream token endpoint

Forward the registered client to onTokenReceived and write client_id (and
client_secret when present) into the create form credentials, so the create request
carries them and the backend persists them through its existing encrypt_credentials
path. token_url is omitted because it is re-discovered on load (RFC 9728 then 8414);
token_endpoint_auth_method is unused because this flow only ever registers as
client_secret_post or none, never client_secret_basic

* fix(ui): prevent stale MCP OAuth credentials

* fix(ui): reset MCP OAuth authorization state

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2026-07-03 12:15:33 -07:00
Cursor Agent
53d2331c70
fix: catch litellm.Timeout in HeadroomGuardrail to support fail_open on timeouts
async_handler.post catches httpx.TimeoutException and re-raises it as
litellm.Timeout (a subclass of openai.APITimeoutError). The except
blocks in _call_compress and _call_retrieve only listed httpx exception
types, so litellm.Timeout propagated uncaught and bypassed the
unreachable_fallback=fail_open path.

Add litellm.Timeout to both except clauses and add regression tests for
the fail_closed and fail_open timeout paths.
2026-07-03 19:13:30 +00:00
yucheng-berri
93cdcca1c5
fix(azure_sentinel): resolve audit stream from AZURE_SENTINEL_AUDIT_STREAM_NAME (#32010)
When AzureSentinelLogger is resolved from the string callback name
"azure_sentinel", it is constructed with no arguments, so audit_stream_name is
always None and resolved_audit_stream_name fell back to the standard
resolved_stream_name. Audit logs then ingested into the access-log DCR stream
whose schema is built from StandardLoggingPayload, so Azure Monitor Logs
Ingestion silently dropped the audit-specific columns and audit rows arrived
effectively empty.

Add an AZURE_SENTINEL_AUDIT_STREAM_NAME env var fallback in __init__, mirroring
the AZURE_SENTINEL_STREAM_NAME idiom already used for the standard stream, so
audit logs can target a separate DCR stream without a custom callbacks file.
2026-07-03 12:09:27 -07:00
tin-berri
15ff389eb4
fix(mcp): persist DCR client_id so interactive OAuth token refresh works (#31912)
* fix(mcp): persist DCR client_id so interactive OAuth token refresh works

Interactive authorization_code MCP servers register an OAuth client via Dynamic
Client Registration (RFC 7591) during the authorize flow, but the minted
client_id and the discovered token_url were returned to the caller and never
written to the server row. The autonomous refresh_token grant reads client_id,
client_secret and token_url off the server, so an expired access token could not
be refreshed; the user was bounced back to re-authorize and tools/list returned
zero tools

Persist the DCR client_id (plus client_secret and token_endpoint_auth_method when
the registration returns them) and the discovered token_url onto the server row,
reusing the encrypt_credentials write that client_credentials and token exchange
already use, then refresh the in-memory registry so the value is live at refresh
time. Both the v1 refresher and the v2 AuthorizationCodeRefresher read those same
fields, so egress needs no change

* fix: reuse persisted MCP DCR clients

* fix: reuse persisted MCP DCR clients

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2026-07-03 10:42:55 -07:00
tin-berri
b59ad212f4
feat(ui): add token endpoint auth method selector to MCP OAuth forms (#31739)
PR #31635 added a per-server token_endpoint_auth_method (client_secret_basic
or client_secret_post) for upstream OAuth token endpoints, but it could only be
set by editing the stored credentials JSON. This surfaces it in the dashboard as
an optional selector directly under the Token URL field, in both the create form
(OAuthFormFields, M2M and interactive flows) and the edit form. The field binds
to credentials.token_endpoint_auth_method, which the backend already reads; the
value is sent only when chosen, so leaving it blank keeps the existing setting
and preserves the client_secret_post default.
2026-07-03 10:25:58 -07:00
tin-berri
d0e785140c
feat(tinyfish): make search provider permissive, attribute errors (#31997)
* feat(tinyfish): make search provider permissive, attribute errors

Reshapes the TinyFish search provider so LiteLLM mirrors the TinyFish
Search API surface instead of maintaining a parallel cherry-pick.

Request side:
- Drop misleading request TypedDict
- Stop sending max_results on wire (TinyFish ignores it); clamp to [1,10]
  client-side via self-threaded state
- Guard non-numeric max_results from bare ValueError
- Auto-JSON-encode dict params; lowercase bool serialization for ux-labs

Response side:
- Drop both Pydantic response models; parse directly into SearchResponse
  so per-result extras flow through via extra="allow"
- Default missing title/url/snippet to "" instead of failing the call
- Read top-level parameter_warnings and re-fire as verbose_logger.warning
  (pre-wired for upcoming TinyFish-side rollout; no-op today)

Error handling:
- Attributed _wrap_error helper at 3 call sites in transform_search_response
  ("TinyFish Search: <msg>. See https://docs.tinyfish.ai/search-api for
  details.")
- Dispatch non-2xx responses through _wrap_error (fixes pre-existing bug
  where 4xx/5xx silently returned empty SearchResponse)
- Wrap json.JSONDecodeError on 200 bodies
- Wrap pydantic.ValidationError for envelope-shape mismatches

Bug fix worth flagging: 4xx/5xx responses now raise an attributed
BaseLLMException instead of silently returning SearchResponse(results=[]).

Follow-up to #30634.

* fix(tinyfish): apply ruff format; guard OverflowError in max_results clamp

- Run ruff format on the touched files (CI lint job rejected the prior
  commit's formatting).
- Add OverflowError to the except clause in the max_results clamp so
  callers passing math.inf (or other non-finite floats) get the same
  warn-and-ignore behavior as other malformed values. Greptile spotted
  this in the first-pass review.
- Add test_max_results_infinity_float_warns_and_skips covering the
  inf case.

* fix(tinyfish): apply --line-length 88 ruff format to match CI

CI uses 'ruff format --check --line-length 88'; my prior format pass
used the default line length, leaving several lines unwrapped. No
behavior change — purely whitespace.

* fix(tinyfish): reduce transform_search_response complexity; sort imports

CI's ruff strict-rule budget rejected the prior commit with:
- C901: transform_search_response complexity 16 > 10 (cap exceeded by 1)
- I001: import sort violation (cap exceeded by 1)

Extract two module-level helpers from transform_search_response to drop
its cyclomatic complexity:
- _default_missing_result_fields: in-place title/url/snippet defaulting
- _emit_parameter_warnings: defensive parameter_warnings reader

Auto-fix the import sort via ruff --fix.

No behavior change; the 59 existing tests still pass.

* test(tinyfish): cover defensive branches in _default_missing_result_fields

Codecov flagged 97.61% patch coverage (2 lines missing). The uncovered
lines were the non-dict raw_json and non-dict per-result item early-exits
in _default_missing_result_fields. Add two unit tests on the helper
directly to bring patch coverage to 100%.

* chore(tinyfish): apply ruff format to fix lint after staging merge

---------

Co-authored-by: Chenlu Ji <jichenlulu@gmail.com>
2026-07-03 10:17:11 -07:00
yuneng-jiang
4e53ae98a7
bump: litellm-proxy-extras 0.4.74 -> 0.4.75, litellm-enterprise 0.1.45 -> 0.1.46 (#32029)
Both directories diverged from main; bump PATCH and relock.
2026-07-02 22:54:59 -07:00
Mateo Wang
ae356cf1fa
fix(azure_ai): preserve content, tables, and keyValuePairs in doc-intelligence /v1/ocr (#32018)
* fix(azure_ai): preserve content, tables, and keyValuePairs in doc-intelligence /v1/ocr

Azure Document Intelligence analyzeResult.content, .tables, and
.keyValuePairs were dropped when normalizing to the Mistral OCR schema.
They are now passed through verbatim as top-level response fields, and
the duplicated sync/async response parsing is consolidated into one
pydantic-validated helper.

Also adds the Azure DI features query param (list[str] or
comma-separated string, e.g. features=keyValuePairs) which Azure
requires for keyValuePairs extraction.

* test(azure_ai): replace fastapi jsonable_encoder with model_dump in ocr unit tests
2026-07-02 22:38:36 -07:00