Pass transcription_cost through additional_costs so cost_breakdown's
input_cost + output_cost + additional_costs sums to total_cost instead
of silently folding it into total_cost only.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(anthropic): require caller api_key and SSRF-validate api_base in advisor tool
The advisor_20260301 interceptor honored a caller-supplied api_base once
allow_client_side_credentials was enabled, even without a caller-supplied
api_key. AnthropicModelInfo.get_auth_header() then fell back to the proxy's
own ANTHROPIC_API_KEY/ANTHROPIC_AUTH_TOKEN, so the server's real credentials
plus the conversation history got sent to a caller-chosen destination
_resolve_advisor_credentials() now only honors api_base alongside a
non-empty caller-supplied api_key, requires the https scheme, and validates
api_base via validate_url() before use, mirroring check_complete_credentials
in auth_utils.py. https is required because validate_url only DNS-pins the
connection for http; for https with TLS verification on it returns the URL
unchanged and relies on certificate validation to block DNS rebinding
* fix(anthropic): also reject advisor api_base when ssl_verify is disabled
validate_url only DNS-pins the connection for http, or for https with
litellm.ssl_verify disabled; the previous https-only check missed the
ssl_verify=False case, where validate_url's rewritten URL was still being
discarded, per Greptile's review of this PR. Reject api_base outright when
ssl_verify is False so the discarded rewrite can no longer matter
* fix(policies): reject non-existent team/key/model scope entries on attachment create
Creating a policy attachment accepted arbitrary team, key, and model values with
no validation, so a typo'd or non-existent team was silently persisted (LIT-4199).
The create endpoint now rejects a concrete (non-wildcard) team, key, or model that
does not resolve to a real entity, wiring the previously-dead PolicyValidator
existence checks and reusing RouteChecks._is_wildcard_pattern so validation agrees
with request-time matching, where only a trailing "*" is a wildcard. Wildcard
patterns are still allowed through since they may match zero entities today and
more later, and tags stay free-form. The Admin UI's Teams field validates the same
rule for immediate feedback when its team list has loaded, deferring to the backend
otherwise.
* style(policies): use builtin list generics and | None in scope validator
Keeps the new find_invalid_scope_entries signature off the UP006/UP045 strict
ruff budgets instead of copying the surrounding legacy typing.List/Optional idiom.
* fix(policies): separate multiple attachment scope errors with ' | '
Addresses Greptile review: joining per-entry validation messages with a bare
space read as one run-on sentence; ' | ' makes the multi-error 400 detail easier
to parse for users and programmatically.
Input didn't wrap its function component in React.forwardRef, so the ref
ConversationList passes for rename auto-focus/select silently never attached
under React 18 (function components need forwardRef to receive a ref; that
requirement is dropped in React 19, but this app is on 18.3.1).
MCPEnhancedStreamingIterator only auto-executed one round of MCP tool calls.
When a model retried a tool (e.g. after an error) in its follow-up turn, that
second tool call was streamed but never executed, and the response ended with
no final text. Route follow-up calls back through the same completion-check
phase as the initial response, so further tool-call rounds are handled the
same way, capped at MAX_MCP_TOOL_CALL_ROUNDS to avoid an unbounded loop.
* feat(mcp): discover the OBO token endpoint via RFC 9728 to RFC 8414 (no IdP guessing)
An oauth2_token_exchange server can now have its token endpoint discovered the
same way the oauth2 (authorization_code) flow already does, instead of always
requiring token_exchange_endpoint/token_url to be configured by hand. The
existing _descovery_metadata chain (RFC 9728 protected-resource metadata ->
RFC 8414 authorization-server metadata -> token_endpoint, SSRF-guarded via
async_safe_get) is reused; both the config-load and DB-build paths gate on a new
_obo_needs_endpoint_discovery so discovery runs only when no endpoint is
configured, and an explicitly configured endpoint still wins and skips the
round-trip. The discovered token endpoint lands on token_url, which
_token_exchange_spec already reads, so no resolver change is needed.
_resolve_oauth2_flow returns None for any non-oauth2 auth_type, so a discovered
token_url on an OBO server is never mis-inferred as the M2M client_credentials
flow.
Discovery for OBO is authoritative only: the resolution order is explicitly
configured endpoint, then RFC 9728 -> RFC 8414 advertisement, then fail closed
(412, on the parent commit). The gateway never guesses the IdP. _descovery_metadata
grows an allow_origin_fallback flag, kept True for the browser oauth2 flow (a
human sees the redirect) but set False for token_exchange so the last-resort
guess that treats the resource server's own origin as its authorization server
is skipped; a subject token is never exchanged against an inferred endpoint.
* fix(mcp): surface a failed OBO exchange at connect instead of an empty tool list
A token_exchange server whose exchange fails with a subject present used to open the MCP
session anyway and mask the failure as an empty tools/list. Single-server routes now run
the exchange preemptively at the transport edge, where a rejected subject raises the RFC
9728 challenge and a gateway fault its public status; the multi-server aggregate keeps
absorbing per-server auth failures. The exchanger caches the preflight result, so the
session's list/call reuses it with no extra IdP round-trip. Discovery now also debug-logs
the authorization server's advertised issuer, grant types, and client auth methods
* fix(mcp): persist the discovered OBO token endpoint to the DB row
A DB-backed oauth2_token_exchange server with no configured endpoint had its token_url
resolved via RFC 9728 -> RFC 8414 only on the in-memory object returned from
build_mcp_server_from_table; the row kept token_url=None, so every rebuild re-ran discovery
and a transient upstream outage during a rebuild left the server with no endpoint until the
next successful discovery. Write the discovered token_url back onto the row so the guard sees
it on the next build. Best-effort and scoped to DB servers: config servers already persist
in-memory, and the write-back never fires from a user connect (only from add/update/reload,
all admin or system driven). Adds DB-path coverage for discovery firing when unset, skipping
when the credentials endpoint is configured, the write-back, and its negative guards
* Revert "fix(auth): deny model access for teamless keys with all-team-models (#32022)"
This reverts commit dfbbda4f19.
* revert: undo teamless all-team-models denial from PR #29746
Reverts the team_id guard in _resolve_key_models_for_auth_check and
get_key_models so teamless keys with all-team-models resolve to []
(unrestricted = all proxy models) rather than being denied.
Adds hardened regression tests across listing (get_key_models), inference
(_enforce_key_and_fallback_model_access, can_key_call_model,
can_key_call_resolved_model), and batch (_enforce_batch_file_model_access)
paths that enforce teamless all-team-models == all-proxy-models and will
fail if anyone re-introduces a team_id guard
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore: retrigger checks
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
The Router's async fallback orchestrator appended fallback structures
(fallback_model_group, fallbacks, context_window_fallbacks,
content_policy_fallbacks) and the inner fallback exception onto
original_exception.message before re-raising. That message is forwarded
verbatim by the proxy as ProxyException.message. When fallbacks are
configured as inline deployment dicts, the raw provider api_key /
aws_secret_access_key inside those dicts reached any authenticated
caller in the response body.
Route the fallback structures through a new mask_sensitive_structure
helper (reuses the existing SensitiveDataMasker), and wrap the inner
fallback exception string in the existing redact_string. Topology names
still render for debugging under the existing expose_router_debug_in_errors
opt-in; only credential values inside inline-dict fallbacks are masked.
The router's own verbose_router_logger calls that embedded the same
structures are updated alongside, so log output stays consistent with the
exception message.
Verified end-to-end against a real proxy hitting OpenAI: before, the
client response body contained the raw fallback api_key; after, with the
flag on, the api_key value is masked to a 4-char prefix while topology
names are still visible for the operator
* chore(ui): remove debug console.log statements from dashboard
Delete 463 leftover console.log/console.debug calls across 87 files in the
Admin dashboard. These logged form payloads, API responses, and render
traces into every user's browser console.
The ESLint policy already encodes the intent (no-console allows only warn
and error), so those are kept, along with the console.log = function(){}
suppression reassignments and the console.log calls that live inside
string/template literals rendered as example code snippets.
Removal used an AST codemod so only standalone console.log/console.debug
expression statements were dropped; non-statement uses (no-op chart
onValueChange props, a placeholder onClick, and a sequence-expression in
TopKeyView) were handled by hand. Ratchets the no-console lint metric from
484 to 15.
* chore(ui): drop empty blocks left after console.log removal
Greptile flagged three empty control-flow blocks (an if and an else in
chat_completion.tsx, an else in networking.tsx) left behind when their
only content was a deleted console.log. Removes those plus one more empty
if in chat_completion.tsx's catch that the review missed.
* test(ui): drop provider_info_helpers test asserting debug log
The getProviderModels debug console.log calls were removed in this PR, so
the test asserting they fire no longer applies. Remove that test and its
now-unused console.log spy; the remaining 57 tests still cover the
function's actual return-value behavior.
Establishes a real design.md/AGENTS.md for the chat UI (tokens,
component patterns, decision trees) after several rounds of hand-rolled
Tailwind shipping invisible or broken states, then audits every
component in the directory against it: raw <button>s replaced with
shadcn Button throughout, spinners replaced with Skeleton for list/table
loading states, dark-mode contrast bugs fixed (MCPAppsPanel cards were
bg-background instead of bg-card, identical to the page background in
dark mode), Badge variants and status colors aligned with the documented
semantics, and the sidebar's active-nav-item styling switched to the
purpose-built sidebar-* tokens instead of the generic accent/secondary
tokens that collapse to the same value in this theme.
Also: disables model comparison mode and multi-select in favor of a
single active model, moves the model picker from a standalone top bar
into the composer, removes the sidebar collapse toggle and the
non-functional "Search chats" entry, and renames the conversation list's
"Today" group to "Recents".
Fixes a real scroll bug: the model picker's dropdown list was
unscrollable because its container used max-height instead of an
explicit height, which doesn't count as a definite size for the
percentage-height Radix ScrollArea viewport to resolve against — so the
viewport silently expanded to full content height instead of clipping,
and scroll events fell through to the page behind it. Same latent bug
fixed in the sidebar's conversation list.
* fix(anthropic): preserve 1h cache-creation TTL breakdown across streaming usage chunks
Anthropic emits the cache-creation TTL breakdown (ephemeral 5m/1h split) only on
the message_start SSE event; the later message_delta carries the flat
cache_creation_input_tokens count but drops the nested cache_creation object.
ChunkProcessor aggregates prompt_tokens_details last-wins, so message_delta's
details (with cache_creation_token_details=None) clobbered the breakdown captured
from message_start. Cost calc then fell into the flat-rate branch of
calculate_cache_writing_cost and billed 1-hour cache writes at the 5-minute rate,
undercounting the cache-creation cost component by ~37.5% on streaming requests.
Track cache_creation_token_details with the same non-null-wins semantics already
used for the flat cache counts and stitch it back onto the final
prompt_tokens_details when the last chunk lacks it. Non-streaming was unaffected
because its usage is parsed once from the full response body.
* refactor(streaming): extract cache-creation breakdown helpers to stay within strict complexity budget
* test(streaming): cover final-chunk cache-creation breakdown path
---------
Co-authored-by: Richard Warburton <Richard.Warburton@theaccessgroup.com>
* feat(mcp): thread the caller token into tools/list discovery for token_exchange
A token_exchange (OBO) server's tools could not be discovered through the aggregator: the list path
never threaded the caller's token, so every tools/list hit the no-subject branch. v1 masked this with
its client_credentials fallback (discovery used a service token); v2 dropped that fallback, so listing
had no credential and the OBO server's tools never appeared - and an MCP client lists before it calls.
Thread the inbound subject_token into the list path the same way the call path does, gated on
auth_type oauth2_token_exchange so the caller's bearer never leaks into other modes:
_get_tools_from_server takes an oauth2_headers param, extracts the token via _extract_bearer_token, and
passes it to _create_mcp_client; server.py forwards oauth2_headers at the list call site.
authorization_code (resolves off identity plus stored token), the static/config modes, and the
background registry refresh are unaffected, and the list path's existing graceful degradation
(catch -> empty list) is preserved.
* fix(mcp): harden token_exchange OBO from the audit (strip, TTL/expires_in, subject_token_type)
- _should_strip_caller_authorization returns True for oauth2_token_exchange, so the inbound subject
token is never forwarded upstream raw - only the IdP-exchanged token is (matches authorization_code).
- _parse_expires_in accepts a JSON float / numeric-string expires_in, and _ttl_seconds caps the cache
TTL at the token's real remaining lifetime so a short-lived exchanged token is never served stale.
- to_server_spec normalizes a falsy subject_token_type to the default URN, parity with v1.
The subject/key disambiguation (never exchange the LiteLLM key; Authorization: Bearer <litellm-key>
support for /mcp) is intentionally a separate cross-cutting PR off staging, not part of this OBO work.
* fix(mcp): stop caller header bypassing OBO exchange; thread subject into prompts/resources
The per-server x-mcp-* override guard in _create_mcp_client only kept the v2 spec
for authorization_code, so a caller-supplied header silently disabled the RFC 8693
exchange on a token_exchange server and forwarded the raw bearer upstream. Extend
the guard to token_exchange so the exchange always runs and the caller cannot
substitute an arbitrary upstream credential.
prompts/list+get, resources/list+read, and resource-templates/list never threaded
the OBO subject token, so those operations failed closed (401 / empty) on a
token_exchange server. Thread the caller's bearer as the subject for those paths
too, gated on the token_exchange mode via a shared _obo_subject_token helper.
* fix(mcp): keep the OBO/authz_code resolver credential authoritative; centralize OpenAPI strip
A guardrail (e.g. MCPJWTSigner), static_headers, or any other injected Authorization could
shadow the resolver-owned credential for token_exchange / authorization_code servers, so the
upstream would receive e.g. the signer's JWT instead of the exchanged token and reject it. In
_create_mcp_client the resolver-owned credential now wins: a conflicting header is dropped and
the minted/stored token reaches upstream. No behavior change for none/passthrough/static modes,
where an injected Authorization still wins as before.
The OpenAPI/local _request_extra_headers forwarder gated its Authorization strip on
has_client_credentials only, so an OpenAPI-backed token_exchange server with
extra_headers:[Authorization] forwarded the raw subject token upstream and never exchanged. It
now uses the centralized _should_strip_caller_authorization so it matches the managed paths.
* feat(mcp): RFC 9728 challenge for token_exchange (OBO) unauthorized
OBO previously returned an opaque 401 (Bearer error="invalid_request") with no discovery
info, and any IdP exchange failure collapsed to a retryable 503. Now an OBO server behaves like
a standards-compliant OAuth resource server:
- A missing/rejected subject token returns the RFC 9728 / RFC 6750 challenge: 401 +
WWW-Authenticate: Bearer resource_metadata="...", error="invalid_token", so a spec-compliant
MCP client can discover the IdP, SSO, and retry with a fresh subject token.
- The protected-resource metadata for a token_exchange server advertises the JWT-auth issuer(s)
(JWT_ISSUER / litellm_jwtauth.issuers) as authorization_servers -- the IdP that issues and
validates the subject -- instead of the gateway.
- An IdP 4xx (subject rejected) is now a non-retryable 401 (the challenge) instead of a 503, so a
caller with a dead token re-authenticates rather than looping; 5xx/transport stays retryable 503.
* fix(mcp): emit the OBO RFC 9728 challenge preemptively so a no-subject client can discover the IdP
A token_exchange server's tools are not discoverable without a subject token (list is lenient ->
empty), and a tool-call-time 401 is wrapped into a JSON-RPC error so the WWW-Authenticate header is
lost. So a cold-start client never saw the challenge and could not start discovery. Add a
token_exchange branch to the preemptive-401: a no-subject connect to an OBO server now returns
401 + WWW-Authenticate: Bearer resource_metadata=..., error="invalid_token" at the transport level,
so a spec-compliant client discovers the IdP (the PRM advertises the JWT-auth issuer), SSOs, and
retries with a subject token. Verified live on the per-server endpoint; the with-subject connect
still proceeds (no challenge).
(Also formats two lines from earlier commits in this stack.)
* refactor(mcp): inject root_path into the OBO/OAuth challenge edge
The adapter's raise_user_oauth_challenge and raise_token_exchange_challenge
reached into os.getenv("SERVER_ROOT_PATH") via get_server_root_path(), a
hidden ambient read in a module that is meant to be a pure edge. That coupling
made the preemptive-challenge test order-dependent under xdist: a sibling test
sets SERVER_ROOT_PATH at import without cleanup, leaking the prefix into the
challenge URL and failing the exact-match assertion.
Resolve the root path at the imperative-shell call sites and pass it in
keyword-only, so both challenge builders become pure functions of their inputs.
Extract the shared resource_metadata path construction into a single
oauth_protected_resource_path helper, collapsing the duplicated prefix/name
logic the two functions carried.
Also reduce _create_mcp_client below the strict complexity ceiling by extracting
the v2 credential resolution into _resolve_v2_auth, and extract the OBO
protected-resource-metadata branch into _obo_protected_resource_response (which
shipped without coverage) so discovery can be unit-tested directly.
Tests are now hermetic: the adapter tests pass root_path as a real input rather
than monkeypatching the environment, the stale-session preemptive test asserts
structural invariants instead of the exact prefixed URL, and five new tests
cover the OBO PRM issuer branch end to end.
* feat(mcp): OBO cache-key tenant isolation, reactive 401 retry, v1-parity logs
From a pass over the OBO behavior contract. Three changes to the
token_exchange arm, none of which alters any other auth mode.
The exchanged-token cache key now folds in the caller's tenant alongside
the subject token and exchange config, so two tenants presenting the same
opaque token can never share a cache entry; cross-tenant isolation is
structural rather than incidental to subject-token uniqueness. tenant_id is
threaded from the resolver's Subject; it is keyword-only with an empty
default so the no-tenant case and the existing call sites are unchanged.
The tool-call path gains one reactive retry. When an upstream rejects the
injected token with a 401/403, the gateway invalidates the cached exchange,
re-mints once through the IdP by rebuilding the client, and retries the call
exactly once before surfacing the upstream error, so a token revoked or
rotated upstream mid-TTL self-heals without an infinite loop. It is gated
strictly to oauth2_token_exchange; passthrough, authorization_code,
client_credentials, api_key, and none keep their single-call behavior.
MCPClient.call_tool gains a raise_on_error flag (mirroring list_tools) so
the path can tell an upstream 401 apart from an ordinary tool error and
avoid re-running a non-idempotent tool on a non-auth failure.
The exchanger also emits the v1-parity log lines it had dropped (attempt
with server, endpoint and audience; success; cache hit), while never
logging the form, subject token, secret, or minted token.
* fix(mcp): fail closed with 412 when a token_exchange server has no endpoint
A true token_exchange (OBO) server must use only an explicitly configured
token endpoint; it must never guess an IdP or silently fall back to a weaker
source. Previously an OBO server with client credentials but no
token_exchange_endpoint/token_url deferred to v1, which no-op'd and let the
request connect to the upstream with no credential (an upstream 401 rather
than a clear gateway error).
Now such a server is owned by the v2 arm: _token_exchange_spec builds the spec
even when the endpoint is absent, and the exchanger fails closed with a
precondition_required error that maps to HTTP 412 before any upstream or IdP
call, with the caller's subject token never sent anywhere. A missing
client_id/secret still maps to misconfigured (500); a present-but-rejected
subject still maps to 401; an unreachable IdP still maps to 503. The no-subject
case keeps its existing 401 RFC 9728 challenge.
* feat(mcp): log a refused non-Bearer token_type in the OBO exchange
* fix(mcp): surface OBO/authorization_code list-time 401 as a challenge instead of masking it
* feat(mcp): classify RFC 6749 gateway-fault token-exchange errors as 500, not a caller 401
* test(mcp): absorb fixture uses 500 now that 401/403 are challenge-class at list time
* style(mcp): PEP 604 union in the OBO retry signature to keep the UP007 budget flat
* feat(mcp): v2-native RFC 8693 token exchanger for the token_exchange mode
Adds the pure Rfc8693TokenExchanger plus its composition root: the OBO exchange POSTs the
RFC 8693 grant through an injected HTTP edge and returns the upstream-bound token as a typed
Result, caching and single-flighting per (subject_token, server) so a repeated caller token
skips the IdP round-trip. The audience is carried on TokenExchangeConfig and sent only when the
operator set one, matching the spec default behavior. Errors are values: a missing endpoint or
client credential is misconfigured, an IdP that returns no usable token is upstream_unavailable.
* feat(mcp): migrate the token_exchange arm to the v2 resolve_credentials
Routes RFC 8693 OBO servers through the v2 resolver: the resolver arm reads the caller's
inbound token and swaps it via the injected TokenExchanger, to_server_spec maps a complete
oauth2_token_exchange server (endpoint plus client credentials) to TokenExchangeConfig, and the
egress wires the LazyTokenExchanger in. A token_exchange server with no caller token fails closed
with a plain 401 rather than v1's fall-through to client_credentials, so the call site now scopes
the per-server browser-OAuth challenge to authorization_code and lets other modes raise their own.
* fix(mcp): bind the token-exchange cache key to the exchange config
The exchanged-token cache was keyed only by (subject_token, server_id), so rotating a server's
audience, scope, endpoint, client_id, or secret kept serving a token minted for the old config
until TTL. The key now hashes the caller token together with the config that minted it, so a config
change forces a fresh exchange. Everything is hashed, so no secret is held in the key.
* refactor(mcp): build the token exchanger eagerly, dropping the lazy wrapper
The token exchanger reads no runtime global at build time (its httpx client is acquired per call),
unlike the per-user store, so it does not need lazy first-use construction. Building it once at
egress construction removes the first-use init path entirely and keeps the process-lifetime cache.
* fix(mcp): map non-object token-exchange JSON to a miss instead of a 500
The post adapter annotated the parsed body as a dict without checking it, so a valid-but-non-object
JSON response (list/string/number) was returned as-is and crashed the field parsing with an
AttributeError. It now validates the shape at the boundary and returns None for a non-object body,
so a malformed IdP response surfaces as a typed upstream_unavailable rather than a server error.
* fix(mcp): fail closed on a non-Bearer token_type in the OBO exchange
* feat(mcp): honor token_endpoint_auth_method (client_secret_basic) in the v2 OBO exchange
* feat(mcp): reject a non-access issued_token_type in the OBO exchange
* fix: include token endpoint auth method in exchange cache key
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
"AI Gateway" in the topnav view switcher only called setMode(), which
is meaningful inside the dashboard SPA shell but a no-op on /chat,
which lives outside it (only "Chat" had a real navigation). Now
switching modes from the chat route does a real navigation back to
the dashboard root.
Also adds a persistent banner across all chat routes flagging it as a
pre-v0 feature not for production use, with a feedback link.
Moves Chats/Integrations/Credentials/API Keys/Usage from client-side
tab state to real nested routes (/chat, /chat/integrations,
/chat/credentials, /chat/api-keys, /chat/usage) so each is bookmarkable
and survives a hard reload. Extracts the chat sidebar into ChatShell
and shared state (MCP server selection, conversation history) into
ChatShellContext, both consumed via the new app/chat/layout.tsx.
Along the way: fixes conversation URLs pointing at the wrong path
(/ui/chat instead of /chat in dev, which 404'd after sending the first
message) by reusing the existing migratedHref helper instead of a
one-off uiConfig-based path; fixes the topnav view-switcher always
showing "AI Gateway" as selected even while on the chat route; and
cleans up several shadcn/tailwind styling bugs introduced by the antd
migration (boxed tab outline instead of underline, model-selector
dropdown overflowing its popover, sidebar nav labels centered instead
of left-aligned, duplicate logo, dead non-interactive controls).
* fix: prevent duplicate budget alert emails on concurrent threshold crossings
Budget alert emails were sent more than once for a single threshold crossing. The email dedup guard read the "already sent" marker, awaited the send, then wrote the marker, so concurrent requests crossing the same threshold within the send window all saw no marker and each sent. This affected the multi-threshold path (default_key_max_budget_alert_emails), the legacy single-threshold path (EMAIL_BUDGET_ALERT_MAX_SPEND_ALERT_PERCENTAGE), and the soft budget path, all in EmailBaseCallback.budget_alerts
All three branches now claim the send slot atomically before sending via async_increment_cache, which is atomic per event loop for the in-memory cache and across workers via Redis INCR; only the caller that observes a count of 1 sends. On send failure the marker is released with async_delete_cache so a transient failure does not suppress the alert for the full 24h TTL
* fix: harden budget alert claim release and skip-path event allocation
Addresses review feedback on the claim-before-send change. The claim release in each send-failure handler now logs the send error first and releases the claim best-effort through a shared helper, so a transient cache error during async_delete_cache cannot propagate out of the fire-and-forget budget_alerts task, drop the send-failure log, and leave the claim stuck for the full 24h TTL. In the multi-threshold branch the increment claim now runs before the WebhookEvent is built, so skipped concurrent crossings no longer construct and discard the event, matching the single-threshold and soft budget branches
The lint rule allows console.warn (allow: [warn, error]) but removeConsole
only excluded error, so approved console.warn calls were silently dropped
from production bundles. Add warn to the exclude list so the prod strip
and the lint allow-list agree; only console.log/debug/info are stripped
now, warn and error both survive (verified: warn 85 to 85, error 906 to
906, log 675 to 14).
Merging litellm_internal_staging dropped 2 console.log calls (the
currentUser logs removed in #32079), so the no-console budget max and
metric move from 486 to 484 to match the current count.
Introduce a gradual ratchet to remove raw console.* calls from the
dashboard, mirroring the existing no-explicit-any budget.
The no-console eslint rule is set to warn with allow: [warn, error] so
the 486 console.log/debug/info calls are tracked without force-deleting
the legitimate console.error/warn error reporting in catch blocks. The
count is grandfathered via eslint-budgets.json (max 486, target 0) and
eslint-metrics.json, so any newly added console.log fails the budget
check and follow-up PRs grind the max down toward zero.
Independently, next.config strips console output from production builds
via SWC removeConsole (exclude: [error]), gated on NODE_ENV=production so
dev keeps full console output. This gives an immediate prod-hygiene net
regardless of how long the source cleanup takes. Verified against a real
production build: app-code console.log dropped from 675 to 14 in the
bundle (remainder is node_modules, which the transform leaves alone),
console.warn app calls stripped, console.error preserved 906 to 906.
_with_resolved_session_model was overwriting the nested
input_audio_transcription.model and audio.input.transcription.model with the
realtime conversation model, silently replacing a caller's transcription model
(e.g. whisper-1) since those are a different model than the realtime deployment.
It now only resolves the top-level session model.
Also restores session.model taking precedence over the top-level model in
acreate_realtime_client_secret, matching the proxy's own
_prepare_client_secret_session ordering and avoiding a backwards-incompatible flip.
Adds routing coverage for arealtime_calls (api_base resolution) and
acreate_realtime_transcription_session (api_key resolution) so all three realtime
HTTP endpoints have router credential-resolution tests, plus regression tests for
the two fixes above.
Co-authored-by: Cursor <cursoragent@cursor.com>
The ui_unit_tests CircleCI job logged ~45k lines for a single run, most of
it React act() warnings, antd deprecation notices and component stack traces
emitted as console output by passing tests, which buried real failures.
Set silent: "passed-only" (Vitest 3.2+) gated on process.env.CI so console
output from passing tests is suppressed while a failing test still prints its
logs and full stack trace. Also drop two stray console.log calls in
UsagePageView that dumped the whole currentUser object on every render in
production, not just tests.
Verified by running the suite the way CI does
(CI=true npm run test -- --run --pool forks --poolOptions.forks.maxForks=6):
45,075 lines before, 981 after, all 4075 tests still passing. A throwaway
failing test confirms its console.log and assertion diff remain visible.
* chore: clarify the linear ticket instruction in pr template
* fix: make it more concise
* Update CLAUDE.md
lol
* chore: tell claude not to search for it if it doesn't have it
---------
Co-authored-by: ryan-crabbe-berri <ryan@berri.ai>
* fix(anthropic): keep context_management working when drop_params is enabled
drop_params (proxy-wide or per-request) silently disabled the in-gateway
context_management polyfill on the /v1/messages -> chat completions adapter
path, even though context_management is a LiteLLM-supported param (native on
Anthropic, polyfilled elsewhere). Gate the polyfill on an explicit
additional_drop_params: ["context_management"] opt-out instead, which also
makes that escape hatch actually work on the adapter path.
* test(anthropic): cover sync adapter polyfill gate for global drop_params and additional_drop_params
The ui_unit_tests job runs vitest with maxForks=8 on an 8-vCPU xlarge
container, leaving no headroom for the main vitest process that services
worker RPCs. Under full CPU saturation the coordinator misses the
onTaskUpdate ack, vitest raises "Timeout calling onTaskUpdate" as an
unhandled error, and the job exits 1 even though every test passes.
Lower maxForks to 6 so the coordinator, jsdom, and OS keep two cores, and
raise teardownTimeout to 60s for extra slack on heavy runs.
* feat(mcp): add all-proxy-mcpservers sentinel to grant every MCP server
Teams can now be scoped to the all-proxy-mcpservers sentinel so they gain
access to every MCP server on the proxy without listing each id. The
sentinel expands to the live registry at request time, so a server added
later is picked up with no change to the team's stored permission. The team
ceiling that validates a key's MCP scope expands the sentinel too, so a key
can be scoped to any server (including one registered after the team) and
still pass subset validation
Expose the option in the team create and edit forms via a new exclusive
"All Proxy MCP Servers" choice in MCPServerSelector, mirroring the existing
"No MCP Servers" sentinel
* Update litellm/proxy/management_helpers/object_permission_utils.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* fix(mcp): honor all-proxy-mcpservers only on the team path, never per-key
The sentinel was expanded inside the shared expand_permission_list, which
also feeds the key, org, end_user and agent resolvers. A key whose stored
object_permission ever held all-proxy-mcpservers (a stale write, a
configured default, or a bug) would silently resolve to every MCP server at
runtime, and a teamless key had nothing to cap it, so all servers got
injected. Only write-time validation stripping the value stood between that
value and a full grant
Move the expansion out of expand_permission_list and into
_get_allowed_mcp_servers_for_team so the sentinel is honored only where it is
settable (a team). Anywhere else it now passes through as an inert literal
that matches no registered server and is denied downstream. Reserved-id
protection already blocks a real server from taking that id
* fix(mcp): require proxy admin to grant a team the all-proxy MCP sentinel
Granting a team every MCP server on the proxy is a proxy-wide authorization
decision, but team create/update let any caller who can manage a team set
object_permission.mcp_servers, with no ceiling check. Org admins reach
/team/update by default (org_admin_allowed_routes) and _verify_team_access
also admits team admins, so a non-proxy-admin could set all-proxy-mcpservers
and self-grant their team access to every MCP server on the proxy, including
servers never assigned to that team
Gate the grant in new_team and update_team: a non-proxy-admin cannot add the
all-proxy-mcpservers sentinel. The check is scoped to newly adding it, so a
team a proxy admin already scoped to all-proxy can still be edited by a team
admin without being forced to strip the sentinel. The UI only offers the
"All Proxy MCP Servers" option to proxy admins in the team create and edit
forms
* fix(ui): render friendly all-proxy MCP label for non-admins editing an all-proxy team
A team scoped to the all-proxy-mcpservers sentinel could be opened in the team
edit form by a team admin or org admin (canEditTeam admits them), but the
"All Proxy MCP Servers" option in MCPServerSelector was rendered only behind the
proxy-admin-gated allowAllProxyMcpServers flag. For a non-proxy-admin the stored
sentinel was hydrated into the selected value with no matching Select.Option, so
antd showed the raw all-proxy-mcpservers literal as a chip, and adding another
server could persist a mixed [all-proxy-mcpservers, <id>] value.
Render the option whenever the sentinel is present in the value, not only when
the caller may grant it, and drive the real-option disabling off presence too so
the selection stays exclusive. A non-proxy-admin now sees the friendly label
read-only and cannot build a mixed state; only a proxy admin can newly add it,
which the backend already enforces.
Adds regression tests: the selector shows the friendly option (not the raw
literal) when the sentinel is stored but the grant flag is off, plus exclusive
emit and disabled-real-options coverage, and MCPServerPermissions renders the
green "All" state instead of the raw sentinel string.
* fix(ui): drop redundant "All servers" hint from the all-proxy MCP chip
antd renders a Select option's children inside the selected tag, so the
all-proxy option showed both "All Proxy MCP Servers" and the green "All servers"
type-hint in the chip, which say the same thing. Collapse the option to a single
green "All Proxy MCP Servers" label so the dropdown row and the chip read cleanly
without the duplication.
* fix(ui): color the all-proxy MCP label blue to match server chips
Use the same blue (#1890ff) as regular MCP server entries for the
"All Proxy MCP Servers" option/chip instead of green.
* fix(ui): make the all-proxy MCP permissions display blue, not green
Match the blue used by the selector chip and regular server entries so the
"All Proxy MCP Servers" badge and row in MCPServerPermissions are consistent
across the team/key/org detail views. The red "Blocked" state for
no-mcp-servers is unchanged.
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Address Greptile P1 (stale state after reset): use key prop to force
BudgetFallbacksEditor remount when parent resets budgetFallbacks to {},
matching the existing routerSettingsKey pattern.
Address Greptile P2 (inconsistent empty payload): guard budget_fallbacks
in edit view to only include when non-empty, matching create form behavior.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Realtime client_secrets, calls, and transcription_sessions were bypassing
the router and falling back to an empty OPENAI_API_KEY for wildcard, team-scoped,
and credential-name deployments.
Co-authored-by: Cursor <cursoragent@cursor.com>
decrypt_value_helper logged `Unable to decrypt value={value}` at DEBUG, which
printed the raw secret whenever decryption failed (for example after a salt or
master key change). This is the same environment_variables config path the
db-config redaction covers, so a DATABASE_URL connection string could still
leak here when the module regex scrubber is bypassed. Drop the value; the key
already identifies the failing pair.
Regression forces a decrypt failure with the redaction filter disabled and
asserts the raw value never reaches a log record while the key stays visible.
Reuse the existing recursive `_redact_secret_values_in_obj` for the worker
config log instead of a hand-rolled top-level pass, so a credential nested
under general_settings is masked at any depth and depth overrun fails closed.
Route the `_update_config_from_db` param_value log (the store_model_in_db
path) and the litellm_settings apply-loop log through the same redactors, so
master_key, database_url, and secret-named settings such as api_key stop
leaking at DEBUG when the module regex scrubber is bypassed. A plain setting
like num_retries still logs its real value.
Regression tests disable _ENABLE_SECRET_REDACTION and cover the nested worker
config shape, the db-config path, and the litellm_settings loop in both
directions.
Three startup log statements in litellm/proxy/proxy_server.py dumped
secret-bearing values in cleartext when the last-line-of-defense regex
scrubber was bypassed (LITELLM_DISABLE_REDACT_SECRETS=true, older versions
that predated the SecretRedactionFilter, or any downstream handler that
snapshots log records before the module filter runs)
ProxyConfig._load_alerting_settings logged the whole general_settings
dict under a label that only referred to the alerting callbacks; a
copy-paste bug that happened to leak master_key, database_url, and every
other secret sitting in general_settings. Now logs only the alerting
callback list
ProxyConfig.load_config logged the resolved DB URL after secret-manager
resolution. The line's stated purpose was to confirm the retrieval ran,
which does not need the value. Now logs a value-less breadcrumb
proxy_startup_event logged the raw WORKER_CONFIG blob, which docker/K8s
deployments hand the proxy as a JSON string containing master_key,
database_url, and provider API keys. Now routes through
_redact_worker_config_for_logging, which combines the segment-matching
SensitiveDataMasker (catches master_key, api_key, *_token) with an
explicit pass over _EXTRA_SECRET_GENERAL_SETTINGS_FIELDS (catches
database_url and other credential-URL fields the segment masker misses)
Regression tests disable the module-level SecretRedactionFilter so
assertions see the raw record; without the fix they would trip on the
secret substring, so a future refactor cannot silently reconstruct the
leaky string
