* feat: add silent CLI token refresh for apiKeyHelper support
lite auth print-token prints a valid proxy credential for use as Claude
Code's apiKeyHelper, transparently refreshing it first if the cached JWT
is stale. This unblocks MDM-managed apiKeyHelper deployments (managed
via `lite auth print-token`) that need silent mid-session credential
rotation without restarting the client.
Refresh capability is backed by a virtual key minted with an empty model
list and cli_refresh metadata, kept strictly separate from the actual
(short-lived, real-model-scoped) call credential -- so a leak of the
credential that flows through every LLM request and subprocess env var
can't also self-renew. The refresh flow is single-use: /sso/cli/refresh
mints a fresh JWT + refresh token pair and blocks the presented refresh
token immediately, so a replay can't mint a second pair from it.
Server: /sso/cli/refresh (rotate) and /sso/cli/logout (revoke) endpoints.
lite login now also stores a refresh token; lite logout revokes it
server-side instead of only clearing the local file.
* fix: allow non-admin users to hit CLI refresh routes; resolve apiKeyHelper base_url from token.json
Found via a live end-to-end test against a real proxy + real Claude Code
session: /sso/cli/refresh and /sso/cli/logout were unreachable for any
non-proxy-admin caller, since Depends(user_api_key_auth) pulls in a
route-RBAC gate that 403s any route not on an explicit allowlist. That
made the feature unusable for actual end users, who authenticate as
internal_user. Add both routes to internal_user_routes; the handlers
already do their own fine-grained check (metadata.cli_refresh) same as
/key/block does today.
Also: `lite auth print-token` required an explicit --base-url/
LITELLM_PROXY_URL matching the stored token's origin, defaulting to
localhost:4000 otherwise. But apiKeyHelper is configured bare (no
flags), so this always mismatched a real deployment. Track whether
--base-url was explicitly passed (via click's ParameterSource) and, if
not, resolve the server from token.json directly instead of the CLI
default.
* test: mock refresh-token minting in test_cli_poll_key_tolerates_missing_user_row
Landed on litellm_internal_staging after this branch's refresh-key minting
change; needs the same mock as the other cli_poll_key tests since minting
now runs unconditionally whenever a JWT is generated.
* fix(ci): update test_cli_auth.py for refresh_token contract, regenerate schema.d.ts
_poll_for_authentication now always includes "refresh_token" in its
returned dict, and _handle_team_selection_during_polling returns a dict
instead of a bare JWT string -- test_cli_auth.py predates this branch's
refresh-token work and still asserted the old shapes.
schema.d.ts regenerated via `npm run gen:api` to pick up the new
/sso/cli/refresh and /sso/cli/logout routes (plus unrelated drift from
other PRs merged since it was last generated).
* fix(ci): apply CI's own schema.d.ts diff (enterprise routes I can't generate locally)
Local `npm run gen:api` only sees OSS routes -- this machine's
litellm_enterprise editable install points at a now-deleted temp
directory, so it silently drops enterprise-only routes from the spec.
Applied the exact diff CI's own generation produced instead of
re-running the generator locally.
* fix: close refresh-token race, fail closed on DB down, fix logout base_url
Addresses Greptile review findings on the CLI refresh-token PR:
- cli_refresh_token minted a new JWT + refresh token BEFORE blocking the
presented one. Two concurrent requests bearing the same refresh token
could both pass auth and both mint fresh pairs, yielding four live
credentials from one consumed token. Now the presented token is
consumed atomically first via update_many (only succeeding if it flips
blocked from False/None to True); the loser gets count=0 and is
rejected before anything is minted.
- When prisma_client is None, refresh silently returned a new JWT
without ever being able to mark the presented token consumed, leaving
it valid indefinitely. Now fails closed with a 500 instead.
- `lite logout` sent its revocation POST to ctx.obj["base_url"], which
defaults to localhost:4000 when --base-url isn't passed -- the same
bug print_token had before the base_url_explicit fix, just missed
here. Now resolves the same way: trust the stored token's origin
unless the caller explicitly overrode --base-url.
* fix(ci): satisfy ruff format and narrow token_data type in logout
* fix(security): never trust refresh-token metadata for authorization
Addresses a real privilege-escalation path Veria flagged: cli_refresh_token
read team_id, team_alias, and max_budget straight off the presented
token's own metadata and used them to authorize the new JWT. Since any
authenticated user can self-mint a virtual key with arbitrary metadata
via the ordinary /key/generate endpoint, a self-forged key with
{"cli_refresh": true, "team_id": "<any-team>", "max_budget": 999999999}
would sail through _require_cli_refresh_token's only check
(metadata.cli_refresh == True) and get a JWT scoped to a team the
caller never belonged to, with a budget it never had -- full
cross-team / budget bypass, and a removed team member could keep
refreshing team-scoped sessions indefinitely.
Metadata's team_id is now treated as an untrusted UX hint only: honored
solely if the CALLER (identified by the authenticated key's own
user_id, not client input) is a current member per a fresh
get_user_object lookup. team_alias and max_budget are never read back
from metadata at all -- team_alias comes from a live get_team_object
lookup and max_budget is recomputed with the exact same capping logic
the initial SSO login poll uses. _mint_cli_refresh_token no longer
accepts or stores team_alias/max_budget, only the team_id hint.
Added regression tests proving: a forged/stale team_id is dropped
(falls back to no team, not silently honored), and a forged max_budget
in metadata never reaches the issued JWT.
* fix(ci): catch HTTPException specifically instead of bare Exception (BLE001)
* fix: un-consume refresh token if minting the replacement fails
Greptile flagged a real reliability gap: cli_refresh_token blocks the
presented token atomically, then does several more DB calls before
returning a replacement (user lookup, team lookup, JWT mint, new
refresh-key mint). Since this endpoint exists specifically for fully
unattended apiKeyHelper operation, a single transient failure in that
window (DB hiccup, etc.) permanently stranded the user: their old
token was already dead and no new one was issued, with no recovery
path short of a full interactive browser re-login.
Wrap that window in try/except; on any failure, best-effort revert the
consumed token back to usable (blocked=False) before re-raising, so a
retry can succeed. Standard compensating-action pattern since
generate_key_helper_fn doesn't take an injectable transaction, so
wrapping the whole thing in a real DB transaction isn't practical here.
* fix(security): refresh key had unrestricted model access, not none
Critical bug: _mint_cli_refresh_token used models=[] intending "no LLM
access", but that's backwards in this codebase. Per
_check_model_access_helper: `len(filtered_models) == 0 and len(models)
== 0` -> all_model_access = True. An empty models list on a key with no
team_id means UNRESTRICTED access to every model, not zero access. The
CLI refresh token -- meant to be usable for nothing but silently
exchanging itself for a new JWT -- was actually a fully unrestricted
API key for its entire 90-day lifetime, completely undermining the
whole point of keeping it separate from the short-lived call
credential.
Fixed with two independent layers: allowed_routes hard-restricts the
key to exactly /sso/cli/refresh and /sso/cli/logout (the real enforced
boundary, checked in the shared user_api_key_auth dependency for every
route); models is set to an unmatchable sentinel string as
defense-in-depth in case any code path only consults the models field.
Added an end-to-end regression test that exercises the actual
model-access-control function against a key shaped like the minted
refresh token, rather than only asserting on what arguments were passed
to the key-generation call -- the latter kind of test is exactly what
let the original bug ship, since asserting `models == []` is equally
consistent with "no access" and "unrestricted access" without checking
what the access-control code actually does with that shape.
Also: the compensating-rollback added for reliability un-blocked a
consumed refresh token even when the underlying user no longer exists.
That's a permanent, intentional rejection, not a transient failure --
un-blocking it would let a stale refresh token become valid again for a
different account if the user_id is ever reused/re-registered. Moved
the user-existence check outside the rollback-on-failure block so it
stays permanently blocked.
* refactor: rotate CLI refresh tokens via regenerate_key_fn instead of hand-rolled consume/rollback
The refresh token is already a plain litellm virtual key, so rotation can
delegate to the same atomic DB update /key/regenerate uses instead of a
bespoke update_many + compensating-rollback dance. This makes silent CLI
refresh an Enterprise feature, same as regular key regeneration.
* refactor: replace CLI stateless JWT + refresh-key pair with one self-rotating virtual key
The CLI previously minted two credentials on login: a stateless self-signed
JWT for LLM calls, and a separate DB-backed refresh-only key (scoped away
from ever calling an LLM) just to authorize minting a new JWT. Collapse
this into a single real virtual key, used directly as the LLM bearer token
and re-presented to /sso/cli/refresh to rotate its own secret in place.
This also means the CLI session key now shows up in the Admin UI's Keys
page and can be revoked/regenerated like any other key, rather than being
an invisible, unmanageable stateless token.
* refactor: drop silent CLI refresh, key just expires and requires re-login
/sso/cli/refresh only ever benefited Enterprise deployments (regenerate_key_fn's
gate), while everyone else already fell through to "re-run lite login" on
failure. Cut the endpoint, the rotation logic, and the client-side refresh
path entirely; print-token now just prints the cached key until it hits its
LITELLM_CLI_JWT_EXPIRATION_HOURS duration, then fails fast telling the user
to log in again. Session key itself is unaffected: still a real, revocable
virtual key visible in the Keys UI, `lite logout` still revokes it directly.
* fix(ci): regenerate schema.d.ts after removing /sso/cli/refresh route
* revert: go back to stateless JWT, keep only lite auth print-token
The virtual-key redesign (revocable, Keys-UI-visible credential) wasn't
needed just to support print-token, and cost real server-side surface
(a mint path, a logout-revoke endpoint, migrated tests/docs) for a property
this repo doesn't need yet. Reverting cli_poll_key/_types.py/schema.d.ts
back to the original stateless-JWT design; the only durable addition from
this whole effort is `lite auth print-token` (reads the cached credential,
prints it while fresh, fails with a clear message once it's past
LITELLM_CLI_JWT_EXPIRATION_HOURS) plus the base_url_explicit plumbing it
needs. `lite logout` goes back to clearing the local file only, since a
stateless JWT can't be revoked server-side.
* refactor: move CLI token freshness check to cli_token_utils, drop unnecessary renames
Addresses review: the freshness check is a pure token-shape/timestamp
util, not command logic, so it belongs alongside the other SDK-level
CLI token helpers (load_cli_token, get_litellm_gateway_api_key) rather
than in commands/auth.py. Also reverted a few incidental jwt_token/
session_key variable and string renames that weren't load-bearing.
Streaming requests return from common_request_processing before
async_post_call_success_hook runs, so response._hidden_params.additional_headers
never gets the v3 x-ratelimit-{descriptor_key}-{remaining|limit}-{rate_limit_type}
entries. Prometheus / logging callbacks that read those values from
standard_logging_object.hidden_params.additional_headers then see nothing;
combined with the pre-existing gap that Prometheus reads from that same slot
(LIT-2577 / PR #28816), per-key remaining RPM/TPM cannot be monitored for
streaming traffic at all.
Fix in three parts:
- Stash the pre-call RateLimitResponse in the metadata channels the async
success-logging callback inherits, alongside the existing top-level entry
the non-streaming path reads.
- Add async_logging_hook to the v3 handler. It fires in a distinct earlier
loop inside async_success_handler (all callbacks' async_logging_hook
complete before any async_log_success_event starts), so mirroring the
pre-call snapshot into standard_logging_object.hidden_params.additional_headers
and response._hidden_params.additional_headers here guarantees every
downstream success callback sees the values regardless of registration
order. Non-streaming keeps the existing async_post_call_success_hook write
and this hook re-populates the same values idempotently.
- Extract the shared `_merge_ratelimit_statuses_into_additional_headers`
helper the non-streaming path already had inlined so both callsites emit
the identical key shape.
Replace the request.path_params read (and its defensive getattr guard) with
the route template. A real Starlette request always exposes path_params, but
common_checks runs on lightweight request doubles that don't, so reading it
directly forced a getattr workaround that only existed to tolerate those
doubles.
Instead, match the route template (/team/{team_id}) to identify the RESTful
update route and take the team id from the last path segment. This drops the
path_params dependency entirely, and because the template distinguishes the
PATCH route from its single-segment siblings (/team/new, /team/list, ...), it
also avoids a spurious team lookup those routes would otherwise trigger if we
matched the resolved path shape alone.
A dynamically registered (RFC 7591) OAuth client persisted onto the MCP server row is bound to the redirect_uri it was first registered with, but that binding was never recorded. After the proxy's public origin changed, every authorize paired the reused client with the new callback and the IdP rejected it permanently.
The DCR persist now records redirect_uris alongside the client identity. The admin register path treats a positive mismatch between the recording and the current callback as stale and re-registers a replacement client; rows without a recording (pre-existing installs and admin-configured clients) are grandfathered so upgrades never re-mint client_ids or orphan refresh tokens. The persist also writes client_secret and token_endpoint_auth_method explicitly as None when absent so the credential blob merge cannot pair a re-registered public client with the previous client's secret. Public register routes and non-admin callers keep existing behavior.
Closes#32473
apply_json_merge_patch recurses into nested objects, which the repo's recursive_detector code-quality check flags because unbounded recursion over caller-supplied JSON has caused CPU/stack issues before. Cap the recursion at a depth far above any realistic team-metadata shape and reject deeper patches with a ValueError so a pathologically nested body fails closed instead of overflowing the stack, then register the function in the detector's ignore list alongside the other depth-bounded JSON walkers
Wire the coarse route gate so PATCH /team/{team_id} is reachable by exactly the roles that can call POST /team/update: proxy admins, org admins of the team's own organization, and JWT admins. Regular internal users and view-only proxy admins stay blocked, matching the existing endpoint
Because the team id lives in the path rather than the body, the org-context resolver now also reads it from path_team_id for the bare /team/{team_id} route, so an org admin's organization is resolved and injected the same way it already is for POST /team/update. /team/{team_id} is added to management_routes rather than the role-agnostic self_managed_routes; the latter would have opened POST /team/new to any authenticated user through the shared /team/{team_id} path pattern
Add a RESTful PATCH /team/{team_id} that partially updates a team using RFC 7386 JSON Merge Patch. team_id comes from the path, and metadata is merged with the team's stored metadata instead of being replaced wholesale the way POST /team/update does: an omitted key is preserved, key: null deletes it, and any other value overwrites, recursing into nested objects. Every other field behaves the same as POST /team/update
The handler delegates to the existing update path, so authorization, budget checks, system-managed-key stripping, metadata encryption, cache refresh, and audit logging are shared rather than reimplemented. POST /team/update is untouched, so the change is purely additive
A trailing slash on --base-url (or LITELLM_PROXY_URL) produced
double-slash URLs like https://host//sso/cli/start, which 404s. Normalize
once in the CLI's top-level group callback so every subcommand benefits.
* fix(proxy): match list/dict guardrail_mode in compliance mode checks
* test(compliance): cover ComplianceChecker guardrail_mode shapes (str/list/dict/None)
* fix(proxy): trust only Mode.default in compliance mode matching (ignore tag overrides)
* fix(proxy): match dict guardrail_mode only when every branch runs in mode (no false-compliant)
* fix(proxy): treat multi-mode guardrail_mode as unresolved (no false-compliant)
The list branch previously counted a guardrail configured with mode:
[pre_call, post_call] under every listed mode. But when the writer cannot
infer the concrete hook that fired (apply_guardrail invocations), the raw
list is logged, and an image-only request that only reaches the post-call
path still records both modes. That let a pre_call compliance check pass on
a request that only ran post_call.
Match the tightened dict semantics: a list now counts for mode only when
every listed mode equals mode. Same trade-off (under-report instead of
false-COMPLIANT). Speculative set support is dropped (spend logs are
JSON-serialized, sets do not cross the wire).
Tests updated to reflect the tightened list semantics, deduplicated (single
TestModeMatching class), and shortened. The invariant is now expressed as
a computed check: True implies every branch runs in the matched mode.
---------
Co-authored-by: Marton Schneider <marton@schneider.co.nl>
* fix(proxy): build redis usage cache from REDIS_* env when cache backend is not Redis
Selecting a semantic (or any non-Redis-KV) response cache left
redis_usage_cache unset, silently downgrading cross-pod rate limits,
parallel-request limits, spend coordination, and the pod lock manager
to per-pod in-memory state. Fall back to a standalone RedisCache built
from REDIS_* environment variables, mirroring the existing
use_redis_transaction_buffer escape hatch, which now shares the same
helper.
Resolves LIT-3861
* feat(proxy): configure the coordination redis independently of the response cache
Adds general_settings.coordination_redis, an explicit block for the Redis
the proxy uses for cross-pod rate limits, parallel-request limits, spend
tracking, the pod lock manager, and shared health checks. Resolution order
is the explicit block, then a plain-Redis response-cache backend, then the
REDIS_* environment. Cluster and sentinel targets are supported, and a
cluster target now builds a RedisClusterCache so cluster-aware consumers
take the cluster path.
Admins can configure it from the Caching page of the dashboard via
/coordination_redis/settings, which reports which source is in effect,
redacts credentials on read, and offers a connection test. Settings saved
there are read back at startup so they take effect on restart.
Also fixes redis client construction so an explicitly configured host
outranks REDIS_URL in the environment. Previously the url branch stripped
the caller's host and port, so an explicit block, or a connection test
typed into the dashboard, silently targeted whatever REDIS_URL named
* fix(ui): move coordination_redis_settings into renamed _components directory
---------
Co-authored-by: Yucheng Zhu <yucheng@berri.ai>
Selecting a semantic (or any non-Redis-KV) response cache left
redis_usage_cache unset, silently downgrading cross-pod rate limits,
parallel-request limits, spend coordination, and the pod lock manager
to per-pod in-memory state. Fall back to a standalone RedisCache built
from REDIS_* environment variables, mirroring the existing
use_redis_transaction_buffer escape hatch, which now shares the same
helper.
Resolves LIT-3861
A correctly signed JWT whose user_id or server_id claim was an empty string
passed claims validation but raised ValidationError from the EnvelopeIdentity
constructor inside open_envelope, breaking its never-raises guarantee. The
claims model now mirrors the identity's min_length constraints, so any claim
set that validates also constructs, and the empty-identity case maps to
MalformedPayload like every other bad claim shape.
Pure, unwired module: mints and opens the single client-held bearer that
carries both a litellm identity and the encrypted upstream OAuth grant with
zero server-side storage. HS256 JWT signing (same approach as the BYOK
session bearer) plus the existing encrypt_value/decrypt_value symmetric
helpers, with all key material and the clock injected as parameters. Opening
returns typed frozen error values (not_an_envelope, bad_signature, expired,
malformed_payload, decrypt_failed); minting rejects envelopes over
MAX_ENVELOPE_BYTES with a typed error instead of truncating. Error values
and reprs never carry token material.
* fix(spend-logs): honor store_prompts_in_spend_logs for guardrail_information (LIT-4314)
_get_spend_logs_metadata passed guardrail_information entries through
verbatim, so guardrail hooks that echo the LLM request into
guardrail_response leaked the raw prompt into LiteLLM_SpendLogs.metadata
regardless of store_prompts_in_spend_logs. This mirrored the pre-existing
gap for the other prompt-carrying fields (vector_store_request_metadata,
error_information, etc.), which already sanitize via
_should_store_prompts_and_responses_in_spend_logs.
Add _sanitize_guardrail_information_for_spend_logs alongside the other
per-field sanitizers and wire it into _get_spend_logs_metadata. When the
flag is False the sanitizer replaces guardrail_request and
guardrail_response with REDACTED_BY_LITELM_STRING while preserving every
other typed field on the entry (name, provider, mode, status, timings,
action, violation_categories, risk_score, masked_entity_count, ...) so
guardrail dashboards keep working. When the flag is True (or the field
is None) the entries pass through unchanged.
Widen StandardLoggingGuardrailInformation.guardrail_request from
Optional[dict] to Optional[Union[dict, str]] so the redacted sentinel
satisfies the TypedDict without needing a cast; guardrail_response
already accepted str.
Regression tests cover the three cases (flag=False redacts,
flag=True passes through, None passes through) plus an end-to-end
get_logging_payload path that fails if the wire-in at line 139 is
reverted.
* chore(spend-logs): review nits (one-shot dict build, scrub identifier in tests)
- _redact_prompt_fields_in_guardrail_entry now returns the redacted
dict in one expression instead of seed-then-mutate (TYPE-3)
- swap the illustrative guardrail_name in the new test fixtures for
a generic 'demo-echo-guard' identifier
* chore(spend-logs): only redact guardrail prompt fields when caller supplied them
Greptile P2: the sanitizer was unconditionally writing REDACTED_BY_LITELM
into both guardrail_request and guardrail_response on the copy, so entries
that never carried one of those fields (e.g. a guardrail that only emits
a guardrail_response) came out with a phantom guardrail_request key added.
Guard both assignments with an in-check so the output shape is stable.
Add a mutation-checked regression test that fails if either guard is
removed.
* fix(spend-logs): also redact match_details and classification in guardrail_information
The initial LIT-4314 fix redacted guardrail_request and guardrail_response,
but two other typed fields on StandardLoggingGuardrailInformation also
carry raw prompt content when a first-party guardrail populates them:
- litellm_content_filter/content_filter.py:1676 sets classification =
dict(CompetitorIntentDetection), whose evidence[*].match is a substring
taken directly from the user's normalized prompt (see
litellm_content_filter/competitor_intent/base.py:184-194).
- block_code_execution/block_code_execution.py:571 sets match_details =
guardrail_response = [dict(d) for d in detections], where detections
carry the fenced-code-block content extracted from the user's message.
Reproduced live against localhost:4000 with store_prompts_in_spend_logs
false and a custom guardrail passing tracing_detail with both fields:
before this commit the raw prompt shows up in metadata.guardrail_information[0]
under match_details and classification; after, both are the sentinel.
Widen the two TypedDict fields to Optional[Union[..., str]] so the
sentinel string satisfies the schema without a cast, and consolidate
the redaction set into a tuple so future prompt-carrying additions are
one-line changes.
* fix(spend-logs): normalize non-list guardrail_information shapes in sanitizer
xecguard's logging hook (xecguard.py:246) assigns a bare dict to
standard_logging_object['guardrail_information'] instead of a list,
violating the typed contract Optional[List[StandardLoggingGuardrailInformation]].
Without defensive normalization, _sanitize_guardrail_information_for_spend_logs
iterates the dict's string keys and _redact_prompt_fields_in_guardrail_entry
raises TypeError on {**'guardrail_name'}, which get_logging_payload's
downstream update_database catches with a broad except and silently drops
the entire spend-log write for that request.
Normalize a bare-dict input to a single-item list at the sanitizer's
entry point, and skip any non-dict entries defensively (matching OTEL's
existing isinstance filter at opentelemetry.py:1751-1753 for the same
field). Downstream readers already model this defensively; make the
spend-log write path match.
The root cause is xecguard's writer, not the sanitizer. That is being
tracked as a separate ticket; this PR keeps xecguard-enabled deploys
from silently losing spend logs when store_prompts_in_spend_logs=false.
* fix(types): declare guardrail Union members str-first to avoid poisoning typing cache
CPython's typing module caches Union[...] order-insensitively (first-
construction wins), and litellm/types/utils.py has no 'from __future__
import annotations', so its unions are constructed eagerly at import
time -- before any proxy model. Declaring guardrail_request,
classification, and match_details with dict-first ordering seeds the
typing cache with a dict-first tuple, and later proxy models that
declare custom_llm_provider / model_aliases / vertex_credentials as
Optional[Union[str, dict]] pick up the same dict-first object.
Downstream, Pydantic's get_args() then reports anyOf in dict-first
order, FastAPI emits the OpenAPI accordingly, and 'npm run gen:api'
produces a schema.d.ts diff on unrelated fields, tripping the schema-
sync CI check.
Behaviorally identical in Python and at the wire; the flip only reorders
the union members so the first construction matches how the codebase
had always declared these unions, and 'npm run gen:api' now produces a
zero diff against the committed schema.d.ts.
The multi-server list path already relays an upstream 401 from a client-forwarded
server (true_passthrough / oauth_delegate) as an MCPUpstreamAuthError so the caller
re-runs its own upstream OAuth. The single-server REST call path did not: an upstream
401 was masked as a graceful isError result, so an MCP client holding an expired
upstream token never learned it had to re-authenticate
Relay the upstream 401 on the call path too. For these modes the manager calls the
client with raise_on_error=True, extracts the WWW-Authenticate through the existing
upstream-auth exception walk, and raises MCPUpstreamAuthError; the REST endpoint turns
it into a real 401 + WWW-Authenticate. Only 401 is treated as a re-auth signal (a 403 is
a genuine authorization failure that re-auth will not fix, so it stays a masked isError
with a visible warning), matching the list path and MCPUpstreamAuthError's contract. The
legacy oauth2 + delegate_auth_to_upstream mode is deliberately left off the call-path
relay since it is being removed
To keep this expected caller-must-reauth signal from tripping error-rate alerts, the
client layer logs at debug when the caller opted into raise_on_error and therefore owns
the exception (both call_tool/list_tools and the run_with_session helper they share, so an
expected re-auth emits no warning per call either), the manager's non-auth branch logs the
exception type only (never str(e), which for an httpx error embeds the upstream URL a
credential can hide in), and the streamable and REST handlers log the relayed 401 at info
rather than as an error with a traceback
Tests cover the manager raising on a client-forwarded 401 while keeping a 403/503 as a
masked isError, the client-layer debug-vs-error logging split, the streamable handler's
informational isError, and the REST endpoint relaying both the direct and virtual
mcp_tool_call branches as a real 401 + WWW-Authenticate; each was mutation-checked to fail
when the corresponding behavior is broken
Keep the full provider exception in server-side logs only; the client
receives a fixed actionable message. Also follow implicit exception
context when detecting context window overflows and pin the detection
variants plus the redaction in tests
Resolves LIT-4284
When the embedding model exceeded its context window, the MCP semantic
tool filter silently passed all tools through and reported N->N success
in the filter header; when the overflow happened while embedding tool
descriptions at router build time, the hook was never registered at all
and filtering was silently disabled
Semantic filtering now fails closed on context window overflows: the
request is rejected with HTTP 400 and a message that names the embedding
model and advises switching to one with a larger context window or
disabling the filter. Build time overflows are recorded on the filter so
the hook still registers and blocks MCP tool requests with the same
actionable error while leaving native-only requests untouched. The
dashboard test panel renders the backend message in an error banner
instead of a success state. OpenAI's embedding overflow message
(maximum input length is N tokens) now maps to ContextWindowExceededError
The per-row delete_many loop becomes a single delete filtered to the enumerated OAuth users'
(user_id IN, server_id) pairs; same rows deleted, same BYOK-sparing precision, same count-mismatch
detection, one round-trip instead of N
LiteLLM_MCPUserCredentials stores BYOK API keys in the same column as per-user
OAuth tokens, so the purge on a mint-relevant config change now deletes only
rows whose payload decodes as an OAuth2 credential, each by its
(user_id, server_id) pair, instead of every row for the server. An api_key
server whose url changes purges nothing. delete_mcp_server now also
invalidates each enumerated user's cached token so a re-created server reusing
the id cannot serve tokens minted for the deleted one, and both cache drops
are best-effort
The snapshot read only feeds the stale-token purge decision; leaving it unguarded meant a failed
read would 500 an edit whose update would have succeeded, and it broke
test_edit_mcp_server_redacts_credentials, whose mocked prisma is not awaitable on the un-patched
get_mcp_server path. A failure now logs and skips the purge, consistent with the purge half already
being best-effort. Adds the first endpoint-level coverage of the edit purge wiring: purge on a
mint-relevant change, no purge when the identity is unchanged, and edit success with purge skipped
when the snapshot read raises
The purge takes an injectable invalidate_token_cache callable defaulting to the manager's shared
invalidation, and MCPServerManager takes an injectable per_user_token_cache alongside the existing
per_user_oauth_token_store, so tests inject fakes instead of monkeypatching the global manager and
the module-level cache. The new identity helpers drop Any for object throughout
Review follow-ups on the stale-token invalidation. The backend identity now decrypts client_id and
client_secret before comparing: the stored values are NaCl-encrypted with a fresh nonce on every
write, so comparing ciphertext flagged every routine save as a mint-relevant change and purged
per-user tokens that were still valid. The identity also gains spec_path, the audience for OpenAPI
servers, and parses credentials stored as a JSON string
The purge now routes each (user, server) through the manager's invalidate_user_oauth_token_cache,
which becomes the single invalidation point covering both the legacy per-user token cache and the
v2 per-user OAuth token store; previously the purge evicted only the legacy cache while the revoke
path evicted only the v2 store, so each path left the other cache serving a replaced token until
its TTL. A credential row racing in between the find and the delete is now detected via the
delete_many count and logged; its cache entry expires by TTL
On the dashboard, CLEARED_ON_INVALIDATION and the staleness check move to types.tsx as the single
shared implementation for both forms. The edit form's transport handler now rechecks the identity
after its programmatic setFieldsValue calls, which antd does not report through onValuesChange, so
a token no longer survives a transport switch that clears the mint target. The create form rebuilds
formValues from the post-reset form state after an invalidation instead of publishing the pre-reset
snapshot, so the tool preview can no longer refetch with the discarded DCR client. Both transport
handlers now share the recheck, which also stops the create form from over-invalidating on an
http to sse swap that keeps the same url and therefore the same audience
