The stale-redirect path logged three warnings for a single re-registration: the staleness probe plus the reuse skip in both register_client_with_server and the persist race guard. The reuse-skip message is a mechanical consequence of the probe's decision, so it now logs at debug; the actionable warning that names both bindings and the re-authentication impact is emitted once by _persisted_dcr_redirect_uri_is_stale
A dynamically registered (RFC 7591) OAuth client persisted onto the MCP server row is bound to the redirect_uri it was first registered with, but that binding was never recorded. After the proxy's public origin changed, every authorize paired the reused client with the new callback and the IdP rejected it permanently.
The DCR persist now records redirect_uris alongside the client identity. The admin register path treats a positive mismatch between the recording and the current callback as stale and re-registers a replacement client; rows without a recording (pre-existing installs and admin-configured clients) are grandfathered so upgrades never re-mint client_ids or orphan refresh tokens. The persist also writes client_secret and token_endpoint_auth_method explicitly as None when absent so the credential blob merge cannot pair a re-registered public client with the previous client's secret. Public register routes and non-admin callers keep existing behavior.
Closes#32473
The PATCH /team/{team_id} org-context wiring reads request.path_params to
resolve the team id from the path. A real Starlette Request always exposes
path_params, but common_checks is exercised with lightweight request doubles
that don't, which raised AttributeError. Read it defensively so a missing or
null path_params falls back to no path team id, matching the "not a bare team
route" outcome; real requests are unaffected
apply_json_merge_patch recurses into nested objects, which the repo's recursive_detector code-quality check flags because unbounded recursion over caller-supplied JSON has caused CPU/stack issues before. Cap the recursion at a depth far above any realistic team-metadata shape and reject deeper patches with a ValueError so a pathologically nested body fails closed instead of overflowing the stack, then register the function in the detector's ignore list alongside the other depth-bounded JSON walkers
delete_model popped the auto_routers/complexity_routers registries by the deleted
deployment's model_name without checking it was actually an auto_router/* deployment.
Deleting a regular DB model that merely shares a name with a config-defined router
therefore evicted that router, which add_deployment never restores, leaving it
unroutable until a proxy restart. This is the same cross-tenant DoS clear_cache was
hardened against; mirror its auto_router/ prefix guard here.
Extracts _deployment_name_and_model to read model_name and litellm_params.model from
the deployment (delete_deployment returns the raw model_list dict at runtime despite
its Deployment annotation), and adds a regression test asserting a same-named config
router survives deletion of an unrelated regular model.
The AI Gateway selector now always renders at the breadcrumb root, even with no plugins and Chat UI disabled, so the Chat feature stays discoverable. The Chat entry is always listed: clickable when enabled, and disabled with an "Admins can enable in Settings" hint when it is off.
Since the selector is now unconditional, the useViewSwitcherVisible hook and the section-crumb fallback added in the previous commit are removed
Concurrent first requests each hit asyncio.to_thread to build the SemanticRouter
index, firing duplicate embedding calls for the static route utterances. Guard the
lazy build with a per-router asyncio.Lock (double-checked) so the index is
constructed exactly once regardless of how many callers race in cold.
Adds a regression test asserting ten simultaneous cold-start requests build the
index the same number of times as a single request, and reworks the fake embedding
router to count builds by how often a route utterance is embedded (robust to which
embedding path the library uses) while still recording sync-call thread ids for the
off-event-loop assertion.
The AI Gateway select (ViewSwitcher) now sits at the root of the DashboardHeader breadcrumb instead of on the right, so the top bar reads [AI Gateway select] > Page to match the redesign. It keeps the same dropdown, including the Chat / Chat UI options.
When no plugins are registered and Chat UI is disabled there is nothing to switch between, so the breadcrumb falls back to the static section crumb rather than rendering a dangling leading separator
Wire the coarse route gate so PATCH /team/{team_id} is reachable by exactly the roles that can call POST /team/update: proxy admins, org admins of the team's own organization, and JWT admins. Regular internal users and view-only proxy admins stay blocked, matching the existing endpoint
Because the team id lives in the path rather than the body, the org-context resolver now also reads it from path_team_id for the bare /team/{team_id} route, so an org admin's organization is resolved and injected the same way it already is for POST /team/update. /team/{team_id} is added to management_routes rather than the role-agnostic self_managed_routes; the latter would have opened POST /team/new to any authenticated user through the shared /team/{team_id} path pattern
Exact cost-map hits resolve before fallback-generalization rules, so the
mapped Sonnet 5, Fable 5 and jp Opus 4.8 Bedrock entries bypassed the
bedrock-anthropic-claude-mid-conversation-system rule and hoisted
mid-conversation system messages, invalidating the prompt cache.
Add deterministic keyword-to-tier overrides and optional embedding-based
(semantic) keyword matching to the complexity router, and surface both in the
Add Auto Router UI behind a Router Type selector: "Auto-Router v2 [Recommended]"
(complexity tiers + keyword overrides + semantic matching, the default) and
"Semantic Router [to be deprecated]" (the existing utterance-based router,
unchanged). Keyword-to-tier overrides resolve to the highest tier matched
rather than the first keyword matched, so match order no longer affects the
routing decision.
Backend:
- config: KeywordTierRule model plus keyword_tier_rules, semantic_keyword_matching,
embedding_model, and match_threshold on ComplexityRouterConfig, with a validator
requiring an embedding model and rules when semantic matching is on
- complexity_router: evaluate keyword rules before scoring; lexical matches escalate
to the most-severe matched tier (order-independent), and semantic mode reuses
LiteLLMRouterEncoder + SemanticRouter to match paraphrases by cosine similarity,
falling back to the scorer when nothing matches
- model management: clear complexity_routers on cache reload so config edits take effect
Frontend:
- Add Auto Router tab restores the Router Type radio (Auto-Router v2 recommended
by default, Semantic Router still available) and sends keyword_tier_rules plus
the semantic settings on the recommended path, instead of flattening keywords
into custom_technical_keywords
- client-side guard blocks submit when semantic matching is enabled without an
embedding model or without any keyword tier rules, mirroring the backend validator
- moved the "How Classification Works" explainer below Custom Technical Keywords
and above Keyword Tier Overrides
- remove the Test Connection action from the recommended flow, which can't build a
valid pre-save payload for a router (leaves a TODO for a JSON preview / config
test follow-up)
Tests cover lexical escalation, semantic matching via the real library with injected
embeddings, the semantic config guard, config validation, the reload-clear
regression, and the frontend payload builder
* fix(anthropic): translate adaptive thinking/effort to pre-4.6 model support
AnthropicMessagesConfig now reshapes the 4.6+ adaptive-thinking interface
(thinking:{type:adaptive} + output_config:{effort:...}) to whatever the routed
model supports. Thinking-capable non-adaptive models (e.g. Haiku 4.5, Sonnet 4.5)
get the effort translated to a legacy thinking budget_tokens. Models with no
reasoning support have thinking/effort dropped under drop_params. And because
adaptive thinking carries no budget while the legacy form must satisfy Anthropic's
max_tokens > budget_tokens rule, the translated budget is capped below max_tokens,
dropping thinking when max_tokens can't fit the minimum budget. 4.6+ models pass
through untouched.
This matters because clients like Claude Code speak native Anthropic /v1/messages
and send the adaptive interface unconditionally, regardless of the routed model.
The native passthrough previously only capability-gated the OpenAI-style
reasoning_effort alias and forwarded native output_config/adaptive thinking raw, so
a pre-4.6 model rejected it with "This model does not support the effort parameter"
and the request failed. Claude Code already gets drop_params auto-set, so its
requests now succeed.
* test(anthropic): gate undersized-max_tokens thinking drop on drop_params; add edge tests
Addresses review feedback on the max_tokens-too-small branch. Previously a
thinking-capable model whose max_tokens could not fit the minimum thinking budget
had thinking silently dropped regardless of drop_params, while a residual
output_config field in the same call still raised when drop_params was off. Gate
both consistently on drop_params: raise a clear error (naming max_tokens for the
undersized case) when drop_params is off, drop otherwise. Claude Code gets
drop_params auto-set, so it still succeeds.
Adds tests for the undersized-max_tokens raise, the residual output_config raise,
and the no-adaptive-interface passthrough on a non-adaptive model.
* fix(anthropic): make adaptive-effort translation silent to avoid breaking provider strip contracts
The previous raise-when-not-drop_params behavior broke existing bedrock and vertex
messages tests: those providers already silently strip unsupported output_config
for pre-4.6 models (issue #22797) with no drop_params required, and the shared
parent transform raising pre-empted that. It also conflicted with the goal of
keeping requests working rather than failing them.
Make the reshape silent: translate effort to legacy thinking for thinking-capable
models, drop thinking for non-reasoning models, and remove only the consumed effort
key from output_config, leaving any residual (e.g. format) for provider subclasses
(bedrock/vertex) to handle. No raise, no drop_params gating. This also resolves the
review note about inconsistent drop_params handling by making every path uniform.
Updates the tests to assert the silent behavior and residual output_config
preservation.
* fix(anthropic): handle output_config-capable but non-adaptive models (Opus 4.5)
Greptile caught a real bug: the early-return guard treated supports_output_config
as equivalent to supporting adaptive thinking. Claude Opus 4.5 advertises
supports_output_config (it accepts output_config.effort) but is not adaptive, so it
rejects thinking:{type:adaptive} with "adaptive thinking is not supported on this
model". The guard early-returned for Opus 4.5 and forwarded the adaptive thinking
block raw, reproducing the exact failure the fix is meant to prevent.
thinking:{type:adaptive} and output_config.effort are independent capabilities.
Only early-return for adaptive-thinking models. For a model that supports
output_config.effort but is not adaptive, keep the native effort and drop only the
unsupported adaptive thinking block. Verified live against Opus 4.5: the Claude Code
payload now returns 200 instead of 400.
Adds regression tests for Opus 4.5 with and without adaptive thinking.
* fix(anthropic): translate adaptive thinking for effort-capable pre-4.6 models
Claude Opus 4.5 advertises supports_output_config but not adaptive thinking,
so the early-return guard forwarded thinking.type=adaptive raw and Anthropic
rejected it. The guard now only skips true adaptive models; effort-only
requests on effort-capable models still pass through untouched. The
_map_reasoning_effort call is wrapped to surface unrecognized effort values
as a clean 400, matching _translate_reasoning_effort_to_anthropic
* fix(anthropic): fall back to legacy thinking when effort level unsupported
Opus 4.5 accepts output_config.effort but only low/medium/high; Claude Code
defaults to xhigh on newer models, so preserving that level raw gets rejected
by Anthropic. Gate the native-effort passthrough on _validate_effort_for_model
and fall through to the budget translation for unsupported levels
* fix(anthropic): keep effort-only requests untouched for provider normalization
The xhigh fall-through consumed effort-only requests on effort-capable
models, breaking bedrock invoke's own normalization which clamps xhigh to
the model's ceiling after the base transform runs
(test_bedrock_messages_normalizes_output_config_effort_for_opus). Restrict
the fall-through to requests that carry adaptive thinking; effort-only
requests pass through so provider subclasses keep owning level clamping
---------
Co-authored-by: Abhimanyu Kapur <38531241+akapur99@users.noreply.github.com>
vertex_ai/claude-opus-4-8@default (and sibling @default models) were
misclassified as non-adaptive because _model_map_lookup_candidates only
stripped provider prefixes but never the @<suffix> portion. The lookup
produced candidates like ["vertex_ai/claude-opus-4-8@default",
"claude-opus-4-8@default"], neither of which exists in model_cost, so
_is_adaptive_thinking_model returned False. LiteLLM then sent
thinking.type=enabled to a @default Vertex AI endpoint that requires
thinking.type=adaptive, resulting in a 400.
_strip_version_suffix now removes @<suffix> from each candidate,
adding the bare model name (e.g. "claude-opus-4-8") to the lookup
chain. Also adds supports_adaptive_thinking: true to the three
@default model_cost entries that were missing it as belt-and-suspenders.
Co-authored-by: Deepanshu <deepanshu.lulla@alpha-sense.com>
adds a top-level guard in _increment_remaining_budget_metrics that returns early
when all four budget gauges are NoOpMetric (excluded from prometheus_metrics_config),
and per-entity guards in each _set_*_budget_metrics_after_api_request helper for
partial disabling. eliminates four async DB/cache round-trips per successful LLM
request when budget metrics are disabled.
Co-authored-by: Deepanshu <deepanshu.lulla@alpha-sense.com>
* fix(responses-api): raise APIError on in-stream error events; widen ErrorEventError.param
- BaseResponsesAPIStreamingIterator._maybe_raise_for_error_event inspects each
chunk and raises litellm.APIError for type=error and type=response.failed events
so callers see an exception instead of a benign stream chunk
- rate_limit* codes map to 429; client error codes (invalid_request_error,
context_length_exceeded, etc.) map to 400; all other codes default to 500;
raw integer codes are never used as-is as HTTP status codes
- ErrorEventError.param widened from Optional[str] to Optional[Union[str, Dict]]
to prevent Pydantic ValidationError on dict-typed param payloads silently
dropping error events before any type inspection
* test(responses-api): add streaming iterator error event tests to CI-covered path
* test(responses-api): cover response.failed, dict-error, null-error, and sync iterator paths
* test(responses-api): set completion_start_time on mock logging objects for internal staging _process_chunk
* fix(responses-api): map insufficient_quota to 429, derive failed-response log status from error code, and record failed-stream usage for spend accounting
insufficient_quota moves out of the 400 bucket; OpenAI returns HTTP 429 for it and the non-streaming exception mapping treats 429 as RateLimitError, so the in-stream mapping now agrees
_handle_logging_failed_response previously hardcoded APIError(status_code=500), so a rate-limited response.failed was logged to integrations as 500 while the caller saw 429; it now shares the same error-code-to-status mapping via _error_event_fields and _status_code_for_error_code
usage carried on a response.failed event is now stashed as combined_usage_object with its computed cost on the logging object before failure handlers run, reusing the mid-stream-interruption spend recovery path (_failure_handler_helper_fn, proxy post_call_failure_hook, _ProxyDBLogger), so failed streams count their billed tokens instead of logging zero cost
dedupe: TestMaybeRaiseForErrorEvent in tests/llm_responses_api_testing duplicated tests/test_litellm/responses/test_streaming_iterator_error_events.py, which is the canonical mirrored location and CI-covered via test-unit-responses-caching-types; the duplicate class is removed
* fix(responses-api): wrap retriable in-stream errors in MidStreamFallbackError and map error type field to status
Mirror chat streaming semantics from _handle_stream_fallback_error: 429 and
5xx in-stream error events now raise MidStreamFallbackError carrying the
mapped APIError so the router's FallbackResponsesStreamWrapper triggers
mid-stream fallback and cooldown; non-retriable 4xx still raise APIError
directly. Status mapping now reads both the OpenAI error type and code
fields, so type-classified client errors (e.g. invalid_request_error with
code invalid_prompt) map to 400 instead of falling through to 500.
* fix(responses-api): accumulate streamed output text so mid-stream fallback continues instead of restarting
MidStreamFallbackError was always raised with generated_content="", so the
router's stream_with_fallbacks treated every mid-stream error as pre-first-chunk
and retried with the original input, streaming duplicated content to clients
that had already received partial output. The iterators now accumulate
response.output_text.delta text (mirroring chat's response_uptil_now) and pass
it as generated_content, letting the router build a continuation input via
_build_responses_continuation_input.
* test(responses-api): pin in-stream token limit error to raised APIError
---------
Co-authored-by: Deepanshu <deepanshu.lulla@alpha-sense.com>
* fix(bedrock): gate in-place system role messages on model support for Claude Invoke
* feat(bedrock): default unmapped Claude 4.8+ to in-place system role handling via fallback rule
* fix(responses): preserve reasoning_tokens through chat->responses usage translation
Remove the unconditional else-branch that wrote reasoning_tokens=0 whenever
completion_tokens_details.reasoning_tokens was None or absent. Also change
OutputTokensDetails.reasoning_tokens from int=0 to Optional[int]=None so that
re-instantiation without explicit reasoning_tokens no longer silently zeroes out
the field, and remove the same hardcoded zero from the mock_responses_api_response
initializer.
* test(responses): update assertions to match Optional[int] reasoning_tokens default
* fix(responses): preserve explicit reasoning_tokens=0 in usage translation
Align the reasoning_tokens guard with the is-not-None guards used for
text_tokens and image_tokens: a provider-reported zero passes through
while an absent value stays omitted.
---------
Co-authored-by: Deepanshu <deepanshu.lulla@alpha-sense.com>
* fix(bedrock): add jp.anthropic.claude-opus-4-8 to model cost map
* test: use apac regional profile for cost-map fallback test since jp now has an entry
* feat(router): add LLM-based classifier option to complexity router
Adds classifier_type: "heuristic" | "llm" to complexity_router_config.
When set to "llm", the router calls a configured model (e.g. a small
model like haiku) via structured output to pick the complexity tier,
falling back to the existing regex/keyword scorer on any error, empty
response, or unparseable output.
* feat(ui): add classifier_type option to complexity router UI, fix edit flow
Adds an "Advanced: Classification Method" section to ComplexityRouterConfig
with a heuristic/LLM toggle, revealing a classifier model picker and timeout
when LLM is selected.
Also fixes the auto router edit modal, which never rendered the complexity
router UI at all (it only handled the semantic router), and the "Edit Auto
Router" button visibility check, which was gated on auto_router_config and
never matched complexity router deployments.
* fix(router): attribute classifier calls to caller, raise default timeout
Forwards the original request's litellm_metadata into the classifier's
acompletion call. Without it, the proxy's cost-tracking gate sees no
user_api_key/team_id/user_id and silently drops spend logging and budget
accounting for every classifier call, letting an authenticated user rack
up unaccounted provider spend via repeated requests.
Also raises the default classifier timeout from 400ms to 3000ms (400ms
undershoots real LLM latency and would silently degrade to the heuristic
scorer on most requests) and corrects the module/class docstrings, which
still claimed zero external API calls after the llm classifier path was
added.
* fix(ci): resolve ruff strict-budget and frontend-lint failures
- Use PEP 585 generics (dict/tuple/list) in the new aclassify/_classify_with_llm
signatures instead of typing.Dict/Tuple/List, and suppress BLE001 on the
intentionally broad except in aclassify's fallback path with a reason.
- Fix prettier formatting in ComplexityRouterConfig.tsx.
- Regenerate eslint-metrics.json (was stale after the classifier UI changes).
* fix(ci): regenerate stale eslint-metrics.json
* fix(router): strip parent budget reservation from classifier metadata
The classifier's internal acompletion call previously forwarded the
parent request's full litellm_metadata, including its budget
reservation (user_api_key_budget_reservation / user_api_key_auth).
That reservation belongs to the routed completion the classifier is
deciding on, not to the classifier call itself, so it's now stripped
while key/team attribution fields are still forwarded for spend
logging.
Add a RESTful PATCH /team/{team_id} that partially updates a team using RFC 7386 JSON Merge Patch. team_id comes from the path, and metadata is merged with the team's stored metadata instead of being replaced wholesale the way POST /team/update does: an omitted key is preserved, key: null deletes it, and any other value overwrites, recursing into nested objects. Every other field behaves the same as POST /team/update
The handler delegates to the existing update path, so authorization, budget checks, system-managed-key stripping, metadata encryption, cache refresh, and audit logging are shared rather than reimplemented. POST /team/update is untouched, so the change is purely additive
DataTableFilterField rendered a bare native label element. Swap it for the
shadcn Label primitive so the filter drawer's field labels share the same
typography and disabled-state handling as the rest of the form primitives
The workflow-runs Status filter leaked the internal "__all__" sentinel as its
displayed value because Base UI's Select.Value renders the raw value when no
items map or children function is given. Drop the sentinel and use Base UI's
native null handling: a null "All statuses" item plus a placeholder, with an
items map so a real selection renders its capitalized label rather than the
raw status string
The shared DataTable rendered every loading-skeleton cell as one identical
half-width bar, which read as a rigid grid instead of the table beneath it.
Vary the skeleton width per column and add a per-column skeleton shape hint
(text or twoLine) on ColumnMeta so identity columns like the workflow "Run"
cell get a two-line skeleton that matches their real content
A trailing slash on --base-url (or LITELLM_PROXY_URL) produced
double-slash URLs like https://host//sso/cli/start, which 404s. Normalize
once in the CLI's top-level group callback so every subcommand benefits.
Adds server- and client-side column filtering to the shared DataTable via a
filterMode prop that mirrors the existing sorting and pagination modes, a global
search filter, a staged filter drawer ("Apply Filters" / "Reset") built on the
shadcn Sheet primitive (added via `npx shadcn add sheet`), and a table-aware
toolbar (search, refresh, active-filter chips, and a Columns menu). Also adds
skeleton loading rows, an icon empty state, and renders toolbar, table, and
pagination as a single card.
Migrates the two pilot tables onto the design as a proof of concept:
TeamVirtualKeysTable drives server-side filtering through the list API (key alias
via the search box, user via the drawer), and WorkflowRuns filters client-side in
memory. Both keep their existing behavioral tests green, with new coverage for the
filter drawer, the toolbar, the global search, and the server-filter query mapping.
Addresses PR review: _to_dict special-cased LitellmParams and returned an
empty dict for any other pydantic model. Switch to isinstance(value,
BaseModel) so it coerces any pydantic model uniformly (BaseModel is already
imported for the response models), which also drops the now-unused
LitellmParams import. Behavior is unchanged for the current call sites.
The /guardrails/usage/{overview,detail,logs} endpoints resolved guardrails only
from the litellm_guardrailstable Prisma table, so guardrails defined in
config.yaml (which live only in IN_MEMORY_GUARDRAIL_HANDLER) were invisible:
detail 404'd, overview omitted them or rendered them as Custom/Guardrail
orphans, and logs missed their logical-name alias.
Add config-owned accessors (list_config_guardrails, get_config_guardrail_by_id)
to the in-memory handler and use them in the usage endpoints, mirroring the
union/fallback already used by list_guardrails_v2 and get_guardrail_info. Also
preserve guardrail_info when storing a config guardrail (type/description were
dropped at initialize time) and read the Prisma-row / dict / LitellmParams
shapes uniformly.
Resolves LIT-2529
* fix(proxy): match list/dict guardrail_mode in compliance mode checks
* test(compliance): cover ComplianceChecker guardrail_mode shapes (str/list/dict/None)
* fix(proxy): trust only Mode.default in compliance mode matching (ignore tag overrides)
* fix(proxy): match dict guardrail_mode only when every branch runs in mode (no false-compliant)
* fix(proxy): treat multi-mode guardrail_mode as unresolved (no false-compliant)
The list branch previously counted a guardrail configured with mode:
[pre_call, post_call] under every listed mode. But when the writer cannot
infer the concrete hook that fired (apply_guardrail invocations), the raw
list is logged, and an image-only request that only reaches the post-call
path still records both modes. That let a pre_call compliance check pass on
a request that only ran post_call.
Match the tightened dict semantics: a list now counts for mode only when
every listed mode equals mode. Same trade-off (under-report instead of
false-COMPLIANT). Speculative set support is dropped (spend logs are
JSON-serialized, sets do not cross the wire).
Tests updated to reflect the tightened list semantics, deduplicated (single
TestModeMatching class), and shortened. The invariant is now expressed as
a computed check: True implies every branch runs in the matched mode.
---------
Co-authored-by: Marton Schneider <marton@schneider.co.nl>
gemini/gemini-3.1-flash-image, vertex_ai/gemini-3-pro-image, and
vertex_ai/gemini-3.1-flash-image existed in the root pricing JSON but not in
litellm/model_prices_and_context_window_backup.json, leaving deployments with
LITELLM_LOCAL_MODEL_COST_MAP=True unprotected. Copies the root entries into
the backup verbatim and extends the regression test to cover all ten gemini
image models, asserting each exists in the local cost map so a missing backup
entry fails the test instead of passing vacuously
