* feat(ui): add key-scoped auto-router usage tab
GET /auto_router/benchmarks takes an optional api_key filter, applied in the
rollup aggregate on the primary key's leading column. Proxy admins get a
separate Auto-router usage tab on key detail pages with spend, baseline,
savings, tier routing, cache metrics and the existing router selector
* fix(ui): share key analytics date range
* feat(proxy): resolve root_path per request from SERVER_ROOT_PATHS
One deployment can encode exactly one client-visible URL path prefix
today: SERVER_ROOT_PATH is a scalar stamped onto the app at startup, so
a pod fronting several ingress prefixes 404s every prefix but one before
any handler runs, and MCP OAuth discovery can emit only one prefix's
URLs (RFC 9728 section 3 exact-match fails for the rest).
Add an opt-in outermost ASGI middleware that matches the request path
against a configured prefix list (SERVER_ROOT_PATHS, comma-separated) on
a segment boundary and sets scope["root_path"] for that request only.
Everything downstream is stock Starlette: route matching strips
root_path so routes stay registered root-relative, and request.base_url
re-includes it, so the discovery documents' resource and the 401
challenges' resource_metadata land under the prefix the client actually
called — with no discovery-builder changes.
LazyFeatureMiddleware now strips the scope root_path (falling back to
the cached SERVER_ROOT_PATH scalar) before feature prefix matching, so
lazily-registered routers — the MCP OAuth discovery router among them —
load under per-request prefixes.
Follow-up to the routing discussion on #35226; composes with, but does
not depend on, #35576.
* fix(proxy): import Sequence from collections.abc (ruff UP035 strict-budget gate)
* review(greptile): trim implementation commentary; fixture-own MCP registry state in tests
Addresses both P2s from the first Greptile pass:
- per_request_root_path_middleware.py (and the related _lazy_features /
proxy_server comments) cut down to the constraints the code cannot
express, per repo comment guidance
- the new discovery tests no longer clear/repopulate the shared MCP
registry inline; a fixture snapshots it, hands the test an empty
registry, and restores it afterwards so no state leaks between cases
* fix(lint): mutable-ok marker on the prefix accumulator (LIT002 type-discipline gate)
* fix(proxy): tie 401 challenges and get_custom_url to the per-request root_path
The per-request root_path middleware sets scope["root_path"] to the
prefix the client actually called, but the OAuth 401 challenges
(raise_user_oauth_challenge / raise_token_exchange_challenge) still
built their resource_metadata from SERVER_ROOT_PATH. On a pod fronting
several prefixes, the challenge advertised a discovery URL under a
different prefix than the discovery document served — the two
disagreed on where the resource metadata lives, and a strict RFC 9728
client refused the challenge. Route the challenges through a small
ContextVar the middleware populates so they read the same effective
root_path Starlette resolves the request under.
The same accessor fixes get_custom_url: when a request lives under a
SERVER_ROOT_PATHS-matched prefix, request.base_url already carries it,
so appending the SERVER_ROOT_PATH scalar on top produced e.g.
/tenant-a/legacy/sso/callback — a path that does not exist. Reading
the per-request prefix instead (and relying on join_paths's tail-dedup)
keeps SSO login/callback URLs under one prefix — the one the request
actually arrived on.
Fallback: outside a request (module-load-time UI URL builders,
background tasks) the ContextVar is unset and the accessor reads
SERVER_ROOT_PATH, matching get_server_root_path() so scalar-only
deployments are byte-identical.
* fix(mcp): challenge URL under per-request prefix must route, and mock parity
Two follow-ups to the review fix that made the 401 challenge use the
per-request root_path:
1. oauth_protected_resource_path must pick the URL structure that
actually routes for the mechanism in use:
- The scalar SERVER_ROOT_PATH deployment registers the well-known
routes with the prefix INSERTED (via well_known_root_suffix at
import time), matching RFC 8414 §3. The challenge URL must use the
same insertion or a client fetching it 404s.
- The per-request SERVER_ROOT_PATHS deployment can't register routes
per prefix; PerRequestRootPathMiddleware strips the prefix from
scope["path"] and the router matches the un-inserted route. The
URL must place the prefix BEFORE .well-known so the strip leaves a
matching path.
The previous fix used the insertion form for both, which 404'd the
discovery fetch on the per-request path — the discovery doc and the
challenge would then disagree on where the resource metadata lives,
the very failure the review flagged. End-to-end verified: the URL
the challenge advertises routes and the doc's `resource` field
equals the URL the client originally called (RFC 9728 §3).
2. get_request_root_path now delegates its fallback through
get_server_root_path() instead of reading the env directly, so every
existing `monkeypatch.setattr("litellm.proxy.utils.get_server_root_path"`
test override keeps working. This unstubbed the mock on the /v2/login
test that failed on the last CI run.
Plus the lint budget: annotate the local accumulator Final, tag the
scope["root_path"] rewrite as an intentional ASGI-contract mutation,
tag the reused `path`/`root_path` rebinds in LazyFeatureMiddleware, and
add reason strings to the two new PLC0415 lazy-import noqas.
* test(mcp): pin the reviewer's expected end-state — challenge URL routes, resource matches called URL
End-to-end regression test that mounts the discoverable router + the
per-request root_path middleware, hits an MCP endpoint that raises
raise_user_oauth_challenge, fetches the resource_metadata URL the
challenge advertises, and checks the returned document's `resource`
equals the URL the client originally called (RFC 9728 §3 exact match).
Covers /tenant-a, /tenant-b, and the unprefixed path on the same app so
a regression on any prefix — challenge URL 404s, or doc emits a
different prefix than the client called — fails at this test rather
than in a strict MCP client's discovery.
---------
Co-authored-by: gym-cmd <186399764+gym-cmd@users.noreply.github.com>
* fix(spend_logs): keep partition DDL transactions alive for their statement timeout
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style: ruff format changed files
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(lint): avoid dict-literal kwargs and keep cast-ok on the cast line
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(lint): cast at the call site instead of widening PrismaClient.tx
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(spend_logs): require partition tx timeout to strictly exceed statement bound
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: jesus <jesus@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(e2e): prove the virtual key lifecycle on every replica
Walks one virtual key through create, read, partial update, clear, enforce
and delete against a live proxy and database, reading every write back on
every gateway replica.
The management suite already had single write-then-read tests for keys, but
none of them proved that a partial /key/update leaves the untouched fields
alone, that an explicit null clears a field, or that a write is visible on
more than the one gateway that took it.
Adds read_back_everywhere to the shared ProxyClient: it polls a GET path on
every URL in PROXY_REPLICA_URLS until each replica's parsed body satisfies
the caller's predicate, and fails naming the replica that never converged.
The CLEAR sentinel in the e2e models makes an explicit JSON null expressible
in a body the transport otherwise strips of None fields.
Documents /key/update's merge patch semantics on the endpoint docstring.
* test(e2e): prove key revocation and field preservation on every replica
Applies the findings from an adversarial review of the first commit.
The delete step only checked that chat was refused on the gateway that took
the write, so it would have passed while a sibling gateway kept serving the
deleted key. It now serves one call from every replica first, so each has the
key cached and the delete has something to revoke everywhere, then polls every
replica for the refusal.
The file also carried its own poll loop that tested the deadline before
attempting, so it gave up one attempt early and skipped the attempt landing
exactly on the deadline. It now shares the harness helper, which is generic
over the polled value rather than over a parsed body, so the same loop covers
both the info read-back and the chat refusal.
The model the enforcement step registers now carries a unique marker in its
alias, matching every other deployment this suite creates, so concurrent runs
never share one model group.
The docstring sentence claimed an explicit null clears any field. It does not:
the metadata-backed fields merge into stored metadata, where a null is a silent
no-op, and only the key's own columns clear. Regenerating the dashboard types
picks up the corrected text.
* fix(e2e): delete a deployment that never becomes servable
Registering a model posts /model/new and then waits for every replica to list
it. When that wait timed out the deployment already existed in the database but
its id had never been returned, so no caller could delete it and the row
outlived the run. It is now deleted before the failure propagates.
Found by review on the key lifecycle suite, whose module fixture registers a
deployment this way, but every caller of the shared helper had the same
exposure.
* docs(e2e): drop the duplicated notes from the lifecycle docstrings
The delete method restated what the warm-up helper already explains, and the
module restated the merge patch rule that the endpoint and the request model
both document.
* test(e2e/ui): cover member role and budget edits, member permission delegation, and team guardrail removal
Three Playwright specs for the Teams flows enterprise customers hit most, each
owning its fixtures and proving the mutation through a read-back rather than a
toast.
- teamMemberEdit: an admin edits a member's team role and per-member budget,
and both survive a reload of the Members table
- memberPermissions: a plain member is refused /key/generate for their team,
a team admin grants it on the Member Permissions tab, and the member then
creates a team key that serves a real completion
- teamGuardrailRemoval: clearing a team's only guardrail on the Settings tab
really clears it, and traffic the guardrail refused starts serving again
* test(e2e/ui): make the new team specs safe to run in parallel
Fixture ids came from Date.now(), so two repeats starting in the same
millisecond minted the same user id: one got a 409 and the loser's teardown
deleted the user the other was still signed in as. Ids now carry a random
suffix.
Also move the member-permissions setup inside the cleanup-protected block so a
half-finished setup cannot leak a team, and close both browser contexts the
test opens.
* test(ui): pin wire contracts for key, model and MCP server forms
Add vitest cases that pin what the key edit, key create, model edit and
MCP server edit forms put on the wire: an edited field reaches the
request with its new value, a cleared field reaches it as an explicit
null, and the dirty-only body is pinned as an expected failure until
each form moves to pickDirty. Model edit also pins the cost-map-derived
model_info fields as an expected failure.
KeyEditView hands a cleared max_budget to KeyInfoView as an empty
string and handleKeyUpdate maps it to null, so the null is pinned at
the /key/update boundary in key_info_view.test.tsx and the KeyEditView
case is an expected failure. buildEditServerPayload passes a cleared
description through as an empty string, so that case is an expected
failure too.
* test(ui): split masked model_info pins and retarget the create tracker
The model_info expected-failure case held three assertions, and it.fails
stops at the first one, so a later revamp that fixed max_input_tokens
while leaving mode leaking would still report an expected failure. Split
it into one case per pinned field group so each flips on its own.
The key create tracker asserted a body of only key_alias, which a create
can never send: key_type, user_id, duration and metadata are always
mounted. Retarget it at the real over-send, which is the Optional
Settings section adding fifteen undefined-valued keys when the user opens
it without filling anything in.
* fix(mcp): apply key and team guardrails to MCP tool calls
Guardrails attached to a virtual key or team were only enforced on LLM
routes. The synthetic request built for MCP tool call guardrail hooks
carried no guardrails in its metadata, so a guardrail with default_on
false never ran on tools/call even when the key explicitly listed it.
Resolve key, team, and project guardrails onto the synthetic request
with the same helper the chat path uses.
* fix(mcp): pass project metadata through without a mutable default
* fix(mcp): mark the request dict parameter mutable-ok with a reason
* test(mcp): explain the premium_user patch and tighten the helper docstring
* ci: limit Rust workflows to Rust directory changes
* ci: run Rust checks when their workflow changes
* ci: report Rust wheels only for successful Rust changes
* ci: keep Rust wheel reports in the workflow summary
* ci: group Rust lint and validation jobs
* ci: keep Rust job names distinct from required lint and test checks
* ci: drop the unused Python setup from the Rust lint job
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Env-credential login (UI_USERNAME/UI_PASSWORD, or the master key when
UI_PASSWORD is unset) is always live today. This adds a general_settings
flag to turn that login path off once real admin accounts exist, and a
warning banner shown to any admin while it remains enabled.
The banner flag is served through /health/readiness/details and stays
quiet when disable_password_login_when_sso_enabled already makes the env
path unreachable.
Two params were advertised for the MAI image models and dropped downstream,
so the caller got a 200 that did not match the request, or an opaque
provider 400.
n: get_supported_openai_params returns ["n", "size"], so n passes validation
and is forwarded. The MAI endpoint (/mai/v1/images/generations) has no count
field at all — its documented body is model/prompt/width/height, plus image
for edits — and ignores both `n` and the native `sampleCount`. Measured
against MAI-Image-2.5 and MAI-Image-2.5-Flash: n=2 and n=4 each return HTTP
200 with exactly one image, billed as one, with nothing in the response
saying the request was reduced. A caller balancing cost against image count
cannot see it. n=1 still passes through; n>1 now raises unless drop_params
is set, which is the existing opt-in for silently dropping a param.
size: _map_size_param's table offered five sizes, of which one is usable.
MAI requires width and height >= 768px and width*height <= 1048576, so
512x512 and 256x256 are under the per-side minimum and 1792x1024 / 1024x1792
are over the pixel budget — all four 400 at the provider with "Model does
not support request parameter value supplied: 'width' must be at least 768
pixels." Only 1024x1024 works. The bounds are now checked where the size is
mapped, so the error names the constraint instead of arriving from Azure.
width/height are deliberately left unchecked: they pass through unmapped, so
a future MAI model with different bounds stays reachable without a code
change.
Verified on a live Azure AI Foundry deployment of MAI-Image-2.5 and
MAI-Image-2.5-Flash (2026-08-17). One existing test asserted the 1792x1024
mapping; its size is changed to a size the provider accepts, keeping what it
was testing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
CI flagged the spec flaky twice more. Both were the same defect in different
places: the Logs drawer renders several nodes per string and the first in DOM
order is often hidden, so .first() waited 20s on an invisible element. The
entity assertions had a second problem on top, since getByText("EMAIL_ADDRESS")
substring-matched the masked prompt div, whose text contains <EMAIL_ADDRESS>,
rather than the entity chip.
Route every drawer assertion through onlyVisible, and match the entity type
and score exactly, which is what the panel renders them as: entity_type and
"Score: N.NN" each get their own span.
Verified on a live stack: 6 of 6 solo runs and the guardrails folder 5 of 5.
Mutating the analyzer to detect nothing turns the spec red on the raw address
reaching the spend log, so the assertions still carry their weight.
The Logs drawer renders the masked prompt in three nodes and the first one
in DOM order is hidden, so the previous commit's .first() traded a strict
mode violation for a locator that waits 20s on an invisible element. Local
runs against a warm stack failed on it every time, resolving the node 22
times and reporting "unexpected value hidden" each time.
Use onlyVisible, the helper this spec already uses for the playground
selectors, which filters to the visible node before taking the first. Solo
runs go 3 for 3 and the guardrails folder passes 5 of 5.
Also drop the 30s wait on the PII step added while chasing this: cold start
was never the cause, and it left the spec sitting on a dead locator longer.
The Logs drawer renders the masked prompt in three places, so matching it
without narrowing raised a strict mode violation instead of asserting
visibility. CI caught it as a flake: the spec failed its first attempt on
b0e53bfbe4 and passed on retry, which is a locator defect rather than a
timing one and would have gone red on any run that saw all three nodes.
Narrow to the first match, matching the guardrail-name assertion above it.
The leak checks below stay on toHaveCount(0), which is unaffected by
multiple matches and is what actually proves nothing raw reached the drawer.
An include entry that matches both a file next to the config that declares it and
one next to the root config now warns naming both, so a config that resolves to a
different file than it used to says so instead of quietly serving other models.
Also from reviewing that change:
- an empty root object in a bucket fails the boot again instead of coming up empty
- a YAML syntax error in a bucket object logs its own line naming the object
- an include already loaded is skipped before it is read rather than after
- reading a config out of GCS builds the plain bucket client, so it needs no
enterprise license and starts no flush loop that nothing ever cancels
