The floor job now runs with a read-only token and hands the tier and summary to a second job through job outputs. Only that report job holds pull-requests and checks write, and it never checks out code, so a compromised dependency or setup action in the compute step cannot forge labels or check runs. A PR that edits the workflow file itself still runs its own copy, which the ruleset's two-approval .github/** path covers
The workflow now triggers on pull_request instead of pull_request_target, which the auto-approve one-pager and the 2026-06-18 CI exposure post-mortem both forbid, and the job skips fork PRs since the read-only token cannot label them. The floor now follows the rubric's author row (Devin-opened low, human-opened medium, fork high) through a new authors.low list, and adds a paths.medium list for agent instruction files, conftest, helm, and docker. Two guards compare both sides of a file through git show: model map rows that change or disappear stop being low, and a budget file that raises a limit stops being low. A skip or importorskip call added to an existing test file with no net new test scores high. The last 400 staging merges score 218 high, 179 medium, 3 low; the author row alone moves 37 human-opened docs and test PRs from low to medium
The two MCP globs only matched files directly under mcp_server, so faults/render_oauth.py, which renders the OAuth and DCR fault responses, scored medium. Both globs now use **/ so nested files match too. The last 400 staging merges still score 217 high, 139 medium, 44 low
The workflow now runs on pull_request_target so the base branch's copy of
the script and config scores every PR, forks included, and a PR can no
longer edit the gate that scores it. The PR head is only ever read as a
diff. Forks are passed --from-fork so the author factor is live
The diff parser reads git renames, so a pure move counts zero lines and
both paths take part in the paths and modules factors. The size factor
counts only files outside the docs, tests, and model map tiers, and
.only( joins the silenced-test markers
Config: every pyproject.toml is always-human, the MCP auth, credential,
OAuth, and discovery files are always-human, and the lint budget files
are low. The last 400 staging merges now score 217 high, 139 medium,
44 low
The author factor now only raises a PR to high when it comes from a fork,
so an internal human-opened docs or test-only PR can reach the low tier
the one-pager's tier 0 needs. Over the last 400 staging merges that moves
the low count from 2 to 29 with no reverted merge among them
Skip markers only count when they silence a test unconditionally:
pytest.mark.skipif, unittest.skipIf/skipUnless, a guarded pytest.skip(),
and Playwright's conditional test.skip(cond, reason) no longer rank a PR
high. Test directories anywhere in the tree count as test files, and the
enterprise auth and management endpoint packages join the always-human
paths
* fix(guardrails): accept on_violation block and alert for mcp_security
The MCP Security policy template sends on_violation: "block", but the shared
LitellmParams model only allowed the /v1/realtime values "warn" and
"end_session", so POST /guardrails returned 422 before the MCP guardrail was
initialized. Widen the literal to include the MCP actions, map every non-alert
value to MCP's default "block" at init, and regenerate the lazy OpenAPI
snapshot and dashboard API types
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(guardrails): restrict on_violation block/alert to mcp_security and keep legacy MCP mapping
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(guardrails): return 422 when PATCH sets an mcp_security-only on_violation on another guardrail
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Several file references under proxy/management_helpers/ and other
paths no longer exist; the code moved to proxy/common_utils/,
proxy/db/db_transaction_queue/, litellm_enterprise/proxy/common_utils/,
proxy/hooks/litellm_skills/, and litellm_core_utils/.
* fix(router): keep provider response headers on streaming chat completions
The Router re-wraps a deployment's CustomStreamWrapper in FallbackStreamWrapper
(and its sync twin) so a mid-stream failure can fail over. Neither wrapper
forwarded `_response_headers`, so every streaming chat completion handed the
proxy's callbacks and its response-header builder a wrapper with no provider
headers, and a successful mid-stream fallback still published the failed
deployment's identity, `x-request-id` and rate limit counters.
Forward `_response_headers` into both wrappers, repoint the wrapper at the
deployment that served the stream once a fallback takes over, and rebuild the
proxy's response headers from that deployment while `create_response` still has
the first chunk buffered.
* fix(router): follow a nested fallback to the deployment that served the stream
A fallback the router picks is itself a fallback-aware wrapper, and it only
repoints at its own fallback once it yields, so reading its hidden params at
selection time named a deployment that produced no output. Re-read them when
the first fallback item arrives, which is still before the proxy commits
response headers.
Also addresses review feedback: the streaming header builder reads self.data
instead of taking a coarse request_data parameter, and the new test recorder
local is Final.
* test(router): cover the fallback header adoption helper directly
The router_code_coverage gate wants every router.py function named in a
router test, and this also pins the weak-reference behavior: a wrapper
collected mid-stream must not break the generator still draining it.
* refactor(proxy): take a read-only mapping for the model-id lookup
_get_model_id_from_response only reads its request payload, so a Mapping
says what it needs and the two metadata hops are narrowed instead of
assumed to be dicts.
* test: drop mutable recorder locals and routine comments from the new tests
An AsyncMock await_count and an asyncio.Event say the same thing as a
list and a dict that the test mutates.
* chore(router): justify the two rebinds in the fallback loops
Both are the one-shot re-read that follows a nested fallback, so they get
the repo's rebind-ok note like the rest of the file.
* fix(ui): show inherited MCP servers on the internal-user editor and flag access groups with no members
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): consult the unfiltered access group registry before calling a group empty
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(skills): semantic search over the LiteLLM-hosted skill registry
Adds GET /v1/skills?query= (custom_llm_provider=litellm_proxy) and a
skill_search MCP virtual tool, ranking the caller's accessible skills by
semantic similarity, mirroring the A2A agent registry search (LIT-6309).
Also fixes a pre-existing bug where create_skill() dropped description and
instructions for the litellm_proxy provider, which left every LiteLLM-hosted
skill with no searchable text.
* fix(mcp): coerce skill_search top_k instead of raising 500 on malformed input
The MCP-REST skill_search dispatch validated raw tool arguments through a
pydantic model directly, so a non-numeric top_k raised a ValidationError
that the endpoint's catch-all turned into an HTTP 500. Mirrors the
agent_search branch's tolerant coerce_top_k handling instead.
* fix(skills): enforce key limits on search embeddings and bound the semantic index
Semantic search embeddings now run the same pre_call_hook the /embeddings
route runs, so key rate limits, budgets and guardrails apply before the
embedding model is called. The shared SemanticTextIndex caps cached vectors
and evicts the least recently searched entries, and each skill's embedded
text is capped so one skill cannot inflate the embedding batch
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(skills): surface proxy 429s from search embeddings instead of a 503
ProxyRateLimitError is also an OpenAIError, so the search engine was folding
a key rate limit into skill_search_unavailable. Proxy HTTPExceptions now
propagate so the caller gets the same 429 the /embeddings route returns
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(skills): import assert_never from typing_extensions for Python 3.10
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(skills): embed the request as the pre-call hooks returned it, not the original text
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(skills): keep the litellm_proxy provider check for GET /v1/skills?query= inside llms/
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(skills): move the GET /v1/skills?query= endpoint tests under tests/test_litellm/proxy
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): hide the Create Vector Store flow from non proxy admins
The vector stores page rendered the Create Vector Store tab, the
+ Add Vector Store button and a GET /credentials call for every role,
while the proxy only lets proxy admins call POST /vector_store/new and
GET /credentials. Internal users landed on the create form and got an
Only proxy admin error toast. Gate all three on isProxyAdminRole and
default everyone else to the Manage tab, matching the Indexes tab and
the Add Model gating.
Resolves LIT-7131
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): exclude view-only admin sessions from the vector store create flow
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): add key-scoped auto-router usage tab
GET /auto_router/benchmarks takes an optional api_key filter, applied in the
rollup aggregate on the primary key's leading column. Proxy admins get a
separate Auto-router usage tab on key detail pages with spend, baseline,
savings, tier routing, cache metrics and the existing router selector
* fix(ui): share key analytics date range
* feat(proxy): resolve root_path per request from SERVER_ROOT_PATHS
One deployment can encode exactly one client-visible URL path prefix
today: SERVER_ROOT_PATH is a scalar stamped onto the app at startup, so
a pod fronting several ingress prefixes 404s every prefix but one before
any handler runs, and MCP OAuth discovery can emit only one prefix's
URLs (RFC 9728 section 3 exact-match fails for the rest).
Add an opt-in outermost ASGI middleware that matches the request path
against a configured prefix list (SERVER_ROOT_PATHS, comma-separated) on
a segment boundary and sets scope["root_path"] for that request only.
Everything downstream is stock Starlette: route matching strips
root_path so routes stay registered root-relative, and request.base_url
re-includes it, so the discovery documents' resource and the 401
challenges' resource_metadata land under the prefix the client actually
called — with no discovery-builder changes.
LazyFeatureMiddleware now strips the scope root_path (falling back to
the cached SERVER_ROOT_PATH scalar) before feature prefix matching, so
lazily-registered routers — the MCP OAuth discovery router among them —
load under per-request prefixes.
Follow-up to the routing discussion on #35226; composes with, but does
not depend on, #35576.
* fix(proxy): import Sequence from collections.abc (ruff UP035 strict-budget gate)
* review(greptile): trim implementation commentary; fixture-own MCP registry state in tests
Addresses both P2s from the first Greptile pass:
- per_request_root_path_middleware.py (and the related _lazy_features /
proxy_server comments) cut down to the constraints the code cannot
express, per repo comment guidance
- the new discovery tests no longer clear/repopulate the shared MCP
registry inline; a fixture snapshots it, hands the test an empty
registry, and restores it afterwards so no state leaks between cases
* fix(lint): mutable-ok marker on the prefix accumulator (LIT002 type-discipline gate)
* fix(proxy): tie 401 challenges and get_custom_url to the per-request root_path
The per-request root_path middleware sets scope["root_path"] to the
prefix the client actually called, but the OAuth 401 challenges
(raise_user_oauth_challenge / raise_token_exchange_challenge) still
built their resource_metadata from SERVER_ROOT_PATH. On a pod fronting
several prefixes, the challenge advertised a discovery URL under a
different prefix than the discovery document served — the two
disagreed on where the resource metadata lives, and a strict RFC 9728
client refused the challenge. Route the challenges through a small
ContextVar the middleware populates so they read the same effective
root_path Starlette resolves the request under.
The same accessor fixes get_custom_url: when a request lives under a
SERVER_ROOT_PATHS-matched prefix, request.base_url already carries it,
so appending the SERVER_ROOT_PATH scalar on top produced e.g.
/tenant-a/legacy/sso/callback — a path that does not exist. Reading
the per-request prefix instead (and relying on join_paths's tail-dedup)
keeps SSO login/callback URLs under one prefix — the one the request
actually arrived on.
Fallback: outside a request (module-load-time UI URL builders,
background tasks) the ContextVar is unset and the accessor reads
SERVER_ROOT_PATH, matching get_server_root_path() so scalar-only
deployments are byte-identical.
* fix(mcp): challenge URL under per-request prefix must route, and mock parity
Two follow-ups to the review fix that made the 401 challenge use the
per-request root_path:
1. oauth_protected_resource_path must pick the URL structure that
actually routes for the mechanism in use:
- The scalar SERVER_ROOT_PATH deployment registers the well-known
routes with the prefix INSERTED (via well_known_root_suffix at
import time), matching RFC 8414 §3. The challenge URL must use the
same insertion or a client fetching it 404s.
- The per-request SERVER_ROOT_PATHS deployment can't register routes
per prefix; PerRequestRootPathMiddleware strips the prefix from
scope["path"] and the router matches the un-inserted route. The
URL must place the prefix BEFORE .well-known so the strip leaves a
matching path.
The previous fix used the insertion form for both, which 404'd the
discovery fetch on the per-request path — the discovery doc and the
challenge would then disagree on where the resource metadata lives,
the very failure the review flagged. End-to-end verified: the URL
the challenge advertises routes and the doc's `resource` field
equals the URL the client originally called (RFC 9728 §3).
2. get_request_root_path now delegates its fallback through
get_server_root_path() instead of reading the env directly, so every
existing `monkeypatch.setattr("litellm.proxy.utils.get_server_root_path"`
test override keeps working. This unstubbed the mock on the /v2/login
test that failed on the last CI run.
Plus the lint budget: annotate the local accumulator Final, tag the
scope["root_path"] rewrite as an intentional ASGI-contract mutation,
tag the reused `path`/`root_path` rebinds in LazyFeatureMiddleware, and
add reason strings to the two new PLC0415 lazy-import noqas.
* test(mcp): pin the reviewer's expected end-state — challenge URL routes, resource matches called URL
End-to-end regression test that mounts the discoverable router + the
per-request root_path middleware, hits an MCP endpoint that raises
raise_user_oauth_challenge, fetches the resource_metadata URL the
challenge advertises, and checks the returned document's `resource`
equals the URL the client originally called (RFC 9728 §3 exact match).
Covers /tenant-a, /tenant-b, and the unprefixed path on the same app so
a regression on any prefix — challenge URL 404s, or doc emits a
different prefix than the client called — fails at this test rather
than in a strict MCP client's discovery.
---------
Co-authored-by: gym-cmd <186399764+gym-cmd@users.noreply.github.com>
* fix(spend_logs): keep partition DDL transactions alive for their statement timeout
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style: ruff format changed files
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(lint): avoid dict-literal kwargs and keep cast-ok on the cast line
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(lint): cast at the call site instead of widening PrismaClient.tx
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(spend_logs): require partition tx timeout to strictly exceed statement bound
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: jesus <jesus@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(e2e): prove the virtual key lifecycle on every replica
Walks one virtual key through create, read, partial update, clear, enforce
and delete against a live proxy and database, reading every write back on
every gateway replica.
The management suite already had single write-then-read tests for keys, but
none of them proved that a partial /key/update leaves the untouched fields
alone, that an explicit null clears a field, or that a write is visible on
more than the one gateway that took it.
Adds read_back_everywhere to the shared ProxyClient: it polls a GET path on
every URL in PROXY_REPLICA_URLS until each replica's parsed body satisfies
the caller's predicate, and fails naming the replica that never converged.
The CLEAR sentinel in the e2e models makes an explicit JSON null expressible
in a body the transport otherwise strips of None fields.
Documents /key/update's merge patch semantics on the endpoint docstring.
* test(e2e): prove key revocation and field preservation on every replica
Applies the findings from an adversarial review of the first commit.
The delete step only checked that chat was refused on the gateway that took
the write, so it would have passed while a sibling gateway kept serving the
deleted key. It now serves one call from every replica first, so each has the
key cached and the delete has something to revoke everywhere, then polls every
replica for the refusal.
The file also carried its own poll loop that tested the deadline before
attempting, so it gave up one attempt early and skipped the attempt landing
exactly on the deadline. It now shares the harness helper, which is generic
over the polled value rather than over a parsed body, so the same loop covers
both the info read-back and the chat refusal.
The model the enforcement step registers now carries a unique marker in its
alias, matching every other deployment this suite creates, so concurrent runs
never share one model group.
The docstring sentence claimed an explicit null clears any field. It does not:
the metadata-backed fields merge into stored metadata, where a null is a silent
no-op, and only the key's own columns clear. Regenerating the dashboard types
picks up the corrected text.
* fix(e2e): delete a deployment that never becomes servable
Registering a model posts /model/new and then waits for every replica to list
it. When that wait timed out the deployment already existed in the database but
its id had never been returned, so no caller could delete it and the row
outlived the run. It is now deleted before the failure propagates.
Found by review on the key lifecycle suite, whose module fixture registers a
deployment this way, but every caller of the shared helper had the same
exposure.
* docs(e2e): drop the duplicated notes from the lifecycle docstrings
The delete method restated what the warm-up helper already explains, and the
module restated the merge patch rule that the endpoint and the request model
both document.
* test(e2e/ui): cover member role and budget edits, member permission delegation, and team guardrail removal
Three Playwright specs for the Teams flows enterprise customers hit most, each
owning its fixtures and proving the mutation through a read-back rather than a
toast.
- teamMemberEdit: an admin edits a member's team role and per-member budget,
and both survive a reload of the Members table
- memberPermissions: a plain member is refused /key/generate for their team,
a team admin grants it on the Member Permissions tab, and the member then
creates a team key that serves a real completion
- teamGuardrailRemoval: clearing a team's only guardrail on the Settings tab
really clears it, and traffic the guardrail refused starts serving again
* test(e2e/ui): make the new team specs safe to run in parallel
Fixture ids came from Date.now(), so two repeats starting in the same
millisecond minted the same user id: one got a 409 and the loser's teardown
deleted the user the other was still signed in as. Ids now carry a random
suffix.
Also move the member-permissions setup inside the cleanup-protected block so a
half-finished setup cannot leak a team, and close both browser contexts the
test opens.
* test(ui): pin wire contracts for key, model and MCP server forms
Add vitest cases that pin what the key edit, key create, model edit and
MCP server edit forms put on the wire: an edited field reaches the
request with its new value, a cleared field reaches it as an explicit
null, and the dirty-only body is pinned as an expected failure until
each form moves to pickDirty. Model edit also pins the cost-map-derived
model_info fields as an expected failure.
KeyEditView hands a cleared max_budget to KeyInfoView as an empty
string and handleKeyUpdate maps it to null, so the null is pinned at
the /key/update boundary in key_info_view.test.tsx and the KeyEditView
case is an expected failure. buildEditServerPayload passes a cleared
description through as an empty string, so that case is an expected
failure too.
* test(ui): split masked model_info pins and retarget the create tracker
The model_info expected-failure case held three assertions, and it.fails
stops at the first one, so a later revamp that fixed max_input_tokens
while leaving mode leaking would still report an expected failure. Split
it into one case per pinned field group so each flips on its own.
The key create tracker asserted a body of only key_alias, which a create
can never send: key_type, user_id, duration and metadata are always
mounted. Retarget it at the real over-send, which is the Optional
Settings section adding fifteen undefined-valued keys when the user opens
it without filling anything in.
* fix(mcp): apply key and team guardrails to MCP tool calls
Guardrails attached to a virtual key or team were only enforced on LLM
routes. The synthetic request built for MCP tool call guardrail hooks
carried no guardrails in its metadata, so a guardrail with default_on
false never ran on tools/call even when the key explicitly listed it.
Resolve key, team, and project guardrails onto the synthetic request
with the same helper the chat path uses.
* fix(mcp): pass project metadata through without a mutable default
* fix(mcp): mark the request dict parameter mutable-ok with a reason
* test(mcp): explain the premium_user patch and tighten the helper docstring
* ci: limit Rust workflows to Rust directory changes
* ci: run Rust checks when their workflow changes
* ci: report Rust wheels only for successful Rust changes
* ci: keep Rust wheel reports in the workflow summary
* ci: group Rust lint and validation jobs
* ci: keep Rust job names distinct from required lint and test checks
* ci: drop the unused Python setup from the Rust lint job
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
