The benchmarks dashboard answered every question by scanning LiteLLM_SpendLogs at
read time: four aggregate queries per auto-router, two of them window functions
over the response JSONB, re-deriving on every page load which model the previous
turn used, how long a tier had been idle, and how big the prefix was last time.
Those are sequential facts and the request that produces them already knows all
of them, so they are now computed once, when the turn happens.
A new LiteLLM_AutoRouterSession row per (session, auto-router) carries both the
counters and the state that classifies the next turn. fold_turn is pure, so every
rate and dollar formula is unit-testable without a database, and the in-memory
queue plus background flusher follow AdaptiveRouterUpdateQueue: atomic increment
upserts, so two pods writing one session compose instead of overwriting. A pod
that has never seen a session loads its row once and classifies from memory
after, which is what keeps a session correct across a restart or a pod move.
The counters are declared once, on TurnDelta. COUNTER_FIELDS derives from that
declaration and the merge, the flush payload and the read query all build off it,
so a metric added there reaches the database and the dashboard without a second
edit. A test asserts the read query aggregates every declared counter; it caught
two that were being written on every request and read by nothing.
The read path is a single aggregate over pre-folded rows covering every
auto-router at once, and touches no per-request table at all. Rollup rows expire
on the existing spend-log retention cutoff, keyed on last activity so a live
conversation is not pruned out from under itself.
Two behaviour fixes came with the move. The turn buckets are now exhaustive: a
session's opening turn used to land in the headline turn count and in none of the
three buckets, so the bucket totals silently disagreed with the headline. And a
turn with no ephemeral cache-creation evidence now reads as the five minute tier
rather than the one hour tier, which had been the default purely because zero is
not less than zero.
Savings come from compute_savings_spend, the same primitive the usage tab uses,
so the two surfaces cannot report different numbers for the same traffic. The
baseline recorded on each row is the one that priced its turns, so the tab names
what the numbers were computed against rather than whatever the config says by
the time someone opens it.
The auto-router savings driver recomputes what the served request cost, but that
request is not a counterfactual: it ran, and the cost calculator already billed it and
wrote the number down. Recomputing means restating every pricing dimension the biller
applied, and the two this missed were enough to halve it. A request billed at a
priority tier is recomputed at standard rates, and a regional host's uplift is dropped
entirely, so the driver writes a savings figure into the same rollup row as the `spend`
it disagrees with. On `gpt-5.4-mini` at priority the row is billed 0.024 and the driver
prices the same usage at 0.012.
Neither omission cancels between the two arms, because both are per-model. The uplift
is a multiplier read off each model's own entry, so 1.1*A - 1.1*B is 1.1*(A-B) and a
model without one does not move at all. Tier coverage is sparser and asymmetric:
`gpt-5.6` has priority rates and `gpt-5.4-nano` has none.
`cost_breakdown` already carries the answer and already reaches the call site. The cost
calculator records it, it rides the standard logging payload into the spend log's
metadata, and OTEL, the log drawer and the response headers all read it rather than
re-deriving; this driver was the only downstream consumer in the tree still pricing a
completed request from its tokens. `input_cost` and `output_cost` sum to exactly what
the pricer returns, so the served arm reads them. Tool spend, discount and margin stay
out, since the counterfactual cannot be priced with them and charging them to one arm
alone would read as the router losing money on every tool call.
The baseline never ran, so it is still priced through the cost engine, now on the basis
the biller used. `CostBreakdown` carries that basis because it cannot be recovered
afterwards: the tier the biller used comes from `optional_params`, which no log record
keeps, and the served tier that does survive on the usage object is a different fact
with the opposite precedence. Rows written before this shipped carry no basis and price
at standard rates, exactly as they do today; there is no backfill.
Two smaller things in the same path. The router is passed as a provider rather than a
router, so a spend write that was never auto-routed no longer fetches and discards one,
and the complexity router resolves its messages once per hook instead of once per
consumer.
* feat(spend): add net auto-router savings to the cost-optimization dashboard
The dashboard credited compression and prompt caching but said nothing about the
optimization that picks the model, so the driver with the largest lever on a bill
was the one an operator could not see.
Savings are the counterfactual: without a router a deployment runs one model, and
it has to be one that can carry the hardest request, so the baseline is the
priciest model in the router's hardest configured tier. A cheap tier is a choice
the router made, not a ceiling it was bounded by. `auto_router_savings_baseline_model`
overrides it for operators who would genuinely have run something else. Both are
provider-qualified before pricing, because a bare name can resolve to a different
vendor's rates or to nothing at all, and a deployment is priced by its `base_model`
where it has one, which is how Azure deployments are priced everywhere else.
Both arms price the request's real usage through `generic_cost_per_token` rather
than re-deriving per-token arithmetic, so tiered rates, ephemeral cache-write tiers
and regional uplifts stay consistent with what was actually billed. `prompt_tokens`
already includes the cache buckets, so charging them again at the input rate would
price the same tokens twice.
Cache state is what makes this hard. The baseline serves every turn, so whether it
had the prompt cached is whether the conversation was already underway. On a
continuing conversation it wrote the prompt earlier and would only read it now, so
this request's write is what switching cost and counts against the saving. On a
first turn nothing was cached for any model, the baseline would have written the
same prompt, and both arms carry the write at their own rates. Charging the write
to both cases understates a first turn to a few percent of its value, and because
the write premium is fixed by prompt size while the saving grows with completion
length, it can render a profitable route as a loss.
That shape is read off the conversation rather than remembered: a second human ask
means an earlier turn was served. No cache, no session id, and no dependence on a
caller sending a session header. It cannot see a switch on a turn the router did
not classify, and it reads a few-shot prompt's synthetic turns as prior
conversation; both err toward charging the write, which under-claims.
The baseline and the shape ride on the existing `routing_decision` record, which is
already carried from the router to the spend log, already classified for redaction,
and already written-or-cleared per attempt. A fallback that re-enters the hook
therefore cannot leave either fact behind to be attributed to a deployment that
never routed, and no new metadata key crosses the trust boundary.
The result is signed. Whether a switch pays off is a race between the rate gap and
the cache-write cost, and a narrow gap loses; flooring at zero would hide exactly
the routing behaviour an operator needs to see. The donut plots only drivers that
saved, while the card and range total keep the sign.
Savings accrue into a new `autorouter_savings_spend` column on the six daily rollup
tables, declared `NotRequired` because rows queued by a pod on the previous release
carry no such key. It is summed by the rollup merge the cross-pod Redis drain also
runs, and carried through the aggregation query, the per-row accumulation and the
response model, so the dashboard reads a value the API actually sends. Tests
enumerate the drivers from the response model itself and assert each is summed,
accumulated, carried and totalled, so one added later cannot be half-wired.
* fix(spend): let the baseline pay for a continuing turn's own growth
`_baseline_usage` moved every cache-creation token into the baseline's read bucket
whenever the conversation was underway. That is right for a switch, where the
baseline never left the model it was on and really would only read, but wrong for a
turn that stayed put: the prompt grew, and the tokens written are that growth. They
are new to every model, so the baseline would have paid to write them too. Forgiving
it that write made the counterfactual cheaper than it was and shrank the reported
saving on ordinary steady-state traffic, by about 2% per turn.
The selected arm was never involved; it has always been priced on the real usage.
The error sat entirely on the baseline.
The condition is that the request read more than it wrote, not that it read anything.
A switch onto a model already holding a small prefix of this prompt still writes most
of it, and that write is the switch's own cost; keying off a nonzero read would have
handed such a request the full rate gap, turning +$0.0056 into +$0.1177. Comparing
the two buckets separates a warm continuation, which reads far more than it writes,
from a cold arrival, which does the reverse, and it leaves the existing invariant
intact: a request reading 0 and one reading 1 both still land in the same place.
* fix(spend): price each arm under the key litellm billed it, and see agent turns
Two ways the savings number read the wrong thing, both from identifying a model by
its name when the name is not what it costs.
The counterfactual was ranked and priced on the public rate for the model a
deployment names. A deployment may not be charged that rate: the router registers
its configured prices under the deployment's own id and deliberately keeps them off
the shared model-name key so deployments sharing a backend model do not pollute each
other. So a hardest-tier deployment configured above its public rate lost the
ranking to a cheaper candidate, and once chosen was priced at a rate nobody pays.
Which key prices a deployment is now `_select_model_name_for_cost_calc`'s decision,
the resolver the real request is billed through, rather than a second rule here that
would have to re-learn that per-second and tiered overrides count, that a partial
override still counts, and that a deployment configured at zero is priced at zero
rather than treated as unpriced.
The arm being subtracted had the same fault and a sharper edge. It priced the spend
log's `model`, which on Azure is the deployment name, absent from the cost map, so
the whole driver silently read zero for that traffic. It no longer re-derives
anything: `model_map_information.model_map_key` is what litellm actually billed the
request under, recorded at request time by that same resolver with `base_model` and
custom pricing already applied.
Separately, the conversation-shape discriminator counted human asks, and an agent
loop can run twenty turns on one of them. Its tool traffic rides `tool_result`
blocks on user turns that flatten to empty text, and `tool` roles that are never
read, so a long agentic conversation looked like its own first turn and was handed
the arithmetic that leaves the cache write on both arms. That is the one direction
this must never fail in, because it inflates. An assistant turn is the direct
evidence that something answered earlier, and it is blind to how the tool plumbing
is spelled on either surface.
* fix(spend): give the cost-key resolver both inputs the selected arm needs
The served model was resolved through one input at a time, and each choice broke the
half the other fixed.
`model_map_key` is the served model already resolved through `base_model`, which is
the only way an Azure deployment name reaches the cost map at all; without it the
selected arm priced a name absent from the map, returned nothing, and the whole
driver silently read zero for that traffic. But it is built without
`router_model_id`, so it never carries a deployment's own price overrides, and a
custom-priced deployment was compared at its public rate while the baseline used the
real override. On a deployment configured well above its public rate that inverted
the answer outright: a route that lost $21.88 reported saving $0.10.
`_select_model_name_for_cost_calc` takes both, so it gets both. Which key prices a
deployment stays its decision rather than a rule restated here.
* fix(spend): same model is only the same cost when it is the same deployment
The short-circuit compared resolved model identity, so two deployments of one model
collapsed to "no switch" and reported zero. They are not the same cost: a deployment
can carry a negotiated rate, and routing from the dear one to the list-price one is a
real saving the dashboard reported as $0.00 against a true $21.93.
Both arms now carry the key litellm prices them under, so the comparison is between
deployments rather than between names.
* refactor(spend): price from resolved rates, not from a name we keep re-resolving
Four review rounds landed on one mechanism: which identifier prices a deployment.
base_model, then the deployment id, then cache-only overrides. Each round added a
clause to a resolution rule that should not exist, and a wrong primitive fails once
per input shape, so each shape arrived as its own finding.
`Router.get_deployment_model_info` already owns this. It merges a deployment's
configured prices over the built-in map, folds in `base_model` defaults for
deployments whose name is not a model, and falls back to the model name when nothing
is overridden. Every shape hand-rolled here (cache-only, partial, per-second, Azure)
was that function re-implemented badly.
`generic_cost_per_token` now accepts already-resolved rates instead of demanding a
name it looks up itself, which is what forced the name-bending in the first place.
Both arms resolve through the owner and pass what they got: the counterfactual by the
deployment the router would have used, the served request by the deployment that
served it. The invented cost-key resolver is gone, and `Baseline` carries a
deployment id rather than a key we chose on litellm's behalf.
Net 64 insertions against 79 deletions.
* test(spend): follow _most_expensive onto the router that prices its candidates
Ranking moved through `Router.get_deployment_model_info`, since what a deployment
costs is the router's answer to give; these four cases were still calling the old
free-function signature.
* fix(spend): rank baseline candidates by what a request costs, not by two rates
"Most expensive" was decided by comparing output rate then input rate. That is a
property of a rate, not of a request: a deployment dearer per output token can be
cheaper per cached token, so the comparison ordered cache-heavy traffic backwards and
recorded the wrong counterfactual.
Candidates are now costed on one reference request through the same engine the
savings themselves use, which leaves cache read and write rates, tiered tables and
every other billing dimension to that engine rather than to another rule restated
here. The reference request is cache-heavy because auto-routed traffic is.
* fix(spend): pick the baseline against the request that ran, not a stand-in for one
Ranking happened in the pre-routing hook, where the request has not executed yet, so
candidates were costed against a hard-coded reference workload: 20k prompt, 19k of it
cached, 1k out. Which candidate is dearest depends on that mix, so a pooled hardest
tier holding a deployment with non-proportional configured rates could be ranked for
a request nothing like the one served.
The mix is known on the spend path, so the ranking belongs there. The routing
decision now carries the tier's candidates rather than a winner already chosen, and
the baseline is resolved against the usage that actually happened. The reference
workload is gone; nothing here assumes a traffic shape any more.
The router is passed in rather than imported from `proxy_server` inside the
computation, so the savings stay a pure function of their arguments and the caller
owns where the router comes from. That also makes the spend path testable without a
running proxy, which the previous shape was not.
* refactor(spend): measure savings against one configured model, not a derived one
The counterfactual was derived per request: enumerate the hardest tier's
deployments, resolve each one's effective pricing, price them all, take the dearest.
That machinery produced a review finding per input shape it had not anticipated,
and every answer it gave was one an operator could have stated in a line of config.
So they state it. `litellm_settings.autorouter_savings_baseline_model` names the
model the traffic would have run on without a router, for every auto-router on the
proxy, and unset means the driver is off rather than a model nobody named being
guessed at. `savings_baseline.py` and its tests are deleted outright, along with the
tier enumeration, the candidate list on the routing decision, and the per-deployment
override that shadowed it.
Cache-state handling is untouched: the baseline is still priced on this request's own
read and write split, so a switch still pays for re-warming the cache and a first
turn still charges the write to both arms.
45 insertions against 482 deletions.
* refactor(router): compute the conversation shape once and pass it down
`_classify_and_route` re-derived it from the messages the hook had already resolved,
so an ordinary routed request walked the turn list twice for one boolean. The hook
computes it and hands it over, which is also where the affinity-hit path already got
it from.
Also moves `_get_llm_router` below the imports it sat among.
* fix(router): drop the dead conversation_continuing parameter off the hook
It was added to `async_pre_routing_hook` by mistake and immediately overwritten by
the value the hook computes, so it never did anything. It also widened a signature
every pre-routing strategy shares with the protocol in `types/router.py`, leaving
this one router diverged from `AutoRouter` and the interface for no reason.
Also records why an unreadable request counts as continuing: no messages is no
evidence a turn was served, so it pays the cache write and under-claims rather than
being handed a first turn's larger saving on nothing.
* fix(spend): charge a baseline its input rate for cache buckets it cannot price
A model with no cache_creation_input_token_cost, which is every OpenAI, Azure and Gemini entry, resolved that rate to 0.0 and carried the whole written prompt for free, so a first turn routed onto a cheaper model reported a loss. Same hole on cache reads. Those tokens are plain input on such a model, so they move into the text bucket.
* refactor(spend): build the daily upsert payloads in one shot
`common_data` and `update_data` were constructed and then appended to: `request_id`
conditionally for tag rows, `endpoint` unconditionally a few lines later. A dict that
grows after its literal cannot be reasoned about by reading the literal, which is the
whole point of building it at once.
The conditional key resolves to a spreadable value before either payload, so both are
single expressions and the tag branch appears once instead of twice.
Not wrapped in MappingProxyType, though it was suggested: these go straight to
prisma, whose query builder branches on `isinstance(value, dict)` to tell a nested
node from a scalar. A mappingproxy is a Mapping but not a dict, so it falls through
to the serializer and raises `TypeError: Type <class 'mappingproxy'> not
serializable` inside the batch upsert, where the surrounding except would log it and
leave the rollups silently unwritten.
* fix(spend): keep the one-shot upsert payloads under the type-discipline budget
Building both payloads as single literals traded a mutation for two dict literals,
and LIT002 counts construction rather than mutation, so the change the review asked
for is the one the gate charges for.
The empty branch is the avoidable half: it is the same value every time, so it moves
to a module constant built once instead of a literal per transaction, and it is a
read-only mapping so none of the call sites that spread it can fill it in later.
"Add keyword rule" seeds a row with no keywords, and the only check that a
rule carried one lived inside getSemanticConfigError, which returns early
when semantic keyword matching is off. Off is the default, so an unfilled
row fell through to serializeKeywordTierRules and was discarded on the way
to the payload; the create reported success and the rule was gone.
The row now reports the gap itself and the submit is withheld while one is
outstanding, on the create form and the edit modal alike, both reading
emptyKeywordTierRuleIndexes so the row named and the row marked cannot
differ. Enter commits a typed keyword: the dropdown is kept closed, which
left antd nothing for Enter to select, and submitting was what used to
supply the blur that saved the word.
The backend already refused such a rule, but only when the router built the
deployment, so a caller that sent one anyway got the row written, dropped on
reload, and a 500. The management write paths now parse the incoming
complexity_router_config with the router's own ComplexityRouterConfig, judged
on the config alone so a patch that writes one without naming a model is
covered too, and reject it with a 400 having persisted nothing.
* feat(team): custom metadata validation hook for team create and update
Operators can point general_settings.custom_team_metadata_validate at an
async Python function that validates team metadata before /team/new,
POST /team/update, and PATCH /team/{team_id} commit their writes. The
hook receives the metadata that will actually be written (the merged
result on PATCH) plus the stored metadata and requester context, and
fails closed: a rejected value returns the function's own message as a
400 while any exception or timeout blocks the write with a configurable
generic message as a 503. Premium-gated like enforced_params.
* fix(team): validate metadata before model alias writes and strip system keys from validator input
Review follow-ups on the team metadata validation hook: run the validator
before the model_aliases table insert so a rejected create leaves no
orphaned model rows, strip system-managed keys from existing_metadata so
the validator sees symmetric input on both fields, and accept class
instances exposing an async __call__ as validators. Adds a three-way
validator implementation matrix (allowlist function, HTTP-service-backed
function, immutability-enforcing class instance) driven through the real
create, update, and patch endpoints, including an HTTP stub service and
outage coverage.
* test(team): run the metadata validation matrix against the DB-backed proxy in CI
Adds the validator matrix to the proxy_store_model_in_db_tests CircleCI
job so every scenario runs full e2e against a Postgres-backed proxy. The
proxy config registers a dispatching validator that routes each request
to one of the three implementations via a metadata key and accepts
anything that does not opt in, keeping the rest of the suite unaffected.
CI starts a stand-in cost center service on the host for the HTTP-backed
implementation, reached from the container via host.docker.internal, and
the outage path targets a closed port to prove the fail-closed 503
without stopping services.
* feat(ui): edit team metadata as key-value pairs in team create and edit forms
The team create and edit forms asked for metadata as a raw JSON blob in a
textarea buried under Additional Settings. Both forms now render a key-value
pair editor directly under the TPM/RPM limit fields, backed by a shared
MetadataKeyValueFields component. Values round-trip losslessly: non-string
values display as JSON and parse back to their typed form on save, and
JSON-ambiguous strings are quoted so their type survives the trip. The edit
form hides UI-managed keys (logging, guardrails, model rate limits, etc.)
that dedicated controls already own and re-add on save.
* fix(ui): explain typed JSON parsing in the team metadata help text
* feat(team): schema-driven metadata fields from team_metadata_schema config
* refactor(team): render schema metadata fields as locked key-value rows, drop allowed_values
* refactor(team): schema fields reduce to key and label, tag-rendered keys, clean rejection toasts
* refactor(ui): prepopulate declared metadata keys as ordinary key-value rows
* fix(team): let non-admin dashboard users read the team metadata schema
* test(proxy): pin timeout wiring, boundary, and error-message contracts for team metadata validation
* fix(proxy): use pooled async httpx client in the e2e team metadata validator example
* refactor(team): satisfy staging lint ratchets inherited by the merge
* fix(proxy): redact credential headers from request logging copies
clean_headers preserves an Anthropic subscription OAuth token, and other
client-supplied provider credentials, so they can be forwarded upstream. The
same dict was also stored as proxy_server_request["headers"] and
metadata["headers"], so those credentials reached every logging callback and
the SpendLogs proxy_server_request column that the Admin UI logs page renders.
Build the observability facing copies through redact_credential_headers, and
drop the transport-only keys (provider_specific_header, headers, api_key) from
the request body snapshot since they have to keep the real values.
* fix(proxy): use the redacted header copy in the request debug log
The stdout secret filter matches Bearer and sk- shaped values, so an MCP auth
token printed by the request-header debug line survived it in cleartext.
* fix(proxy): resolve the configured MCP auth header name through the secret manager
get_secret_str also consults a configured secret manager, so a deployment that
stores the header name there now gets that header masked too. Drops the added
comments in favour of a named constant.
* perf(proxy): resolve the MCP auth header name once per process
get_secret_str issues a blocking secret-manager SDK call when one is configured,
and configured_credential_header_names runs on every proxied request.
* fix(proxy): read the MCP auth header name live, cache only the secret manager
The config reloader rewrites os.environ on an interval and after /config/update,
and MCPRequestHandler resolves the same setting per request, so caching the env
lookup left a renamed header logged in the clear until the process restarted.
Only the blocking secret-manager call stays cached.
* refactor(proxy): narrow header redaction to the reported credential set
Drops the MCP header-name resolution, its per-request config and secret-manager
lookups, and the x-mcp- prefix rule. Those cover a separate credential family
than the one this ticket reports and carried their own config-reload staleness
surface; they belong in their own change.
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Team-scoped DD credentials (dd_api_key, dd_site) set via POST /team/{id}/callback were silently dropped because _request_blocked_callback_params blocks them from standard_callback_dynamic_params. The security block is correct for request-level injection, but team callback_vars are admin-configured and trusted.
Store the raw init kwargs on the Logging instance and read dd_* params from there in _process_dynamic_callback_list instead of from standard_callback_dynamic_params.
Adds an integration test that exercises the full Logging.__init__ flow with team callback_vars to prevent regression.
Co-authored-by: Aanchal Khandelwal <aan2210khandelwal@gmail.com>
Key and team `router_settings.model_group_alias` was accepted, persisted and
echoed back by `/key/info`, but never applied at request time, so the request
ran on the group the caller asked for. `route_request` forwards only the
settings the Router accepts as per-request kwargs, and `model_group_alias` is
not one of them: the Router resolves aliases from its own instance attribute,
which holds the global config map and is shared across requests.
Resolve the alias in the proxy instead, alongside the existing model-alias
rewrites and ahead of the pre-call hooks, so per-model limits and guardrails
key off the group that actually serves the request. Authorize the alias target
before the rewrite; model access was checked against the requested group, so a
key whose alias points at a group it cannot call gets the usual 403 rather than
being quietly served it.
Resolves LIT-4879
* feat(teams): apply default organization to new teams from default team settings
Adds organization_id to DefaultTeamSSOParams so proxy admins can pick a
default organization in Default Team Settings. new_team applies it before
org validation whenever a team is created without an explicit
organization_id, so API, Admin UI, SCIM, SSO, and team upsert creations
all inherit it and go through the same existence and org-limit checks.
Explicit organization selections win and existing teams are untouched.
The default is validated at save time (PATCH /update/default_team_settings
returns 400 for an unknown org) and at create time, where a missing org now
surfaces as a clean 400 instead of a 500 by routing OrganizationNotFoundError
into the previously dead org_table None guard.
The Admin UI Default Team Settings tab gets a Default Organization row
backed by the shared OrganizationDropdown.
* fix(teams): validate org limits against final team state including defaults
Applies default_team_params and the legacy max_budget fallback before the
organization validation block, so _check_org_team_limits sees the values the
team will actually be persisted with. Also loads the org's budget table in
the lookup; without include_budget_table every budget comparison in
_check_org_team_limits was skipped because litellm_budget_table was None.
* test(proxy_behavior): pin org team limits as enforced on /team/new
The dead-code pins existed to turn red when include_budget_table went
live; that happened, so the scenarios now assert the 400 rejections plus
within-cap acceptance, and the unknown-org pin asserts the handler's 400
instead of the surfaced 500.
* fix(proxy): backfill null user_email on existing users during JWT auth
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): guard mapped-key email backfill and make null update atomic
Resolve Greptile review on the JWT user_email backfill:
- only backfill when the mapped virtual-key owner is the JWT principal, so a
mismatched admin-created mapping cannot write one user's email onto another
- make the best-effort mapped-key enrichment non-fatal so a database outage on
a cached-key request no longer fails otherwise-valid authentication
- persist the backfill with an atomic null-guarded update_many so concurrent
writers cannot overwrite an already-populated email
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): keep cache coherent when a concurrent backfill wins the null-email update
* fix(proxy): cache DB-persisted email after JWT backfill, not the proposed value
Resolve the Greptile finding that a successful null-guarded backfill could
cache this request's proposed email even if a concurrent ordinary user update
wrote a different email first. The helper now always re-reads the row after the
atomic update and refreshes the cache from the value the database holds, so
cache-hit auth and attribution stay consistent with the persisted record.
Annotate the Prisma and model_copy dict literals to keep the LIT002 budget within its ceiling.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: shivam <shivam@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: ryan-crabbe-berri <ryan@berri.ai>
disable_team_logging cleared only metadata["callback_settings"], but callbacks
registered through POST /team/{team_id}/callback and the Admin UI live in
metadata["logging"], and request-time resolution stops at that slot without
ever reading callback_settings. The endpoint reported success while the team
kept sending request and response data to its third-party destination.
Empty the logging slot alongside the existing callback_settings reset, and
refresh the cached team object so the change applies to keys that are already
in flight rather than at the next cache expiry. The same refresh is added to
add_team_callbacks, which has the symmetric problem of a newly registered
callback staying dormant until the entry expires.
Resolves LIT-5101
* fix(team-callbacks): report API-registered callbacks from GET /team/{team_id}/callback
POST /team/{team_id}/callback writes metadata["logging"] while the GET read
metadata["callback_settings"], so every team configured through the API or the
Admin UI got back an empty list. c620d76fe4 migrated the writer to the new key
and left this reader on the old one.
Resolve the read the same way request-time resolution does in
_get_dynamic_logging_metadata: a logging slot that is present wins outright and
callback_settings stays as the deprecated fallback, so the endpoint reports what
a request would really do rather than the union of both shapes. An empty logging
list therefore reports no callbacks, matching a request that fires none.
Decrypt callback_vars for the response and mask the credential keys. Ciphertext
would be unusable to the caller, and a value encrypted under a key that is no
longer classified as sensitive would otherwise come back as a raw blob.
Resolves LIT-5093
* Update litellm/proxy/management_endpoints/team_callback_endpoints.py
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(team-callbacks): mask callback vars that fail to decrypt
decrypt_callback_vars passes a value through untouched when it cannot be
decrypted, which happens to existing rows after a salt-key rotation. Under a
key that is not classified as sensitive that blob reached the caller as opaque
ciphertext it could not use or tell apart from a real value, so mask anything
still carrying the encrypted prefix.
Raised by Greptile on the first commit.
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
A keyless internal user signing in to the Admin UI was redirected off the
post-login landing to /ui/connect, which renders nothing but the MCP apps panel,
so a plain gateway sign-in ended on an MCP OAuth surface the user never asked
for. The landing now renders the keys dashboard for every role. The key lookup
that existed only to make that routing decision goes with it, along with the
useKeys enabled flag it was the sole caller of and the role-hydration hold that
guarded its one-frame dashboard flash
The gateway DCR consent flow moves the other way. Its /authorize handed the
browser to /ui/chat/integrations, whose layout hard-blocks when enable_chat_ui
is off, which is the default, and client-side redirects to /ui/ without the
query string; that destroys the connect_flow handle and strands the MCP client
until the 600s flow cookie expires. It now lands on /ui/connect, which reads
connect_flow and connect_client, mounts the consent banner and puts the apps
panel in connect mode. /ui/chat/integrations keeps its connect-mode handling
this release so flows sealed before the deploy still finish
Resolves LIT-5104
Resolves LIT-4911
About 35,000 fixes ruff marks safe across 32 rules (UP006/UP045/UP007
modern annotations, UP032 f-strings, SIM114/SIM118, RET501, and
friends), removal of the 1,296 typing imports the rewrite orphaned, and
hand fixes for what the fixers could not see: five star-import
freeloaders of typing names, two F823 late-import annotations, the
/get/config/list introspection crash on types.UnionType, redundant
function-local RoleMappings imports in ui_sso.py that shadowed the
module-level name once the annotation lost its quotes, and one FURB168
tautology.
B009/B010/PIE804/RUF019 are excluded on purpose: their safe fixes
rewrite getattr/setattr/**-splat/key-in-dict escape hatches into forms
basedpyright then rejects (283 new errors measured), so their budgets
stay at base values.
ruff-strict-budget.json drops by 39,579 this commit (39,968 across the
branch) with 28 rules at an actual 0 and 9 more sharply down.
type-discipline-budget.json ratchets LIT002/LIT006/LIT009 down; LIT001
moves to the now-honest total: the checker matches the spelling `set`
but not the alias `Set`, so the 160 typing.Set annotations rewritten to
set[...] were always mutable-set annotations and only now count.
Gating the mock testing request params behind
general_settings.dangerously_allow_mock_testing_request_params (#35423) turned
every fallback, retry and timeout drill in tests/test_fallbacks.py into a 400:
the build_and_test job mounts proxy_server_config.yaml, which never opted in.
Opt that config in. It is the config the CI proxy runs with, and the suite it
serves exists to drive synthetic failures.
Add a unit test that ties the two together: it scans the top-level tests/test_*.py
files build_and_test globs for gated param names and fails if the config they run
against has not opted in, so the next change to either side is caught in a fast
lint-tier job rather than a Docker E2E.
A daily-spend batch upsert that outlives prisma-client-py's 30s HTTP read
timeout keeps running server side after the client gives up, holding its
row locks for as long as the database takes. Every later flush cycle
queues behind those locks, which is how one slow batch cascaded into
exhausted database sessions.
The query engine's own transaction timeout cannot end that wait: it
cannot interrupt a statement that is already executing. Measured against
real Postgres, a batch wrapped in db.tx(timeout=60s) still held its locks
for the full 90s the statement ran. Only a Postgres-side statement_timeout
bounded it.
database_statement_timeout and database_lock_timeout (seconds) are now
first-class general_settings keys, emitted as libpq
options=-c statement_timeout=<ms> on DATABASE_URL. They are opt-in, so an
unset config keeps today's behavior, and they are never applied to
DIRECT_URL, which serves migrations that legitimately run long.
Resolves LIT-4718
Co-authored-by: Yassin Kortam <yassin.kortam@gmail.com>
The Prisma query engine is a separate Rust process whose resident memory is
a high-water mark: it grows with the payload of the largest single statement
it executes and glibc never returns that memory to the OS, so a pod's memory
floor ratchets up to its worst-ever write and stays there for the life of
the worker. Memory-based autoscaling then reads a number that reflects the
largest write the pod has ever done rather than what it is doing now.
The spend-log flush handed Prisma a fixed 1000 rows per create_many. With
store_prompts_in_spend_logs enabled a single row carries the full prompt and
response, so one statement can be tens of megabytes and permanently costs
hundreds of megabytes of RSS. Row counts cannot express that budget: the
same 1000 rows range from well under a megabyte to tens of megabytes.
Split each flush into statements bounded by encoded payload size
(SPEND_LOG_WRITE_BATCH_MAX_BYTES, default 2MB) on top of the existing
1000-row cap. What is measured is the encoded statement, so the budget
counts what actually goes on the wire: the JSON escaping of quotes and
newlines, multibyte characters at their encoded width, the field names and
separators a 25-column row carries, and the brackets and row separators the
rows carry as one collection. Deployments that do not store prompts keep one
statement per 1000 rows and are unaffected; prompt-carrying flushes get
several small statements instead of one huge one. A row larger than the
budget is still written on its own rather than dropped, and a row the
serializer refuses counts as zero rather than raising out of the flush and
dropping every row queued behind it.
Splitting a flush must not multiply what a poison-row flood costs, so the
poison-isolation allowance is threaded through every statement of a 1000-row
group instead of being handed out fresh per statement. That is only safe
because the allowance now counts failed inserts rather than every insert:
the one insert a statement needs when nothing is poisoned is not charged, so
a healthy flush never runs the allowance down however many statements it
splits into, and a statement reached after the allowance is spent is still
attempted so clean rows behind a flood still persist. Failed inserts for a
group are bounded by the allowance plus one baseline insert per statement,
which restores the constant-per-group ceiling the single-statement path had.
Resolves LIT-4765
The public A2A guide tells users to declare agents under a top-level
`agents:` key, but the proxy only ever read `agent_list:`, so the
documented config was silently ignored and GET /v1/agents returned an
empty list. Accept `agents` as the documented spelling and keep
`agent_list` working for anyone who found it by reading the source.
Selection is by key presence, so an explicitly empty `agents: []` is not
overridden by leftover legacy entries.
Config-defined agents were also dropped on any database-backed gateway:
the periodic reload rebuilt the registry from the DB rows plus a module
global that was declared and never assigned. The registry now remembers
the agents it loaded from config.yaml and replays them on every rebuild.
A database row wins a name collision, mirroring how config-declared MCP
servers are unioned under the database registry, so name lookups and
deregistration keep addressing exactly one agent.
Resolves LIT-4978
Aligns the fix with the constraints in LIT-4800. A zero-increment
counter now blocks at current >= limit, matching RPM's semantics; the
previous current > limit let a pool sitting exactly at its reservation
admit one extra request. reserve_tpm_tokens rebuilds its descriptors
with only tokens_per_unit so the requests dimension stays out of the
reservation pass, which deliberately leaves RPM to the separate
should_rate_limit check.
Keys created with key_type=llm_api get allowed_routes=["llm_api_routes"],
which covered /v1/models but not /v1/model/info, so a client could list model
names but not read pricing, mode, or max_tokens without a second key.
Adds both /model/info and /v1/model/info (same handler) to llm_api_routes only.
Membership there is not the same as RouteChecks.is_llm_api_route(), which is
what gates DISABLE_LLM_API_ENDPOINTS, global/virtual-key budget enforcement,
enforce_user_param and JWT team attachment; /guardrails/apply_guardrail already
sits in the group the same way. /v2/model/info stays out: it is the paginated
Admin UI listing, not model metadata a caller needs at request time.
public_routes moves from set([...]) to a frozenset literal to keep the LIT002
and ruff-strict ceilings from rising; both budgets ratchet down by one.
add_deployment already reapplies DB router settings through _update_llm_router,
so gating router_settings out of the pub/sub publish set left the push path
covering less than the resync actually applies
Resolve the requested member user_ids with a single find_many instead of one
lookup per member, so a large member list no longer turns into that many
round-trips before the permission check runs. Write the member-add audit
entries concurrently rather than one after another, and list at most a few
ids in the rejection message instead of echoing the whole request back.
Update the team-admin member-add case that covered adding a user_id with no
user row, which the endpoint now leaves to proxy admins.
Caps fleet-wide reload rate at one resync per 10s per pod so a burst of
authenticated writes cannot amplify into continuous cross-pod reloads, and
skips publishing config params (environment_variables, router_settings) that
no resync callback applies outside proxy startup
After any management write to a DB-backed config table, publish an
invalidation event on the coordination Redis; every pod runs a
subscriber that debounces, jitters, and triggers an immediate
add_deployment plus get_credentials resync. The interval polls stay
as slow reconciliation fallback and behavior without Redis is
unchanged since publish and subscribe both no-op.
Adding a team member by a user_id with no user row created that row as a
side effect for any caller permitted to add members, while creating users
directly is restricted to proxy admins. Restrict that path to proxy admins
too; adding an existing user, and inviting a new one by user_email (where
the user_id is allocated server-side), are unchanged.
Also record the membership change, and any user row it creates, in the
audit log, matching /team/update, /user/new and /key/*.
* fix(ui): show pass through route selections in team/key forms and match team id substrings in team search
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* perf(teams): keep team id search index-friendly with a prefix match
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(teams): keep /v2/team/list search id matching exact by default and add an opt-in prefix mode
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: milan <milan@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): stop model writes 500ing on another pod's delete
A model write judges the reload it triggers by diffing this pod's router before and
after, and reports anything that stopped serving as damage. On a pod that has not yet
polled a delete another pod made, the snapshot still lists that model; the reload then
evicts it because the db no longer has it, and the guard reads its own correct
reconcile as degradation. The row is written and served, but the caller gets a 500.
Since propagation between pods is a 30s db poll, any delete followed by a create
inside that window can land on a pod that has not caught up, so a delete-then-create
pair returns 500 whenever the two requests hit different pods.
_delete_deployment already computes exactly the set that settles it: the ids the db
and config still want. Thread it up through _update_llm_router, add_deployment and
clear_cache to the verdict, and intersect the drop set with it so an id the db no
longer has stops counting as collateral. Where no reconcile ran the set is None and
every drop is still reported, so a genuinely broken reload is caught as before.
_delete_deployment now returns that set instead of a delete count; the count had no
callers in the proxy, and the tests asserting it already assert the eviction calls.
* test(proxy): fold reload-verdict test commentary into docstrings and assertions
Greptile flagged the inline comments against the repo's no-new-comments rule. The
case-by-case context moves into the test docstring, and the two return-contract
assertions carry their reasoning as failure messages instead.
* test: fix clear_cache mock return type in model block/unblock tests
Review feedback: the relaxed predicate admitted negative increments,
which both atomic backends would apply as decrements. Restrict the new
behavior to zero-valued pure checks and assert negatives neither check
nor mutate counters.
The atomic check-and-increment path skipped any counter whose increment
was <= 0. The dynamic rate limiter always passes a zero token increment
pre-call because usage lands on the counters post-response, so on a model
configured with only tpm the limiter evaluated no counters at all: no
model-wide TPM cap and no priority reservation, in either generous or
strict mode. Regressed in dd57ae6691 when the pre-call flow moved off the
read-only should_rate_limit check, which did evaluate token limits.
Keep zero-increment counters in the payload so they act as a pure check
(current + 0 > limit), matching the pre-regression semantics in both the
Lua and in-memory paths. Adds unit regressions at the primitive and hook
level plus a live e2e covering the priority_generous/priority_strict
registry rows.
Handling of the client-supplied mock testing params was split across three
places with different behavior for each. Three were dropped from every proxy
request, two reached the router untouched, and a request that asked for a
synthetic failure came back as an ordinary success with nothing to indicate
that no failure had been injected
Put all six behind one opt-in, general_settings.
dangerously_allow_mock_testing_request_params, and reject rather than drop
when it is unset, so a fallback drill cannot report a pass for a test that
never ran. The rejection names the params it saw and the config key to set,
which is also the answer for anyone following the older docs
The flag is config-file only. It is deliberately absent from
ConfigGeneralSettings, and that absence is what makes /config/update drop it
on parse and /config/field/update reject it; the tests pin both so the field
cannot be added back for tidiness without the reason surfacing. Enabling it
logs a startup warning naming every param it unlocks
BREAKING CHANGE: mock_timeout and mock_testing_rate_limit_error now require
general_settings.dangerously_allow_mock_testing_request_params to be set in
config.yaml. Previously they were accepted unconditionally
Tool-level MCP entitlements are enforced in one place,
check_tool_permission_for_key_team, reached from pre_call_tool_check. Two
dispatch paths reached a tool handler without passing through it.
execute_mcp_tool's legacy fallback dispatched into the local tool registry
after retrying the unprefixed name, with no allowed/banned-tool check, no
key/team/org tool permissions and no parameter validation. It now runs the same
gate, and only when something can actually dispatch: when the unprefixed name is
absent from the local registry too, the existing 404 stands rather than becoming
a misleading "server unavailable".
The server the tool-level checks need is available even though the tool name is
not in the tool -> server mapping: a non-empty prefix has already been compared
against the caller's allowed_mcp_servers by exact name, so the named server is
in that list. It is resolved from allowed_mcp_servers rather than from the
manager's registry, because the registry can return a server the caller holds no
grant for, and matching on anything other than name would accept a server the
server-level check never validated. The remaining case is a prefix segment that
is empty, which the server-level check skips entirely because it is gated on a
non-empty server name; that now fails closed with 503 instead of dispatching for
a caller holding no server grant at all.
An entitled caller's legacy call therefore still dispatches, so a configuration
that worked before keeps working; only the unentitled call is refused, now with
the entitlement gate's own 403.
call_tool ran pre_call_tool_check inside `if proxy_logging_obj:`, so an absent
logging object would have skipped authorization silently. This half is defensive
with no live hole: all four call sites source the module-level ProxyLogging
singleton from proxy_server.py, which is never None. The shape was still wrong.
pre_call_tool_check now runs its three authorization checks unconditionally and
only the guardrail hooks, which are dispatched through the logger, depend on one
being present.
A third reported path, where allow_all_keys, BYOM-submitted and
upstream-delegated servers are unioned in after the resolver's ceilings, was
investigated and found not to be a defect. The widening is real, but a server's
tool surface is already boundable for every caller at registration through
MCPServer.allowed_tools / disallowed_tools, enforced by
check_allowed_or_banned_tools ahead of the entitlement check, and per-caller
narrowing plus the org tool ceiling remain available. Nothing here changes that
path.
Resolves LIT-4956
The hook resolves the newly created user through UserRepository and builds the
audit entry from that row. Pin both halves: the entry carries the persisted
row's fields rather than the /user/new response, and a user id that resolves to
nothing produces no entry at all.
* feat(spend-logs): record when a spend log row is the auto-router's own classifier call
The complexity router's classifier sub-call copies the parent request's metadata
verbatim, so its spend log row carries the caller's key, team and user and is
indistinguishable from traffic the caller actually sent. Nothing on the row says
otherwise: call_type is "acompletion" either way, model_group is overwritten to the
classifier's own model group so the row never looks auto-routed, and routing_decision
is absent exactly as it is on an ordinary request.
Record the fact the system already knows at call time. internal_call_origin is
declared on SpendLogsMetadata, which is the allowlist _get_spend_logs_metadata
projects onto, and stamped in _classifier_call_metadata; both classifier paths
already route through that one function and it feeds the metadata and
litellm_metadata buckets alike, so every request surface is covered at one site.
The key is reserved rather than caller-supplied, so it joins routing_decision in the
untrusted-metadata strip and a caller cannot label their own traffic as router
overhead.
The classifier call also inherited no session identity, so the router minted a fresh
trace id and the row landed in a session of its own. Forwarding the parent's session
puts it in the trace of the request that triggered it, which is where an operator
looks for what the routing cost.
* feat(ui): show which log rows are the auto-router's own classifier calls
A classifier row now carries internal_call_origin and shares its parent's session,
so the session trace lists it beside the request that triggered it. Without a marker
in the sidebar it reads as another call the caller made, which is the confusion this
resolves.
The tag renders only for a recognized origin, so ordinary traffic and any future
origin this build does not know about stay unlabelled rather than being asserted as
classifier calls.
* feat(proxy): add a generic list contract for management/v1 entity lists
Paging, sorting, filtering and search for an entity collection, declared once
as a ListSpec and served by handle_list. The route injects a ListExecutor that
owns its table, so this module never imports Prisma.
The caller's scope is derived from the caller alone and ANDed with whatever
they filtered on, so a query parameter can only narrow what they may read.
This is the shared half of the budgets list; it lands here so the endpoint has
something to register against, and drops out when the framework arrives on its
own branch.
* feat(proxy): add GET /management/v1/budgets
The Budgets page reads /budget/list, which returns the whole table as a bare
array with no way to page, sort or filter it. A customer with enough budgets to
fill the page has no way to find one.
Registers LiteLLM_BudgetTable against the management/v1 list contract: sortable
on budget_id, max_budget, tpm_limit, rpm_limit and created_at, default order
newest-first with budget_id breaking ties, search on budget_id, and filters for
budget_duration, max_budget and created_at. budget_duration is deliberately not
sortable; the column holds "7d"/"30d" strings, so a lexicographic ORDER BY puts
"30d" ahead of "7d".
tpm_limit and rpm_limit are BigInt? in Prisma, so rows validate through a
pydantic model on the way out and serialize as JSON numbers.
A caller without admin view is refused 403 as a problem document rather than
served an empty page. /budget/list is untouched.
* fix(proxy): rework the budgets list onto the merged list contract
PR #35308 landed a different shape than this branch was written against: `where`
is a tuple of frozen predicates rather than a Prisma-shaped mapping, `ListSpec`
carries both the row and the wire type, and `where_sql` / `order_by_sql` render
for a raw-SQL executor. The budgets executor now queries through `query_raw` the
way the spend logs facet does, selecting only the columns it serves.
Also casts datetime binds in `where_sql`. They cross into the query engine as
JSON, so an uncast placeholder arrives as text and Postgres refuses
`timestamp >= text` outright; every `filter[created_at][gte|lte]` was answering
500. The cast reads the bind as an instant and drops it to naive UTC to match
Prisma's TIMESTAMP(3) column, the same one /spend/logs/ui applies.
* refactor(proxy): fold the predicate renderer instead of recursing
recursive_detector flags `_render_all`, and the flag is fair: it recursed once
per predicate, so the stack grew with the number of filters on the request for
no reason. Walking a predicate list is a running bind index, which is a fold.
`_render` still re-enters for `AnyOf`, but its clauses are plain comparisons
built by `?q=`, so that nesting is one level deep and no caller can drive it
deeper.
* fix(tool-management): drop unsupported prisma select kwarg from team lookup
POST /v1/tool/policy returned HTTP 500 for every request carrying a team_id,
with "LiteLLM_TeamTableActions.find_unique() got an unexpected keyword argument
'select'". _resolve_team_id_to_object_permission_id looked the team up with
select={"object_permission_id": True}; prisma-client-py 0.11.0 has no select
kwarg on find_unique, so the call raised TypeError and the handler's except
block turned it into a 500. Both the initial read and the fallback read taken
when a concurrent writer wins the update_many race were failing the same way,
so per-team tool blocking was unreachable
The kwarg is dropped rather than replaced; the generated client has no
projection API, and this is a single-row lookup where selecting all columns
costs nothing worth working around
The existing tests missed this because the team table was an AsyncMock, which
accepts any keyword. The new double binds each call against the real
find_unique and update_many signatures, so an unsupported kwarg raises the same
TypeError production does
* refactor(test): tighten typing on the tool policy team table double
Replaces the double's Any annotations and bare list types with concrete ones:
kwargs are object, rows are Sequence[MagicMock] held as a tuple, and the call
logs are list[dict[str, object]]. Behaviour is unchanged; the double still
binds every call against the real generated prisma action signature, verified
by reintroducing the select kwarg and watching both regression tests fail
A bedrock guardrail configured mode: during_mcp_call never ran. ProxyLogging
remapped the event to during_mcp_call and dispatched, but bedrock's own
async_moderation_hook then hard-coded during_call and re-checked, so the second
check rejected the very requests the guardrail was configured for and the tool
call proceeded unscanned with no error.
Remap call_mcp_tool the way model_armor already does, which matches the remap
ProxyLogging.during_call_hook itself performs, and teach the shared
get_guardrails_messages_for_call_type helper that an MCP tool call carries its
payload in the same messages key, without which the hook passes the gate and
then bails on an empty message list.
The ID-JAG egress arm could only assert a caller that presented its own IdP
identity token on the request, so an agent holding a brokered LiteLLM
credential got a 412 and never reached the upstream. The assertion captured at
SSO login was already persisted per user for exactly this purpose, but nothing
read it back.
The arm now falls back to that stored assertion, keyed on the authenticated
principal's user_id. The identity is always taken from the credential the
gateway authenticated, never from a caller-supplied field, so no caller can
select whose identity is asserted upstream. A missing, expired, or
unidentified subject stays a 412; ID-JAG exists to assert a specific user and a
missing subject has no safe substitute. A store outage is the one exception: it
is surfaced as a typed AssertionStoreUnavailable and mapped to 503, so a
database blip cannot 500 the egress or the upstream-401 retry, and does not
tell the user to sign in again over something they cannot fix.
Sourcing a subject from the store rather than the request changed what
invalidation can rely on, so the exchanged-token cache changed with it. The
entry is now addressed by a slot key derived from the principal, plus the
caller's own token when it presented one, with a fingerprint of the subject
token and config stored beside the bearer and compared on every read. A
mismatch reads as a miss and re-mints, so a rotated assertion or an edited
server config cannot be served a bearer authorized under the old inputs, and
two callers cannot receive each other's. Invalidation is a single delete of a
key it can always compute, needing no store lookup on the recovery path.
The upstream-401 invalidate-and-retry path was also gated on a truthy inbound
subject token, which skipped recovery entirely for store-sourced calls. The
gate is now mode-aware: token_exchange still requires an inbound token because
it has nothing else to mint from, id_jag does not.
oauth2_id_jag is also now selectable in the admin dashboard with its own field
set, instead of being reachable only from config.yaml or the REST API. The
auth-type selects drop antd list virtualization: at eleven options the last one
no longer mounts, which is a scroll in a browser but makes the option
unreachable to anything reading the rendered list.
Co-authored-by: Yassin Kortam <yassin.kortam@gmail.com>
Replaces the double's Any annotations and List/Dict aliases with concrete
types, matching the equivalent double in the tool policy tests: kwargs are
object, records are Sequence[Mock] held as a tuple, and the call log is
list[dict[str, object]]. Behaviour is unchanged; the double still binds every
call against the real generated prisma action signature, verified by
reintroducing the select kwarg and watching the regression tests fail
