* fix(auto-router): stop the embedding model's context window from failing long requests
The auto-router embeds the last user message to pick a model and sent it to the
embedding model unbounded. Embedding models carry 512 to 8k token windows while the
chat models they route to carry 200k+, so any prompt over the encoder's window failed
at the routing step with a 400 the destination model would never have raised.
Cut every doc to a character cap inside LiteLLMRouterEncoder, which is the one choke
point the auto-router, complexity-router, semantic guard and MCP tool filter all share.
Default 2000 chars, roughly 500 tokens, which fits even a 512-token self-hosted encoder,
overridable per deployment with auto_router_max_input_chars and globally with
DEFAULT_MAX_EMBEDDING_INPUT_CHARS.
Truncation alone cannot cover provider-side batch and byte limits, so any failure of
the route call now falls back to the auto-router's default model instead of propagating.
That path also fixes two latent bugs: a no-match left the auto-router alias in place as
the model name, which fails downstream with "Unmapped LLM provider" rather than reaching
default_model, and an empty route list raised IndexError.
Fixes#17869Fixes#20277
* fix(auto-router): make the embedding input cap opt-in so guards still see whole prompts
Defaulting the cap inside the shared encoder truncated every consumer, not just the
auto-router. The semantic guard builds the same encoder, so its pre-call check would
have classified only the first 2000 characters while the full message still reached the
model, which a benign opener in front of an injection payload walks straight past. The
MCP tool filter and complexity router were silently narrowed the same way.
The encoder now defaults to sending docs whole and cuts only when a caller passes
max_input_chars. The auto-router is the only caller that does, so guard, MCP filter and
complexity-router behaviour is unchanged from before this branch.
DEFAULT_MAX_EMBEDDING_INPUT_CHARS becomes DEFAULT_AUTO_ROUTER_MAX_INPUT_CHARS, since it
is now specific to the auto-router, and drops its env override: the per-deployment
auto_router_max_input_chars already covers it, and every env var in constants.py has to
be documented, which is what broke the documentation and code-quality checks.
Also drops the added comments and the redundant type: ignore that review flagged.
* test(auto-router): cover the max_input_chars wiring from litellm_params
Nothing asserted that auto_router_max_input_chars on the deployment reaches the
AutoRouter that embeds prompts. Dropping the wiring left every test green while the cap
silently reverted to the default, so an operator with a 512-token embedding model could
not lower it and every long prompt would fall back to the default model instead of
being routed.
* test(auto-router): cover the populated route-choice list branch
The route layer can hand back a list, and picking its first element is where the
IndexError lived: the empty case was covered but the populated one was not, so the
branch that reads route_choice[0].name could be deleted with every test still green.
* refactor(ui): replace hand-rolled query-param routing with nuqs
The dashboard carried five copies of the same pushState-based detail
routing hook plus a shared navigateWithParams helper, each with its own
plumbing test and a copy-pasted reactive useSearchParams mock in
component tests. nuqs provides the same shallow history-API routing
behind useQueryState/useQueryStates, so the key, team and org hooks are
deleted in favor of inline useQueryState at their single consumers,
while the models and logs hooks keep their interfaces but drop their
hand-rolled internals. Component tests now mount NuqsTestingAdapter
(via renderWithProviders or locally) instead of patching window.history,
and URL assertions go through onUrlUpdate spies that can additionally
distinguish push from replace, which the old window.location checks
could not
* test(ui): assert browser back closes the log drawer after in-drawer selection
Greptile flagged that the nuqs port of the switching-logs test stopped
at asserting emitted push and replace modes. The test now replays those
recorded modes against a history stack and performs the back step, so a
regression to push-on-select or broken URL-derived drawer state fails
the test instead of passing silently
Folds every successful auto-routed request into LiteLLM_AutoRouterSession with one
conditional upsert at spend-write time, classifying each turn (same model, first
visit, return to tier, out of order) against the row's own columns so nothing is
read before the write. The upsert's placeholders and argument tuple both derive
from the transaction dataclass's own field order, so the SQL and the call site
cannot drift apart. GET /auto_router/benchmarks aggregates the rollup, grouped
by the full (router, type) identity, and never scans LiteLLM_SpendLogs. A turn's
cache interaction is derived once from its usage record (savings.py owns the
extraction; compute_savings_spend derives cache reads from usage_object itself),
hits are counted order-independently so the overall hit rate matches its covered
denominator, caller-chosen session ids are bounded before entering the primary
key, and a poisoned statement drops only its own session's remaining turns.
Return misses inside the recorded TTL are named for what the telemetry shows
(within_ttl) rather than a presumed cause, since a provider can evict early.
Savings ride each router's derived baseline by default, so the response carries
no deployment-wide baseline label. Rollup retention has its own
maximum_autorouter_session_retention_period setting, pattern-identical to the
spend-logs knob and running in the same cleanup job on its own cutoff. Every
drain trigger sizes the queues through one owner and the enqueue honors
disable_spend_logs beside the tool-usage queue it mirrors.
* feat(auto-router): let operators replace the LLM classifier's system prompt
The complexity router's LLM classifier has always sent one built-in rubric, so the
router could only ever grade difficulty. Operators can now supply their own system
prompt, which replaces the rubric outright and repurposes the same tier machinery for
whatever taxonomy the prompt defines, data sensitivity being the obvious case.
Replacement is total: neither the rubric nor its closing line is appended, since both
describe grading difficulty over a "current message" and a prompt grading something
else is entitled to contradict them. That closing paragraph is also the classifier's
prompt-injection defense, so the config field and the dashboard editor both warn that
a replacement omitting it lets a caller ask for a tier and get it.
The heuristic fallback still scores complexity, which is meaningless for a repurposed
taxonomy, so classifier_fallback now chooses between the heuristic scorer and routing
straight to default_model. The default_model path bypasses tier pools, the adaptive
bandit, and escalation, because no tier was decided and the point of that fallback is
a known destination. It reports itself as default_model_fallback in the spend logs.
The dashboard's prompt editor prefills from a new
/auto_router/classifier/default_prompt endpoint rather than a copy of the rubric in
the frontend, and stores no override when the draft matches the default, so later
rubric improvements still reach every router that never customized it.
Tier names stay SIMPLE/MEDIUM/COMPLEX/REASONING; a custom prompt redefines what they
mean, not what they are called.
* fix(complexity-router): don't let the default_model classifier fallback bypass routing plugins
* fix(complexity-router): don't pin a session to the default model after a classifier failure
* fix(complexity-router): omit the tier from a default-model-fallback routing decision
The classifier never answered, so no tier was decided. The record reported the
tier whose pool happens to hold default_model, which reads in the spend log and
the UI as if the request was classified. Matches how default_fallback already
records a route that no tier produced.
* fix(proxy): allowlist /auto_router/ on the UI backend component
The new GET /auto_router/classifier/default_prompt is a UI-consumed management
route, so it belongs on the control plane. Without the prefix it was exposed by
neither component and test_gateway_plus_backend_covers_full_app failed.
* docs(ui): reword the classifier prompt disclaimer
Frames the closing paragraph as a strong recommendation rather than a
description of what gets dropped, names prompt injection explicitly, and
notes the tier names stay fixed regardless of their display names.
* fix(complexity-router): stop logging a fabricated tier on the plugin fallback path
The classifier-failed fallback resolves a tier so the routing-plugin pipeline has a
pool to filter, but nothing about the request produced that tier. The non-plugin
short-circuit already dropped it from the logged decision; the plugin path still
reported it, so a spend log claimed a classification the request never received.
Record the pool as a plugin-filtered-pool signal instead.
Also name the real problem when the resolved tier has no models at all: that raised
"No candidate models left after routing-plugin filtering" and sent operators hunting
for a policy plugin that never narrowed anything.
SGR has had two independent definitions. The admin UI derived it from
SpendLogs, so it counted what litellm's logging callbacks observed and could
attribute and price. BillableRequestMetricsMiddleware counted what the proxy
actually answered at the ASGI edge, but only exported to OTLP for enterprise
metering. The two disagree by design in places, and the SpendLogs figure goes
quiet whenever spend logging is disabled or the callbacks are bypassed.
This adds LiteLLM_DailyGatewayRequests, written by the middleware, and points
the dashboard's Successful Requests tile at it.
Requests fold into an in-memory map at record time rather than going through a
queue like the spend path. A count is a pure aggregate, and every dimension of
the key is chosen by the proxy from a closed set: the date, the category, and a
route that the classifier maps to one of a fixed list of strings rather than
passing the raw path through. Nothing a caller sends can add a key, so the fold
and the table are bounded by (days x categories x routes) however much traffic
arrives; the spend queue blocks once full, which is not acceptable in the
response path. A scheduler job drains it on the existing batch interval, and a
failed flush merges its counts back so a database blip undercounts nothing.
The middleware previously returned early when no billing recorder was
injected, which is the unlicensed case. The new sink is not license-gated, so
that early return now requires both sinks to be absent. The billing recorder
keeps its 2xx-only gate; the sink takes every status so failed_requests is
real. The sink is not told which deployment served the request, unlike the
billing recorder. That id is a sha256 over litellm_params, credentials
included, so a caller who puts a credential in the request body mints a fresh
one per distinct value. No configuration is needed for that: api_base and
base_url are on _BANNED_REQUEST_BODY_PARAMS and need allow_client_side_
credentials, but api_key is not on that list, and both reach the same
_handle_clientside_credential branch. The read endpoint aggregates the
dimension away regardless, so the key is better off without it.
The new table carries no key, user or team dimension, so /gateway/daily/activity
is restricted to proxy admin roles and the per-key and per-model breakdowns
keep reading the daily spend tables. The old path is left running and marked
with TODOs.
A fetched result carries the range key it was fetched for, and the render
selects it only when that key matches the range on screen. Both the gateway
counts and the spend aggregate go through that rule: the request tiles read the
first and fall through to the second, so stamping only one of them would leave
the tile showing a superseded range by the other route.
The paginated pages behind that aggregate are reached through a failure flag,
so the flag is stamped too. A flag left over from the previous range would let
those pages through while a new range is in flight, which is the same defect
one fallback further down.
PR #35929 zeroed the eslint budget headroom while #35893 added UI code in parallel, so staging went over budget by one complexity violation and two no-large-inline-object-arg violations, failing frontend-lint on every UI-touching PR until #35964 reverted the ratchet. This removes the three violations at the source so the budgets can ratchet back down: the submit-blocked-reason chain in add_auto_router_tab moves to a module-level helper, taking the component arrow from complexity 21 to 18, and the two four-property object literals in build_complexity_router_config.test.ts move into named variables. No behavior change; the touched suites pass (101 tests)
* fix(proxy): give proxy_admin_viewer read parity with proxy_admin
Route-level checks already default-allow management GETs for the viewer
role, but ~15 handlers compared user_role to PROXY_ADMIN only, dropping
viewers into regular-user scoping (/key/list, /user/info, /model/info,
guardrails, prompts, agents, memory, workflows, MCP catalog, coordination
redis settings, credential migration check, enterprise projects). Swap
those read paths to user_api_key_has_admin_view; write gates unchanged.
The dashboard now presents the viewer session as Admin for all gating
(effectiveSessionRole) so every page fetches with admin visibility, with
userRoleLabel/isViewOnly preserving the account-menu label and the
playground cost guard. The server remains the write authority.
* refactor(agents): remove side-effectful health_check param from GET /v1/agents
Addresses a security review finding on the admin viewer read parity change:
listing agents with health_check=true made the proxy issue a server-side GET
to every agent URL, so a read-scoped caller could trigger request fan-out
beyond their object permissions. The list endpoint is now a pure read for
every role.
Removes the query param, the URL probing helper and its timeouts, the
AgentHealthCheck httpx provider tag, and the dashboard's Health Check
toggle. Requests still passing health_check=true get the full list back
with the param ignored.
* fix(proxy): keep credential encryption check proxy_admin only
The residual scan behind GET /credentials/migrate-encryption/check loads
every model, credential, MCP, team, and verification-token row and runs a
decryption attempt on each stored value. Extending it to proxy_admin_viewer
let a read-only account repeatedly trigger deployment-wide scans, so the
route keeps its original full-admin gate.
* fix(agents): restore health_check, keep list fast path proxy_admin only
Restores the agent health_check feature exactly as before this PR: the
query param, the URL probing helper, the httpx provider tag, and the
dashboard toggle all return, so existing callers keep the filtering
contract. The viewer expansion is instead reverted at its source: the
GET /v1/agents admin fast path stays PROXY_ADMIN only, so a
proxy_admin_viewer goes through the object-permission scoped branch as
before and cannot fan out health checks beyond their allowlist. The
viewer read of a single agent stays viewer-inclusive since it has no
side effects.
Import-block conflicts against #35962's unused-import removals resolve
to this branch's side: its modernization already removed every typing
import staging trimmed, plus the usages
Resolve litellm/types/google_genai/main.py and litellm/types/utils.py by
keeping this branch's modernized annotations on top of staging's removal
of inert type: ignore comments. Rebuild ruff-strict-budget.json from
measured merged-tree counts where de-excluding litellm/types adds
violations, keeping the stricter of the two sides' limits everywhere
else so no rule gains headroom. Fix the four type-discipline additions
the merge surfaced: freeze GEMINI_1_5_ACCEPTED_FILE_TYPES, drop a
callback_args parameter rebind in guardrails, and give the two remaining
mutations reasoned suppressions
* feat(complexity_router): let operators rename the four complexity tiers
Adds an optional tier_labels map to complexity_router_config so a deployment can
put its own vocabulary on the four tiers, e.g. Cheap / Standard / Premium / Deep,
instead of reading SIMPLE / MEDIUM / COMPLEX / REASONING in its dashboard, its
spend logs, and the rubric the LLM classifier reasons with.
Labels are display-only. Every config key stays canonical, so tiers,
keyword_tier_rules[].tier, and tier_boundaries are written exactly as they are
without labels, and partial maps are fine with unlisted tiers keeping their
default name. A validator rejects blank labels, two tiers sharing a label, and a
label that is another tier's canonical name, since any of those would make a log
row or a rubric line ambiguous. That validator runs on the /model/new and
/model/update write path already, so an ambiguous config gets a 400 rather than
being stored for the router to refuse later.
Under the default heuristic scorer the names are cosmetic: the scorer maps a
weighted score to a rung and never reads a tier name, verified by running the
eval corpus with and without a rename and getting identical tier and identical
score on all 29 cases. Under classifier_type: llm the labels are the names in the
rubric and the values the classifier must return, so the response format's enum
is now built from the configured labels and a reply is resolved back to its tier
against labels first, then canonical names, case-insensitively. An unresolvable
reply degrades to the heuristic on the existing fallback path. A test pins the
generated schema for an unrenamed deployment as equal to the shipped
TierClassification schema, so the wire shape can't drift.
Spend logs keep routing_decision.tier canonical so rows from before and after a
rename stay comparable, and gain routing_decision.tier_label on the tiers that
were renamed.
* refactor(complexity_router): drop added comments and the Counter construction
Review feedback: the repository guide bans new comments, so the explanatory
comments and the appended docstring paragraphs this branch added come back out.
One-line docstrings stay in complexity_router.py, matching that file's own
convention.
The duplicate-label check no longer builds a Counter, which the mutable-collection
budget counts, and the error text drops its list() reprs for joined strings. The
labels are stripped in tier_label() now rather than by rewriting the field in the
validator, so the stored config keeps exactly what the operator wrote.
schema.d.ts is regenerated: ComplexityRouterConfig is exposed in the OpenAPI spec,
so tier_labels surfaces there.
* fix(ui): carry tier_labels through the auto-router preset prefill
buildPresetPrefill maps every payload key onto form state, but the tier_labels
key added by this branch had no line, so a preset shipping labels would apply
its tiers and silently drop its names.
ruff.toml has excluded litellm/types/* since 2024, so no lint rule ever ran
on the types tree. Remove the exclusion, apply ruff --fix and ruff format
across litellm/types, and hand-fix what autofix cannot reach so the
pyupgrade budgets stay at zero: implicit type aliases converted to PEP 604
unions, RootModel[Union[...]] bases, duplicate imports, and a stray print.
Load-bearing import X as X re-exports deleted by preview-mode F401 are
restored, and the six star-imported hub modules keep their re-export
surface via per-file F401 ignores. Star-import consumers that silently
relied on typing names leaking from those hubs are modernized to builtin
generics and PEP 604 unions.
Runtime annotation introspection that only recognized typing.Union is
taught types.UnionType (guardrail UI field schemas, volcengine response
fill), with regression tests for both. Strict budget limits for the rules
the types tree now trips are raised to exact measured totals, so any
net-new violation still fails the gate
Add model_itpm_limit and model_otpm_limit to project create and update requests, storing both quota maps in project metadata without a database migration
Reserve input and output tokens independently before provider dispatch, expose separate project rate-limit headers, and reconcile counters across successful calls, failures, retries, fallbacks, streaming, caching, and cancellation
Harden token estimation for pre-tokenized embeddings, multimodal inputs, Responses API requests, native Gemini requests, multiple candidates, and conflicting output-cap aliases
Reject negative output caps, preserve conservative reservations when usage is missing or zero, bind reconciliation and refunds to the reservation window, prevent double refunds or negative counters, update generated API types, and add regression coverage
A tier request against a model that publishes tier output pricing but no
tier reasoning key (every current Gemini flash entry) billed reasoning
tokens at the standard output_cost_per_reasoning_token, undercounting
priority and fast traffic where thinking tokens dominate completions
generic_cost_per_token now resolves the reasoning rate with explicit
precedence: an explicit output_cost_per_reasoning_token_<tier> key wins,
then the tier-resolved output rate when the model prices that tier, then
the standard reasoning key, then the output base cost. The two tier
reasoning keys are wired through ModelInfo so providers can publish real
tiered reasoning prices when they exist
* feat(spend): derive a default auto-router savings baseline from the hardest tier
The savings driver shipped off by default: unless an operator names
litellm_settings.autorouter_savings_baseline_model, every auto-routed request
records $0.00 and the dashboard card never populates. Nobody discovers a knob
whose feature they have never seen work, so the default has to come from
somewhere the proxy already knows.
The router's own tier ladder is that place. Without a router a deployment runs
one model that can carry the hardest request it will see, so the derived
baseline is the priciest model in the hardest configured tier, REASONING when
present, otherwise the most severe tier the router actually defines. A cheap
tier is a choice the router made, not a ceiling it was bounded by.
An earlier draft of #35521 derived this per request and was deleted for it:
ranking candidates against the request that ran meant reading the request, and
every input shape it could take produced its own review finding. This
derivation is ranked against one fixed reference request instead, a cache-heavy
shape matching real auto-routed traffic, so it never reads the request at all.
Candidates still resolve through the router's deployments, so Azure base_model
and per-deployment pricing overrides rank correctly.
The deciding router records the result on its routing_decision, because one
model name can carry several tag-scoped routers with different tier ladders and
only the deciding instance knows which of them routed the request. The spend
writer's precedence is: configured baseline, then the recorded one, then off.
When the setting is present the router skips deriving entirely rather than
pricing candidates per decision only to be ignored.
Resolution never raises; an unresolvable baseline zeroes the driver instead of
failing a live request. Rows queued by a pod on the previous release carry no
recorded baseline and fall back to the configured setting, exactly as today.
The schema.d.ts regeneration also picks up the reminder_markers field that
UI-19232 (#35874) added without regenerating, so one hunk there is inherited
staleness rather than part of this change.
* fix(spend): cache the derived baseline, price it by deployment, keep it out of the routing preview
Three review findings on the derived baseline, addressed together because they
all sit on the same value's path from derivation to consumer.
Derivation walked and priced the hardest tier's whole pool inside a property
read on every routing decision, unbounded by pool size. The router now caches
the result per instance with a 30 second TTL, None results included, so the
hot path is a clock compare and a deployment edit still lands within a window
no operator watches closer than.
Ranking used each deployment's effective pricing but recorded only the model
name, so the spend writer priced the winning baseline at its public rate: a
hardest tier whose deployment carries a negotiated rate produced materially
wrong savings. The decision now also records savings_baseline_deployment_id
and the writer resolves it through Router.get_deployment_model_info, exactly
as the selected arm already does. The id is ignored whenever the configured
setting overrides the recorded baseline, since the setting names a model, not
a deployment.
/auto_router/test_routing returns the routing decision verbatim to team admins
while only authorizing the classifier and embedding models, so a derived
baseline would resolve another team's model-group alias into its backend
provider/model mapping and hand it to a caller never authorized for it. The
preview's throwaway router is built with derive_savings_baseline=False; its
decisions are never spend-tracked, so nothing is lost, and a source-pinning
test keeps the flag on the endpoint.
Also strips the explanatory comments this PR had added.
* refactor(spend): pin the derived baseline per router instance instead of a TTL
Creating or editing a router already rebuilds its ComplexityRouter instance,
through unregister and re-add on upsert and through the registry reset on a
full model_list load, so a value derived once per instance refreshes on
exactly the flows that can change it. That makes the TTL a solution to a
problem the rebuild lifecycle already solves, and it goes.
Derivation stays deferred to first use rather than running in __init__: during
a config load this router can be constructed before the deployments its tiers
name, and a baseline pinned at that moment would be empty for the process
lifetime.
The one behavior the TTL had that the pin does not: editing a tier deployment
without touching the router itself refreshed the baseline within a window.
That edit path rebuilds only the edited deployment's own strategies, so the
pin holds the old answer until the router is next saved or the config next
loads. A stale deployment id degrades to public-rate pricing rather than
failing, which is where every other unresolvable baseline already lands.
The template tests hardcoded the model names the presets happened to ship
with, so editing autorouter_presets.json to name newer models turned every
preset red in the fixtures and hung six waitFor calls
* feat(ui): add Test Routing to the auto router create form
Route a test prompt through the complexity-router config on screen before the router
is saved, showing the model it lands on and the same decision trace the Logs page renders.
Adds POST /auto_router/test_routing, which classifies with the live pre-routing hook and
sends nothing to the routed model.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): reset the routing test modal on reopen and expose /auto_router on the UI backend
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): enforce caller model access and key budget on the routing test's classifier call
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: tin <tin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(claude-code): make skill registration create-only with a PUT update route
POST /claude-code/plugins upserted by name, so re-registering an existing
name silently overwrote the stored skill's source and metadata. The "Add
New Skill" UI button posts here, so a name collision clobbered a different
skill with no signal to the user.
Make POST create-only: it returns 409 if the name already exists, with a
unique-violation guard mapping the find-then-create race to the same 409.
Add an explicit PUT /claude-code/plugins/{plugin_name} for updates (404 if
the name is missing). PUT is a full replace and documents that omitted
fields reset to their defaults, so UpdatePluginRequest defaults version to
None instead of fabricating the create-time 1.0.0.
The shared mutable fields move to a PluginSpec base; RegisterPluginRequest
keeps its name and its generated schema unchanged, UpdatePluginRequest
carries no name. Regenerated the dashboard types and the lazy openapi
snapshot for the new route.
Resolves LIT-4110
* fix(ui): surface the proxy error detail so the skill 409 conflict is legible
The add-skill form rendered the raw HTTPException envelope on failure
because deriveErrorMessage did not unwrap an object-shaped detail
({"detail": {"error": ...}}), so the new create-only 409 reached the user
as a JSON blob. Unwrap object-shaped detail at the client layer, which
covers every handler that returns detail={"error": ...}, and surface the
resulting message verbatim on the form instead of burying it under a
generic prefix.
* refactor(claude-code): replace blind excepts in plugin mutations with typed handling
Narrow register_plugin's create-conflict guard from a broad 'except Exception'
+ isinstance dance to a direct 'except UniqueViolationError', using an Exception
subclass sentinel (not None) as the prisma-absent fallback so the sentinel can be
caught directly. Drop update_plugin's outer 'except Exception -> 500' wrapper so
HTTPExceptions propagate on their own and unexpected DB errors surface as FastAPI's
default 500 rather than echoing str(e). Keeps the BLE001 strict-rule budget green.
* fix(claude-code): restore structured 500 handling on update_plugin via typed PrismaError catch
Flattening update_plugin to satisfy the no-blind-except rule dropped its error
wrapper entirely, so a data-layer failure (e.g. a dropped DB connection) would
skip the intentional verbose_proxy_logger.exception call and degrade the response
from the endpoint's structured {"error": ...} body to FastAPI's default
{"detail": "Internal Server Error"}, inconsistent with every sibling route.
Wrap update_plugin in 'except PrismaError' instead of the blind 'except Exception'
the other routes use: it logs and returns the structured 500 for real DB failures
while letting genuine code bugs surface rather than masking them as 'Update failed',
and stays off the BLE001 budget. Add a regression test that a PrismaError during
the update maps to a structured 500.
* fix(claude-code): import prisma error types at function level to satisfy LIT009
* refactor(claude-code): typed plugin mutation responses and lint gate fixes
Return RegisterPluginResponse models from POST and PUT instead of ad-hoc
dicts, declare them as response_model so the OpenAPI schema and dashboard
types carry the real response shape, build the stored manifest via
model_dump, and drop update_plugin's unused auth parameter (the route
dependency already enforces auth). Keeps the LIT002/B008/UP045 budgets at
their ratcheted ceilings after merging litellm_internal_staging
e2e_ui_testing and e2e_ui_testing_server_root_path run on
cimg/python:3.12-browsers, the one UI executor whose image supplies Node
rather than taking it from a cimg/node tag. That image ships Node 24.14.0,
which bundles npm 11.9.0, so both lanes have failed EBADENGINE against the
engines floor added in #35801. Every Node 24 release through 24.14.0 bundles
an npm below 11.10.0, so engines.node also rises to 24.14.1 (npm 11.11.0),
the first release where the two floors agree
The pinned install goes into /opt/node with /opt/node/bin prepended to PATH
instead of unpacking over /usr/local. On this image /usr/local already holds
npm 11.9.0, and extracting the tarball on top of it merges the two trees into
an npm that reports 11.17.0 and then exits 1 on npm ci printing no error text
at all, which is a worse failure than the one being fixed
The install moves into a reusable install_node command so the version and its
checksum have one home, shared with proxy_pass_through_endpoint_tests, and the
command refuses to run when it disagrees with ui/litellm-dashboard/.nvmrc. A
lane drifting off the version the rest of the toolchain uses is what produced
this failure, so that mismatch now stops the job instead of surfacing later as
an install error
The e2e node_modules cache key moves to v4 because the saved trees were built
by the old npm
* feat(ui): add template picker to the Add Auto Router flow
Add Auto Router now opens straight into name + an optional Template
dropdown (Anthropic/OpenAI model-family presets or Custom). A preset
prefills the full complexity-router config and collapses the Detailed
Configuration section to a one-line tier summary; choosing Custom (or
nothing yet) leaves it expanded, and a caller can toggle it manually
at any point. A preset option greys out with the specific missing
model(s) named when the caller lacks a model it needs, or while the
model list is loading or failed to load.
Prefill and submit-gating logic live in testable pure functions
(buildPresetPrefill, getReferencedModelsError) rather than inline in
the component, per the dashboard's own testing guidance.
* refactor(ui): memoize presetAvailability
Consistency with the other memoized derived values it closes over
(availableModelSet, presets). Negligible perf impact with two
presets today, but keeps the pattern uniform as more get added.
* refactor(ui): drop pointless useMemo around getAllPresets()
getAllPresets() already returns a stable module-level array
reference; wrapping it in useMemo added React machinery for
something that can't change.
* refactor(ui): hoist presets to module scope
getAllPresets() was still being called from inside the component
body on every render even after dropping the useMemo wrapper.
Resolving it once at module load, alongside PRESETS' own
module-level initialization in autorouter_presets.ts, is the
actually-clean version of the previous fix.
* fix(ui): collapse Detailed Configuration by default
It was defaulting to expanded before any template was chosen, so
the modal still opened onto the full tier/classifier form instead
of just Name + Template. Custom still auto-expands it, and a
preset still collapses it after prefilling.
* fix(ui): list Custom Configuration last in the Template dropdown
Custom is the escape hatch, not the headline choice, so the bundled
presets now come first with Custom listed after them.
Also lets the collapsed Detailed Configuration summary wrap onto
its own line(s) instead of sharing a line with the section label
and truncating mid-model-name.
* feat(ui): match preset models across "-"/"." version separators
Admins spell version numbers inconsistently (claude-sonnet-4-5 vs
claude-sonnet-4.5), so a preset's hardcoded name and a caller's
registered one can refer to the same model while differing only in
that punctuation. getMissingModels (and therefore presetAvailability
and the submit-blocking check) now treats the two as equivalent.
Applying a preset writes the caller's actual registered spelling
into the tiers, not the preset's literal string, since the caller
may only have the dotted (or hyphenated) form and never the other
one - buildPresetPrefill now takes the available-models set for
this rewrite. Two different model names never collide; only the
separator within one version number does.
* fix(ui): re-check referenced models inside submitRecommendedRouter
submitBlockedReason disables the button for a stale/missing model
reference, but Form's onFinish (wired to the same handler) fires on
a real form submission regardless of the button's own disabled
state. The other four blocking checks already re-validate inside
submitRecommendedRouter for this exact reason; this one was missing
it, so a router could still be created referencing a model no
longer in availableModelSet.
Found by Bugbot.
* Update autorouter_presets.json
Second slice of the same sweep, covering src/components. Same rule as the
first: every removal is an unused import, an unused interface or type alias,
or a local const whose only mention was its own declaration.
The modelGroupOptions computation in add_auto_router_tab goes whole rather
than losing only its binding, since a Set and two arrays allocated per render
and then discarded is no better than the dead const was.
ToolDetail is deliberately left alone. Its unread teamsData traces back to a
useQuery that still issues a /team/list request, so removing it drops a
network call; that is a behavior change and belongs in a slice that gets QA'd,
not this one.
Stacked on litellm_dead_locals_1_app_routes; review that one first.
Part of LIT-5162.
Dropping the binding but keeping the initializer left two statements that
compute a value and throw it away: a ternary in ChatUI returning rawSelected
from both branches under a comment about resolving server IDs, and an
isAdminRole call in the prompts panel that also kept its import alive.
Both computations were already unreachable in effect; remove them whole.
mcpTokenStore was the only OAuth path writing straight to window.sessionStorage;
useMcpOAuthFlow, useToolsOAuthFlow, the callback page and the edit-screen UI state
all already go through secureStorage. Align it so the OAuth surface has one storage
format instead of two.
The stored payload also carried a refresh_token that nothing ever read back. All
three read sites take access_token only, and nothing reads the mcp-session-token:
keys directly, so the field was write-only. Drop it from the store and from the four
callers that populated it. The client-forwarded modes (true_passthrough and
oauth_delegate) re-authorize rather than refresh, and authorization_code is
unaffected because it persists through storeMCPOAuthUserCredential on the backend,
which keeps its own refresh token.
Entries written before this change decode to null and are treated as absent, which
surfaces the normal Authorize prompt; they are session-scoped and expire in an hour.
Add two regression tests that decode the stored value before asserting, so neither
can pass merely because the payload is no longer plain text.
Backend parity check: a membership-granted org admin key gets 401 on
/v1/tool/list (route absent from org_admin_allowed_routes), so the
capability map denying the formatted Org Admin runtime value is the
intended behavior, now pinned by a test
@typescript-eslint/no-unused-vars is disabled in the dashboard eslint
config, so unused locals accumulated with nothing to catch them. This is
the first slice: symbols under src/app that no code reads.
Every removal is an unused import, an unused interface or type alias, or a
local const whose only mention was its own declaration. Nothing else on the
touched lines changes, so no behavior moves with it.
Part of LIT-5162.
api.ts owned the base-vs-origin precedence, the trailing-slash trim and the
base+path join inline. That logic belongs with the rest of base resolution and
was only reachable through a fetch client, so it could not be tested directly.
Extract resolveRequestUrl into resolveApiBase.ts with its own unit tests.
api.ts now only wires the shared resolver into openapi-fetch's Request option.
No behaviour change: same precedence, same trimming, same output.
* chore(build): move the Admin UI toolchain to Node 24
Node 18 and Node 20 both reached end of life (2025-04-30 and 2026-04-30), and
the release images along with every CI lane were still building on them. Node 24
is the current LTS through 2028-04-30, so this moves the four UI build images,
the CircleCI lanes, and the four GitHub Actions workflows onto it
Node 24 also ships npm 11.17, which is the first line that implements the
min-release-age setting this repo already carries in its .npmrc files. On npm 10
the key is parsed and discarded, so the release-age gate has had no effect
regardless of its value. Tightening the dashboard's engines range and turning on
engine-strict makes an unsupported npm fail loudly rather than skip the gate
quietly, and a new step in the UI build workflow probes an impossible cooldown
so an inert setting cannot pass unnoticed again
Node 24's bundled undici tightened its brand check on RequestInit.signal, which
rejects the AbortSignal jsdom installs and broke the two cases in
src/lib/http/api.test.ts that rebase a request onto a runtime base url. Under
jsdom the Request global comes from Node while AbortSignal comes from jsdom;
tests/jsdomFetchEnv.ts delegates to the jsdom environment and then restores
Node's native AbortController and AbortSignal so both come from one realm.
Upgrading jsdom does not address this, as jsdom still does not own Request
The workflows now read ui/litellm-dashboard/.nvmrc instead of repeating a
literal, so the Node version has a single source of truth, and ui/Dockerfile is
pinned by digest to match the other three build images. The lockfile changes are
npm 11 normalising the engines range and dropping optional peer entries it no
longer records
* fix(build): point every Admin UI build script at .nvmrc
The enterprise Docker path was left on Node 18. docker/build_admin_ui.sh runs
only when enterprise/enterprise_ui/enterprise_colors.json is present, which it
never is in the OSS tree, so neither CI nor a default image build reaches it;
it pinned nvm to v18.17.0 and then built the dashboard, which now requires Node
24, so a customized enterprise image would have failed EBADENGINE
All three UI build scripts now resolve the version from
ui/litellm-dashboard/.nvmrc rather than carrying their own pin, so the Node
version has a single home across Docker, CI, and local builds. build_ui.sh was
on v20 and build_ui_custom_path.sh on v18.17.0
Also drops the dependency-cooldown probe from the UI build workflow. The
engines floor plus engine-strict already fails an unsupported npm loudly at
install time, so the probe was redundant, and treating any nonzero exit from a
live registry call as proof of enforcement made it unsound besides
api.ts read globalThis.location when the module loaded, which froze the base
URL at import and pinned its test file to jsdom. The creation-time baseUrl and
the middleware's runtime rebase were also two mechanisms doing overlapping
work, and the rebase hand-copied eleven RequestInit fields on every call.
Pass openapi-fetch's Request option instead, so the constructor applies
whatever getRequestBaseUrl() returns at the moment the request is built.
registerBaseUrlGetter is now the single source of the base URL, rebaseUrl and
rebaseRequest are deleted, and the request is constructed once, so the init
openapi-fetch assembled reaches the platform Request untouched. The abort
signal is no longer copied by hand.
This preserves behaviour rather than approximating it: getProxyBaseUrl() falls
back to location.origin, so the runtime base was never empty in a browser and
the old middleware already rebased every request, discarding the creation-time
value each time.
setupTests.ts gates its DOM-only tail behind a window check; setup files run
for every environment, so that tail previously stopped any node-environment
test file from loading.
api.test.ts now runs under @vitest-environment node with its assertions intact
and no location stub, plus regressions for per-call base resolution and abort
forwarding. api.sameOrigin.test.ts covers the browser fallback to the page
origin, which needs a DOM environment.
Internal users saw the Tool Policies page but its /v1/tool/list call always
returned 401. This adds a single source of truth for which roles may trigger
which UI fetches (utils/capabilities.ts) plus a useCan hook, and wires the
Tool Policies route through it: the nav item, the page, and the query all
read the same capability, so the sidebar hides the entry, deep links render
an admin-only notice, and the query never fires. The tools list call also
moves onto a queryOptions factory
* feat(ui): add admin-configurable user banner
Proxy admins can publish a markdown announcement that renders as a
dismissible banner on every dashboard page for all authenticated users,
editable from Admin Settings > UI Settings without a redeploy. Backed by
new /get/user_banner and /update/user_banner endpoints persisting to the
existing LiteLLM_UISettings table
* fix(ui): re-surface dismissed banner on identical republish
Stamp a server-side revision on every banner update and fold it into
the client dismissal signature, so unpublishing and republishing the
same message reaches users who dismissed the earlier run
* fix(ui): stamp banner revision as an opaque uuid instead of a counter
Two overlapping admin updates could read the same prior revision and
both persist the same incremented value, letting an identical republish
collide with a previously dismissed signature. A server-generated uuid
per update makes every publication identity unique by construction with
no read-modify-write
* refactor(ui): drop the server-side banner cache
Reads go straight to the single-row table; the dashboard already
throttles fetches client-side, so the cache only added staleness
windows under concurrent updates and multiple workers
* refactor(ui): move banner storage behind a domain repository and drop the store_model_in_db gate
UserBannerRepository owns the row shape instead of the endpoint
reaching through the generic .table bridge, and publishing no longer
depends on the unrelated STORE_MODEL_IN_DB flag; a connected database
remains the only requirement
Adds the auto-router as a third optimization driver beside compression and prompt
caching: a summary card, a donut segment, and a series in the savings graph across
both the cumulative and per-day views.
The number is signed, because a switch that thrashes the prompt cache can cost more
than the cheaper rates save and an operator needs to see that. The donut plots only
drivers that saved, since a negative slice has no meaning, while the card and the
range total keep the sign. `usd()` sizes and signs off the magnitude so a small loss
renders as -$0.01 rather than "$-0.00".
The card's popover states the counterfactual and its two consequences: that a switch
pays to re-warm the cache, and that a first turn the router could not identify is
charged that write and therefore under-reported.
* feat(spend): add net auto-router savings to the cost-optimization dashboard
The dashboard credited compression and prompt caching but said nothing about the
optimization that picks the model, so the driver with the largest lever on a bill
was the one an operator could not see.
Savings are the counterfactual: without a router a deployment runs one model, and
it has to be one that can carry the hardest request, so the baseline is the
priciest model in the router's hardest configured tier. A cheap tier is a choice
the router made, not a ceiling it was bounded by. `auto_router_savings_baseline_model`
overrides it for operators who would genuinely have run something else. Both are
provider-qualified before pricing, because a bare name can resolve to a different
vendor's rates or to nothing at all, and a deployment is priced by its `base_model`
where it has one, which is how Azure deployments are priced everywhere else.
Both arms price the request's real usage through `generic_cost_per_token` rather
than re-deriving per-token arithmetic, so tiered rates, ephemeral cache-write tiers
and regional uplifts stay consistent with what was actually billed. `prompt_tokens`
already includes the cache buckets, so charging them again at the input rate would
price the same tokens twice.
Cache state is what makes this hard. The baseline serves every turn, so whether it
had the prompt cached is whether the conversation was already underway. On a
continuing conversation it wrote the prompt earlier and would only read it now, so
this request's write is what switching cost and counts against the saving. On a
first turn nothing was cached for any model, the baseline would have written the
same prompt, and both arms carry the write at their own rates. Charging the write
to both cases understates a first turn to a few percent of its value, and because
the write premium is fixed by prompt size while the saving grows with completion
length, it can render a profitable route as a loss.
That shape is read off the conversation rather than remembered: a second human ask
means an earlier turn was served. No cache, no session id, and no dependence on a
caller sending a session header. It cannot see a switch on a turn the router did
not classify, and it reads a few-shot prompt's synthetic turns as prior
conversation; both err toward charging the write, which under-claims.
The baseline and the shape ride on the existing `routing_decision` record, which is
already carried from the router to the spend log, already classified for redaction,
and already written-or-cleared per attempt. A fallback that re-enters the hook
therefore cannot leave either fact behind to be attributed to a deployment that
never routed, and no new metadata key crosses the trust boundary.
The result is signed. Whether a switch pays off is a race between the rate gap and
the cache-write cost, and a narrow gap loses; flooring at zero would hide exactly
the routing behaviour an operator needs to see. The donut plots only drivers that
saved, while the card and range total keep the sign.
Savings accrue into a new `autorouter_savings_spend` column on the six daily rollup
tables, declared `NotRequired` because rows queued by a pod on the previous release
carry no such key. It is summed by the rollup merge the cross-pod Redis drain also
runs, and carried through the aggregation query, the per-row accumulation and the
response model, so the dashboard reads a value the API actually sends. Tests
enumerate the drivers from the response model itself and assert each is summed,
accumulated, carried and totalled, so one added later cannot be half-wired.
* fix(spend): let the baseline pay for a continuing turn's own growth
`_baseline_usage` moved every cache-creation token into the baseline's read bucket
whenever the conversation was underway. That is right for a switch, where the
baseline never left the model it was on and really would only read, but wrong for a
turn that stayed put: the prompt grew, and the tokens written are that growth. They
are new to every model, so the baseline would have paid to write them too. Forgiving
it that write made the counterfactual cheaper than it was and shrank the reported
saving on ordinary steady-state traffic, by about 2% per turn.
The selected arm was never involved; it has always been priced on the real usage.
The error sat entirely on the baseline.
The condition is that the request read more than it wrote, not that it read anything.
A switch onto a model already holding a small prefix of this prompt still writes most
of it, and that write is the switch's own cost; keying off a nonzero read would have
handed such a request the full rate gap, turning +$0.0056 into +$0.1177. Comparing
the two buckets separates a warm continuation, which reads far more than it writes,
from a cold arrival, which does the reverse, and it leaves the existing invariant
intact: a request reading 0 and one reading 1 both still land in the same place.
* fix(spend): price each arm under the key litellm billed it, and see agent turns
Two ways the savings number read the wrong thing, both from identifying a model by
its name when the name is not what it costs.
The counterfactual was ranked and priced on the public rate for the model a
deployment names. A deployment may not be charged that rate: the router registers
its configured prices under the deployment's own id and deliberately keeps them off
the shared model-name key so deployments sharing a backend model do not pollute each
other. So a hardest-tier deployment configured above its public rate lost the
ranking to a cheaper candidate, and once chosen was priced at a rate nobody pays.
Which key prices a deployment is now `_select_model_name_for_cost_calc`'s decision,
the resolver the real request is billed through, rather than a second rule here that
would have to re-learn that per-second and tiered overrides count, that a partial
override still counts, and that a deployment configured at zero is priced at zero
rather than treated as unpriced.
The arm being subtracted had the same fault and a sharper edge. It priced the spend
log's `model`, which on Azure is the deployment name, absent from the cost map, so
the whole driver silently read zero for that traffic. It no longer re-derives
anything: `model_map_information.model_map_key` is what litellm actually billed the
request under, recorded at request time by that same resolver with `base_model` and
custom pricing already applied.
Separately, the conversation-shape discriminator counted human asks, and an agent
loop can run twenty turns on one of them. Its tool traffic rides `tool_result`
blocks on user turns that flatten to empty text, and `tool` roles that are never
read, so a long agentic conversation looked like its own first turn and was handed
the arithmetic that leaves the cache write on both arms. That is the one direction
this must never fail in, because it inflates. An assistant turn is the direct
evidence that something answered earlier, and it is blind to how the tool plumbing
is spelled on either surface.
* fix(spend): give the cost-key resolver both inputs the selected arm needs
The served model was resolved through one input at a time, and each choice broke the
half the other fixed.
`model_map_key` is the served model already resolved through `base_model`, which is
the only way an Azure deployment name reaches the cost map at all; without it the
selected arm priced a name absent from the map, returned nothing, and the whole
driver silently read zero for that traffic. But it is built without
`router_model_id`, so it never carries a deployment's own price overrides, and a
custom-priced deployment was compared at its public rate while the baseline used the
real override. On a deployment configured well above its public rate that inverted
the answer outright: a route that lost $21.88 reported saving $0.10.
`_select_model_name_for_cost_calc` takes both, so it gets both. Which key prices a
deployment stays its decision rather than a rule restated here.
* fix(spend): same model is only the same cost when it is the same deployment
The short-circuit compared resolved model identity, so two deployments of one model
collapsed to "no switch" and reported zero. They are not the same cost: a deployment
can carry a negotiated rate, and routing from the dear one to the list-price one is a
real saving the dashboard reported as $0.00 against a true $21.93.
Both arms now carry the key litellm prices them under, so the comparison is between
deployments rather than between names.
* refactor(spend): price from resolved rates, not from a name we keep re-resolving
Four review rounds landed on one mechanism: which identifier prices a deployment.
base_model, then the deployment id, then cache-only overrides. Each round added a
clause to a resolution rule that should not exist, and a wrong primitive fails once
per input shape, so each shape arrived as its own finding.
`Router.get_deployment_model_info` already owns this. It merges a deployment's
configured prices over the built-in map, folds in `base_model` defaults for
deployments whose name is not a model, and falls back to the model name when nothing
is overridden. Every shape hand-rolled here (cache-only, partial, per-second, Azure)
was that function re-implemented badly.
`generic_cost_per_token` now accepts already-resolved rates instead of demanding a
name it looks up itself, which is what forced the name-bending in the first place.
Both arms resolve through the owner and pass what they got: the counterfactual by the
deployment the router would have used, the served request by the deployment that
served it. The invented cost-key resolver is gone, and `Baseline` carries a
deployment id rather than a key we chose on litellm's behalf.
Net 64 insertions against 79 deletions.
* test(spend): follow _most_expensive onto the router that prices its candidates
Ranking moved through `Router.get_deployment_model_info`, since what a deployment
costs is the router's answer to give; these four cases were still calling the old
free-function signature.
* fix(spend): rank baseline candidates by what a request costs, not by two rates
"Most expensive" was decided by comparing output rate then input rate. That is a
property of a rate, not of a request: a deployment dearer per output token can be
cheaper per cached token, so the comparison ordered cache-heavy traffic backwards and
recorded the wrong counterfactual.
Candidates are now costed on one reference request through the same engine the
savings themselves use, which leaves cache read and write rates, tiered tables and
every other billing dimension to that engine rather than to another rule restated
here. The reference request is cache-heavy because auto-routed traffic is.
* fix(spend): pick the baseline against the request that ran, not a stand-in for one
Ranking happened in the pre-routing hook, where the request has not executed yet, so
candidates were costed against a hard-coded reference workload: 20k prompt, 19k of it
cached, 1k out. Which candidate is dearest depends on that mix, so a pooled hardest
tier holding a deployment with non-proportional configured rates could be ranked for
a request nothing like the one served.
The mix is known on the spend path, so the ranking belongs there. The routing
decision now carries the tier's candidates rather than a winner already chosen, and
the baseline is resolved against the usage that actually happened. The reference
workload is gone; nothing here assumes a traffic shape any more.
The router is passed in rather than imported from `proxy_server` inside the
computation, so the savings stay a pure function of their arguments and the caller
owns where the router comes from. That also makes the spend path testable without a
running proxy, which the previous shape was not.
* refactor(spend): measure savings against one configured model, not a derived one
The counterfactual was derived per request: enumerate the hardest tier's
deployments, resolve each one's effective pricing, price them all, take the dearest.
That machinery produced a review finding per input shape it had not anticipated,
and every answer it gave was one an operator could have stated in a line of config.
So they state it. `litellm_settings.autorouter_savings_baseline_model` names the
model the traffic would have run on without a router, for every auto-router on the
proxy, and unset means the driver is off rather than a model nobody named being
guessed at. `savings_baseline.py` and its tests are deleted outright, along with the
tier enumeration, the candidate list on the routing decision, and the per-deployment
override that shadowed it.
Cache-state handling is untouched: the baseline is still priced on this request's own
read and write split, so a switch still pays for re-warming the cache and a first
turn still charges the write to both arms.
45 insertions against 482 deletions.
* refactor(router): compute the conversation shape once and pass it down
`_classify_and_route` re-derived it from the messages the hook had already resolved,
so an ordinary routed request walked the turn list twice for one boolean. The hook
computes it and hands it over, which is also where the affinity-hit path already got
it from.
Also moves `_get_llm_router` below the imports it sat among.
* fix(router): drop the dead conversation_continuing parameter off the hook
It was added to `async_pre_routing_hook` by mistake and immediately overwritten by
the value the hook computes, so it never did anything. It also widened a signature
every pre-routing strategy shares with the protocol in `types/router.py`, leaving
this one router diverged from `AutoRouter` and the interface for no reason.
Also records why an unreadable request counts as continuing: no messages is no
evidence a turn was served, so it pays the cache write and under-claims rather than
being handed a first turn's larger saving on nothing.
* fix(spend): charge a baseline its input rate for cache buckets it cannot price
A model with no cache_creation_input_token_cost, which is every OpenAI, Azure and Gemini entry, resolved that rate to 0.0 and carried the whole written prompt for free, so a first turn routed onto a cheaper model reported a loss. Same hole on cache reads. Those tokens are plain input on such a model, so they move into the text bucket.
* refactor(spend): build the daily upsert payloads in one shot
`common_data` and `update_data` were constructed and then appended to: `request_id`
conditionally for tag rows, `endpoint` unconditionally a few lines later. A dict that
grows after its literal cannot be reasoned about by reading the literal, which is the
whole point of building it at once.
The conditional key resolves to a spreadable value before either payload, so both are
single expressions and the tag branch appears once instead of twice.
Not wrapped in MappingProxyType, though it was suggested: these go straight to
prisma, whose query builder branches on `isinstance(value, dict)` to tell a nested
node from a scalar. A mappingproxy is a Mapping but not a dict, so it falls through
to the serializer and raises `TypeError: Type <class 'mappingproxy'> not
serializable` inside the batch upsert, where the surrounding except would log it and
leave the rollups silently unwritten.
* fix(spend): keep the one-shot upsert payloads under the type-discipline budget
Building both payloads as single literals traded a mutation for two dict literals,
and LIT002 counts construction rather than mutation, so the change the review asked
for is the one the gate charges for.
The empty branch is the avoidable half: it is the same value every time, so it moves
to a module constant built once instead of a literal per transaction, and it is a
read-only mapping so none of the call sites that spread it can fill it in later.
"Add keyword rule" seeds a row with no keywords, and the only check that a
rule carried one lived inside getSemanticConfigError, which returns early
when semantic keyword matching is off. Off is the default, so an unfilled
row fell through to serializeKeywordTierRules and was discarded on the way
to the payload; the create reported success and the rule was gone.
The row now reports the gap itself and the submit is withheld while one is
outstanding, on the create form and the edit modal alike, both reading
emptyKeywordTierRuleIndexes so the row named and the row marked cannot
differ. Enter commits a typed keyword: the dropdown is kept closed, which
left antd nothing for Enter to select, and submitting was what used to
supply the blur that saved the word.
The backend already refused such a rule, but only when the router built the
deployment, so a caller that sent one anyway got the row written, dropped on
reload, and a 500. The management write paths now parse the incoming
complexity_router_config with the router's own ComplexityRouterConfig, judged
on the config alone so a patch that writes one without naming a model is
covered too, and reject it with a 400 having persisted nothing.
* feat(team): custom metadata validation hook for team create and update
Operators can point general_settings.custom_team_metadata_validate at an
async Python function that validates team metadata before /team/new,
POST /team/update, and PATCH /team/{team_id} commit their writes. The
hook receives the metadata that will actually be written (the merged
result on PATCH) plus the stored metadata and requester context, and
fails closed: a rejected value returns the function's own message as a
400 while any exception or timeout blocks the write with a configurable
generic message as a 503. Premium-gated like enforced_params.
* fix(team): validate metadata before model alias writes and strip system keys from validator input
Review follow-ups on the team metadata validation hook: run the validator
before the model_aliases table insert so a rejected create leaves no
orphaned model rows, strip system-managed keys from existing_metadata so
the validator sees symmetric input on both fields, and accept class
instances exposing an async __call__ as validators. Adds a three-way
validator implementation matrix (allowlist function, HTTP-service-backed
function, immutability-enforcing class instance) driven through the real
create, update, and patch endpoints, including an HTTP stub service and
outage coverage.
* test(team): run the metadata validation matrix against the DB-backed proxy in CI
Adds the validator matrix to the proxy_store_model_in_db_tests CircleCI
job so every scenario runs full e2e against a Postgres-backed proxy. The
proxy config registers a dispatching validator that routes each request
to one of the three implementations via a metadata key and accepts
anything that does not opt in, keeping the rest of the suite unaffected.
CI starts a stand-in cost center service on the host for the HTTP-backed
implementation, reached from the container via host.docker.internal, and
the outage path targets a closed port to prove the fail-closed 503
without stopping services.
* feat(ui): edit team metadata as key-value pairs in team create and edit forms
The team create and edit forms asked for metadata as a raw JSON blob in a
textarea buried under Additional Settings. Both forms now render a key-value
pair editor directly under the TPM/RPM limit fields, backed by a shared
MetadataKeyValueFields component. Values round-trip losslessly: non-string
values display as JSON and parse back to their typed form on save, and
JSON-ambiguous strings are quoted so their type survives the trip. The edit
form hides UI-managed keys (logging, guardrails, model rate limits, etc.)
that dedicated controls already own and re-add on save.
* fix(ui): explain typed JSON parsing in the team metadata help text
* feat(team): schema-driven metadata fields from team_metadata_schema config
* refactor(team): render schema metadata fields as locked key-value rows, drop allowed_values
* refactor(team): schema fields reduce to key and label, tag-rendered keys, clean rejection toasts
* refactor(ui): prepopulate declared metadata keys as ordinary key-value rows
* fix(team): let non-admin dashboard users read the team metadata schema
* test(proxy): pin timeout wiring, boundary, and error-message contracts for team metadata validation
* fix(proxy): use pooled async httpx client in the e2e team metadata validator example
* refactor(team): satisfy staging lint ratchets inherited by the merge
* fix(ui): hide guardrail review buttons from non-admin users
The team guardrail submissions list rendered Approve/Reject buttons for
non-admin users even though the backend correctly rejected the calls.
Thread userRole from the page through GuardrailsPanel into
TeamGuardrailsTab and gate the row-card and detail-panel review buttons
on isAdmin so the UI matches the backend authorization.
Defense in depth only — the backend remains the source of truth and is
double-gated at both the route admin check and the explicit endpoint
role check.
Refs LIT-2494
* refactor(ui): read userRole from useAuthorized hook instead of prop drilling
Drop the userRole prop chain through GuardrailsPage → GuardrailsPanel →
TeamGuardrailsTab. Each component reads userRole directly from the
useAuthorized hook, matching the pattern used elsewhere in the dashboard.
Tests now mock useAuthorized per case (the same pattern as
top_key_view.test.tsx) instead of passing userRole as a prop.
Refs LIT-2494
* fix(ui): drop userRole prop on GuardrailsPanel call site in src/app/page.tsx
Missed in the earlier refactor — GuardrailsPanel no longer accepts
userRole as a prop (reads from useAuthorized hook), so callers must
not pass it. The build was failing in production type-check.
Refs LIT-2494
* fix(ui): gate guardrail forward-key toggle and header editors on proxy admin
* refactor(ui): remove dead app_admin case from user role formatting
