* fix(proxy): give proxy_admin_viewer read parity with proxy_admin
Route-level checks already default-allow management GETs for the viewer
role, but ~15 handlers compared user_role to PROXY_ADMIN only, dropping
viewers into regular-user scoping (/key/list, /user/info, /model/info,
guardrails, prompts, agents, memory, workflows, MCP catalog, coordination
redis settings, credential migration check, enterprise projects). Swap
those read paths to user_api_key_has_admin_view; write gates unchanged.
The dashboard now presents the viewer session as Admin for all gating
(effectiveSessionRole) so every page fetches with admin visibility, with
userRoleLabel/isViewOnly preserving the account-menu label and the
playground cost guard. The server remains the write authority.
* refactor(agents): remove side-effectful health_check param from GET /v1/agents
Addresses a security review finding on the admin viewer read parity change:
listing agents with health_check=true made the proxy issue a server-side GET
to every agent URL, so a read-scoped caller could trigger request fan-out
beyond their object permissions. The list endpoint is now a pure read for
every role.
Removes the query param, the URL probing helper and its timeouts, the
AgentHealthCheck httpx provider tag, and the dashboard's Health Check
toggle. Requests still passing health_check=true get the full list back
with the param ignored.
* fix(proxy): keep credential encryption check proxy_admin only
The residual scan behind GET /credentials/migrate-encryption/check loads
every model, credential, MCP, team, and verification-token row and runs a
decryption attempt on each stored value. Extending it to proxy_admin_viewer
let a read-only account repeatedly trigger deployment-wide scans, so the
route keeps its original full-admin gate.
* fix(agents): restore health_check, keep list fast path proxy_admin only
Restores the agent health_check feature exactly as before this PR: the
query param, the URL probing helper, the httpx provider tag, and the
dashboard toggle all return, so existing callers keep the filtering
contract. The viewer expansion is instead reverted at its source: the
GET /v1/agents admin fast path stays PROXY_ADMIN only, so a
proxy_admin_viewer goes through the object-permission scoped branch as
before and cannot fan out health checks beyond their allowlist. The
viewer read of a single agent stays viewer-inclusive since it has no
side effects.
The Prisma CLI is a Node program that installs a private Node runtime on its
first invocation. That one-time install shared the 60s budget that bounds each
migration command, so on a slow or cold machine it was killed before it could
finish. Prisma then decides whether to reinstall by testing the cache
directory for existence alone, and a killed install leaves that directory
behind, so every later attempt skipped the install and failed on a node binary
that was never written. The existing four-attempt retry loop could not help:
each attempt hit the same missing binary, which turned a slow start into a
container that never migrated again.
Migrations now prepare the toolchain as its own step under its own budget, and
a cache directory that exists without a node binary is deleted first so an
interrupted install reinstalls instead of persisting. Both budgets are
overridable, LITELLM_PRISMA_BOOTSTRAP_TIMEOUT for the install and
LITELLM_PRISMA_COMMAND_TIMEOUT for each Prisma command, and every previously
hardcoded timeout now goes through one helper rather than thirteen literals.
The per-command default stays at 60s.
An override is only honoured when it parses as a finite positive number.
Infinity and NaN parse as floats and survive a plain positivity check, and
subprocess treats either as no deadline at all, so a value like `inf` or a
fat-fingered `1e400` would have silently disabled the timeout it was meant to
configure.
A custom tier set broke the derived savings baseline: _hardest_tier_models
walked the built-in severity order and matched nothing, so savings metadata
silently vanished from spend logs. Custom sets define no severity order, so
every defined pool is now a candidate and resolve_baseline's cost ranking
picks the counterfactual; the proxy-wide autorouter_savings_baseline_model
setting still wins over derivation.
Registration now validates complexity_router_config once and derives the
default model from the validated config, so a custom-tier deployment without
complexity_router_default_model registers with its fallback tier's model
instead of failing on the MEDIUM/SIMPLE lookup. Every defined tier must map
to at least one model, and tier_labels is rejected alongside tier_definitions
since labels rename tiers a custom set removes.
Three follow-ups surfaced while merging current staging into this branch.
`exc_info=True` at both fallback-failure log sites handed a live exception to
the logging machinery. SecretRedactionFilter rewrites `record.exc_text`, but
`record.exc_info` stays an exception object no filter can reach, so a handler
that renders it itself (Datadog and OTel log bridges do) received the
unredacted provider key. Both sites now pass `redact_string(traceback.format_exc())`
as a `%s` arg, keeping staging's lazy-logging form. The existing test only
asserted on `exc_text`, so it passed under the bug; it now renders `exc_info`
the way a bridge handler would and covers every record the call emits.
The eager deferred-stream fetch existed only on the async path. Vertex and
Bedrock build the same `completion_stream=None` plus `make_call` wrapper on
their sync branches, so `Router.completion(stream=True)` still surfaced the
provider error on first iteration, outside `_completion`'s except block, and
never reached the fallback chain. `_completion` now calls `fetch_sync_stream()`
under the same guard `_acompletion` uses.
The first of the three header-strip passes in the proxy error path was dead:
only the custom-header update and the response-headers hook run before the
second pass re-filters everything. Collapsed to one `safe_headers` binding.
tier_labels renames the built-in four tiers for display and the classifier
rubric; tier_definitions replaces the set outright. The merged classifier path
keeps this branch's (name, description) rubric plumbing and feeds the labeled
built-in tiers through it, so a labeled router still emits labels on the wire
and resolves replies label-first, while a custom set uses its defined names
exactly. The two features are mutually exclusive at config validation since
labels are keyed by the tiers a custom set removes. The README section this
branch added moves to the docs repo instead of riding here.
* feat(complexity_router): let operators rename the four complexity tiers
Adds an optional tier_labels map to complexity_router_config so a deployment can
put its own vocabulary on the four tiers, e.g. Cheap / Standard / Premium / Deep,
instead of reading SIMPLE / MEDIUM / COMPLEX / REASONING in its dashboard, its
spend logs, and the rubric the LLM classifier reasons with.
Labels are display-only. Every config key stays canonical, so tiers,
keyword_tier_rules[].tier, and tier_boundaries are written exactly as they are
without labels, and partial maps are fine with unlisted tiers keeping their
default name. A validator rejects blank labels, two tiers sharing a label, and a
label that is another tier's canonical name, since any of those would make a log
row or a rubric line ambiguous. That validator runs on the /model/new and
/model/update write path already, so an ambiguous config gets a 400 rather than
being stored for the router to refuse later.
Under the default heuristic scorer the names are cosmetic: the scorer maps a
weighted score to a rung and never reads a tier name, verified by running the
eval corpus with and without a rename and getting identical tier and identical
score on all 29 cases. Under classifier_type: llm the labels are the names in the
rubric and the values the classifier must return, so the response format's enum
is now built from the configured labels and a reply is resolved back to its tier
against labels first, then canonical names, case-insensitively. An unresolvable
reply degrades to the heuristic on the existing fallback path. A test pins the
generated schema for an unrenamed deployment as equal to the shipped
TierClassification schema, so the wire shape can't drift.
Spend logs keep routing_decision.tier canonical so rows from before and after a
rename stay comparable, and gain routing_decision.tier_label on the tiers that
were renamed.
* refactor(complexity_router): drop added comments and the Counter construction
Review feedback: the repository guide bans new comments, so the explanatory
comments and the appended docstring paragraphs this branch added come back out.
One-line docstrings stay in complexity_router.py, matching that file's own
convention.
The duplicate-label check no longer builds a Counter, which the mutable-collection
budget counts, and the error text drops its list() reprs for joined strings. The
labels are stripped in tier_label() now rather than by rewriting the field in the
validator, so the stored config keeps exactly what the operator wrote.
schema.d.ts is regenerated: ComplexityRouterConfig is exposed in the OpenAPI spec,
so tier_labels surfaces there.
* fix(ui): carry tier_labels through the auto-router preset prefill
buildPresetPrefill maps every payload key onto form state, but the tier_labels
key added by this branch had no line, so a preset shipping labels would apply
its tiers and silently drop its names.
* feat(spend): derive a default auto-router savings baseline from the hardest tier
The savings driver shipped off by default: unless an operator names
litellm_settings.autorouter_savings_baseline_model, every auto-routed request
records $0.00 and the dashboard card never populates. Nobody discovers a knob
whose feature they have never seen work, so the default has to come from
somewhere the proxy already knows.
The router's own tier ladder is that place. Without a router a deployment runs
one model that can carry the hardest request it will see, so the derived
baseline is the priciest model in the hardest configured tier, REASONING when
present, otherwise the most severe tier the router actually defines. A cheap
tier is a choice the router made, not a ceiling it was bounded by.
An earlier draft of #35521 derived this per request and was deleted for it:
ranking candidates against the request that ran meant reading the request, and
every input shape it could take produced its own review finding. This
derivation is ranked against one fixed reference request instead, a cache-heavy
shape matching real auto-routed traffic, so it never reads the request at all.
Candidates still resolve through the router's deployments, so Azure base_model
and per-deployment pricing overrides rank correctly.
The deciding router records the result on its routing_decision, because one
model name can carry several tag-scoped routers with different tier ladders and
only the deciding instance knows which of them routed the request. The spend
writer's precedence is: configured baseline, then the recorded one, then off.
When the setting is present the router skips deriving entirely rather than
pricing candidates per decision only to be ignored.
Resolution never raises; an unresolvable baseline zeroes the driver instead of
failing a live request. Rows queued by a pod on the previous release carry no
recorded baseline and fall back to the configured setting, exactly as today.
The schema.d.ts regeneration also picks up the reminder_markers field that
UI-19232 (#35874) added without regenerating, so one hunk there is inherited
staleness rather than part of this change.
* fix(spend): cache the derived baseline, price it by deployment, keep it out of the routing preview
Three review findings on the derived baseline, addressed together because they
all sit on the same value's path from derivation to consumer.
Derivation walked and priced the hardest tier's whole pool inside a property
read on every routing decision, unbounded by pool size. The router now caches
the result per instance with a 30 second TTL, None results included, so the
hot path is a clock compare and a deployment edit still lands within a window
no operator watches closer than.
Ranking used each deployment's effective pricing but recorded only the model
name, so the spend writer priced the winning baseline at its public rate: a
hardest tier whose deployment carries a negotiated rate produced materially
wrong savings. The decision now also records savings_baseline_deployment_id
and the writer resolves it through Router.get_deployment_model_info, exactly
as the selected arm already does. The id is ignored whenever the configured
setting overrides the recorded baseline, since the setting names a model, not
a deployment.
/auto_router/test_routing returns the routing decision verbatim to team admins
while only authorizing the classifier and embedding models, so a derived
baseline would resolve another team's model-group alias into its backend
provider/model mapping and hand it to a caller never authorized for it. The
preview's throwaway router is built with derive_savings_baseline=False; its
decisions are never spend-tracked, so nothing is lost, and a source-pinning
test keeps the flag on the endpoint.
Also strips the explanatory comments this PR had added.
* refactor(spend): pin the derived baseline per router instance instead of a TTL
Creating or editing a router already rebuilds its ComplexityRouter instance,
through unregister and re-add on upsert and through the registry reset on a
full model_list load, so a value derived once per instance refreshes on
exactly the flows that can change it. That makes the TTL a solution to a
problem the rebuild lifecycle already solves, and it goes.
Derivation stays deferred to first use rather than running in __init__: during
a config load this router can be constructed before the deployments its tiers
name, and a baseline pinned at that moment would be empty for the process
lifetime.
The one behavior the TTL had that the pin does not: editing a tier deployment
without touching the router itself refreshed the baseline within a window.
That edit path rebuilds only the edited deployment's own strategies, so the
pin holds the old answer until the router is next saved or the config next
loads. A stale deployment id degrades to public-rate pricing rather than
failing, which is where every other unresolvable baseline already lands.
The dashboard pins engines node >=24.14.1 with engine-strict, so make
bootstrap dies with EBADENGINE on any shell whose default node is older.
Wrap the npm install in scripts/with_dashboard_node.sh: it execs the
command as-is when node already meets the floor, otherwise activates the
.nvmrc version via nvm or fnm, and fails fast with install instructions
when neither manager exists
