* feat(complexity_router): plan-mode tier floor for coding-agent clients
Claude Code and Copilot signal plan mode only through client-injected prompt
text, which the ask-extraction path deliberately strips, so the router could
never see it. Detect the sentinels on the raw wire body and route those
requests to at least plan_mode_min_tier.
The floor is raise-only and transient: classifier results above it still win,
it overrides a session-affinity pin only on turns carrying the sentinel
without rewriting the pin, and plan_mode decisions are not pinnable, so the
first turn after plan mode exits routes as if plan mode had never happened.
Classification is skipped when the floor is the top configured tier. On
adaptive routers the floor rides _soft_floor_pick as a hard_floor that
excludes below-floor candidates, closing the adaptive_eligible=all gap where
a request classified at or above the floor could still route below it.
Detection is staleness-aware: only leading system content and the newest-ask
tail count, so sentinels surviving in history after plan mode exits, built-in
or operator-supplied, never fire. Custom tier sets are supported with
severity from the tier_definitions list order, same as keyword_tier_rules.
Off by default; decisions are recorded with the new plan_mode cause and the
matched sentinel in matched_keyword
* fix(complexity_router): gate pin writes and the failure exit on sentinel presence, not the floor binding
A plan-mode turn classified at or above the floor keeps its ordinary cause,
but pinning it would carry a plan-mode-shaped choice past plan mode's exit
(on adaptive routers the hard floor constrained that pick), so no
sentinel-carrying turn writes the session pin. The default_model failure
exit is skipped for sentinel turns for the same reason: default_model's
placeholder tier can equal the floor while default_model itself sits in no
pool the floor can vouch for
Moves CreateTagModal, the tag detail editor and MemoryEditModal off antd Form
onto react-hook-form with the shadcn field kit, and onto semantic colour
tokens so the pages follow the dashboard theme.
Submitted payloads are unchanged and pinned by tests. The collapsible budget
sections keep values typed into them when collapsed and reopened, matching
what antd's store did, rather than reverting to the seeded default.
* refactor(ui): migrate budget and skill forms to react-hook-form and shadcn
Moves three dashboard forms off antd Form onto react-hook-form plus the shared
shadcn field kit, and onto semantic color tokens so they render correctly in dark
mode. The submitted request bodies are unchanged.
Budget create and edit previously relied on antd InputNumber precision={2}, which
rounds the submitted value rather than only the display. That rounding is now an
explicit shared helper so the wire payload stays identical, and the helper carries
unit tests covering key presence, null passthrough, negatives and non-finite input.
Characterization tests for both budget modals were written against the antd
implementation first and pass unchanged against the migrated components, which is
what pins the payload. They are named .integration.test.tsx per the dashboard test
tiers, with the pure rounding logic unit tested separately.
* fix(ui): keep collapsed Optional Settings values so reopening does not lose them
react-hook-form shouldUnregister deletes a field's value when its section
unmounts, so typing a budget, collapsing Optional Settings and reopening it
submitted the seeded default instead of what was typed. antd reported only
mounted fields in onFinish but preserved their values in its store, so the
two behaviours have to be reproduced separately.
Drop shouldUnregister, drive the section from controlled state, and blank the
section's fields at submit while it is closed. Seed the edit form from the
five form fields rather than the whole budget record, which shouldUnregister
had been masking.
The ChatUI playground test helper looked up the model combobox with a
synchronous getByPlaceholderText. That control renders its placeholder
from the model-loading flag, so the text is "Loading models..." until
the mocked fetch resolves, and the element the helper wants does not
exist yet. Under a loaded full-suite run the query could land inside
that window and fail with "Unable to find an element with the
placeholder text of: Select a Model", while the same test passed in
isolation every time.
Switch the helper to findByPlaceholderText so it waits for the control
to come back after loading.
Annotate _prime_oauth_metadata_discovery_for_servers and
_reconcile_oauth_discovery_slots_for_servers with Sequence and snapshot
registry views with tuple() at the call sites. This drops the Iterable
addition to the collections.abc import, restoring that line to its base
spelling so the branch merges cleanly with litellm_internal_staging,
which adds Mapping on the same line
* fix(proxy): strip callback credentials from the auth object stamped into request metadata
* style(proxy): drop the restating half of the stamp-site comment
* test(proxy): pin that the stamped auth copy carries header-derived identity
GET /guardrails/usage/overview and GET /guardrails/usage/detail/{id} 500ed on a
database that has not applied 20260817143646_add_daily_guardrail_usage_units yet
(pip installs on litellm-proxy-extras 0.4.86 with DISABLE_SCHEMA_UPDATE=true).
Both endpoints now return their metrics with empty units and log one warning
until the migration lands.
The daily guardrail metrics and usage-unit upserts are non-idempotent
increments, but the retry loop re-sent every failed row on any exception.
An ambiguous post-send failure such as a read timeout after the write had
already committed therefore stacked a second increment and inflated the
billable unit totals served by the guardrail usage endpoints.
Retry only DB_RETRY_SAFE_ERROR_TYPES (httpx.ConnectError), the same rule
the spend writer and autorouter rollup use for increment upserts, and log
any other failure once as terminal for that row while the rest of the
batch still lands.
Follows up #37225
* refactor(ui): move dashboard toasts from antd message/notification onto sonner
Add lib/toast.ts as the single toast surface (success/info/warning/error/
fromError/dismiss) backed by sonner, with a <Toaster /> in the root layout.
fromError titles a toast from the proxy error type or the HTTP status instead
of matching prose phrases, and shows the extracted proxy message as the
description.
MessageManager and NotificationManager become thin facades over lib/toast so
the ~250 existing call sites keep working; the mutable antd instance setters,
setMessageInstance/setNotificationInstance, and the antd App/message/
notification providers in AntdGlobalProvider are gone. Prunes the eslint
suppression baseline accordingly.
* test(ui): mock the MessageManager seam in the Fallbacks tests and drop toast doc comments
AddFallbacks and FallbackSelectionForm asserted on a mocked antd message spy
that MessageManager no longer calls; they now mock the facade the components
import. Also removes the explanatory comments Greptile flagged in lib/toast.ts
and both facades.
* fix(ui): keep NotificationManager's antd config-object contract on the sonner facade
success/info/warning/error accept the { message, description, duration } object
form again (CreateMCPServer's admin-review notice uses it) and fromBackend keeps
its extra.duration seconds argument, both mapped onto lib/toast. Prunes stale
suppressions picked up by the rebase.
* feat(ui): read the proxy error type and code out of JSON envelopes embedded in string errors
Legacy networking helpers throw new Error(responseText) and callers prefix
that text, so the envelope arrives as a substring. fromError now parses the
first embedded JSON object for type/code and shows the unwrapped message in
its place, so those toasts get a status title (Request Error, Not Found) and
a readable description instead of raw JSON.
* feat(ui): configure the auto router's heuristic scorer from the Admin UI
The complexity router has always read tier_boundaries, token_thresholds and
dimension_weights from its config, and /model/new already persists them, but the
dashboard had no control for any of the three, so tuning the scorer meant editing
config.yaml by hand.
Adds an "Advanced scoring" panel to the classification section, shown whenever the
scorer actually runs: on a heuristic router, and on an LLM classifier that falls back
to the heuristic. An untouched knob is omitted from the payload, so a router keeps
tracking the shipped defaults instead of freezing today's numbers.
The three keys join MANAGED_COMPLEXITY_ROUTER_KEYS, so the edit modal now rebuilds
them from form state rather than carrying the stored copy through. That makes
hydration load-bearing, and it hydrates an absent knob to undefined rather than to
the defaults, so an untouched save cannot pin a router that was tracking them.
The "How Classification Works" card now reads the configured boundaries instead of
hardcoding 0.15 / 0.35 / 0.60, which would otherwise start lying the moment an
operator changed them.
* test(complexity_router): pin the dashboard scorer defaults against config.py
The Admin UI keeps its own copy of the boundary, threshold and weight defaults to
prefill its controls. The copy is display only, since an untouched knob is omitted
from the payload, so drift shows a stale placeholder rather than pinning a router.
Nothing caught that drift before, and a blank or dead control is worse, so the two
copies and the dimension key set are pinned against each other here.
* fix(ui): surface out-of-order scorer thresholds as an error, not a hint
Boundaries that decrease make the tiers between them unreachable, which silently
changes where traffic goes, so amber body text undersold it. Saving stays allowed:
a router configured this way in config.yaml would otherwise become uneditable in
the UI for every unrelated change.
* fix(test): search the default-model picker instead of trusting option order
The pinned model is appended after every model the presets contribute, and that list
has reached 11, so the option fell outside the virtualized dropdown's rendered slice
and the two default-model-pin cases failed on staging. CI only runs them when this
file is touched, which is why they went unnoticed. Searching for the model filters
the list to it, so the cases no longer depend on how long the preset list grows.
* revert(test): drop the dashboard scorer defaults parity test
It parsed TypeScript from Python with a hand-rolled brace matcher and a numeric
literal regex, which is not a mechanism this repo should carry: two review rounds
went into fixing the parser rather than the feature. The UI copy of the defaults is
display only, since an untouched knob is omitted from the payload, so drift shows a
stale placeholder and cannot pin a router.
* fix(ui): clamp the scorer inputs and drive the panel from one group spec
min and max are inert attributes on a text input, so the fields accepted a weight of
999, a boundary of -50, and Infinity, and persisted them into the router config.
Values are now clamped on commit and non-finite input is refused.
The three sections were near copies of each other, so they now render from a single
group spec, which also removes the triplicated warning logic.
Moves the scorer constants and types into heuristic_scoring_knobs, the leaf module.
Reading them back through ComplexityRouterConfig was a cycle, so the top-level
DIMENSION_KEYS.map in the panel ran while the constant was still undefined and every
test importing it failed to collect.
* feat(ui): serve the scorer defaults from the proxy instead of mirroring them
The dashboard kept its own copy of DEFAULT_TIER_BOUNDARIES, DEFAULT_TOKEN_THRESHOLDS
and DEFAULT_DIMENSION_WEIGHTS to prefill the Advanced scoring controls. Two copies of
one fact, and the earlier attempt to police the gap parsed TypeScript from a Python
test, which was worse than the problem.
GET /public/complexity_router/scorer_defaults now returns them, following the
/public/providers/fields pattern: a typed response model, the dashboard fetching it
through a react-query hook next to useProviderFields. The controls and the "How
Classification Works" card both read that, so a recalibration of the defaults can no
longer leave the form stating numbers the router stopped using.
The dimension set now comes from the proxy too, so a dimension added backend-side
renders without a dashboard change, under its raw key until it is given a label.
Hydration keeps a stored dict exactly as stored rather than filling it from a local
copy, since the backend already defaults any key omitted at scoring time.
* fix(types): type the scorer defaults response as Mapping, not dict
LIT001 gates mutable collections in annotations, and the three dict fields tripped it.
Mapping is what the codebase already uses for a read-only map on a response model, and
the endpoint hands the config constants over directly rather than copying them into a
fresh dict, which would have traded the LIT001 hit for a LIT002 one.
* test(ui): stub the scorer defaults request for the auto-router tree
The Advanced scoring panel and the classification card read the shipped defaults over
the network, so every render of that tree in a test paid for a request jsdom cannot
serve. That was enough to push the slowest default-model-pin case past its 30s timeout
on CI, where the suite runs 14 forks in parallel.
One fixture in tests/mocks, pulled in by a single vi.mock line per test file, rather
than the same stub pasted into each of the seven that render the tree.
* fix(ui): tell a failed scorer-defaults load apart from a slow one
The panel read only the query's data, so a permanent failure was indistinguishable
from a request still in flight and it sat on "Loading the shipped defaults..." for
good. It now branches on the query state: pending says loading, an error says so and
offers a retry, and the values the router already overrides stay visible and editable
either way.
Two more places had the same flaw. The classification card silently dropped the tier
ranges it used to always show, and now says they could not be loaded. The weight total
was summed over whatever keys were present, so a failed load made it state a total
built from the overrides alone; a total is only shown when the dimension set is known.
A pass that raises (a missing Slack webhook, say) now waits the daily interval instead of logging the
same exception every 30 seconds, and the Admin UI alerting settings list the new alert type so it can be
toggled like the others
The trailing-slash normalization test used gateway.litellm-sandbox.ai as
its base URL. Swap it for gateway.example.com so the test file does not
reference a real-looking hostname. The test is fully mocked, so the host
value has no effect on what is exercised.
Co-authored-by: yuneng-jiang <yuneng@berri.ai>
resolve_fireworks_resource_name prefixes bare names with
accounts/fireworks/models/ (or routers/ for *-fast). Azure AI Foundry
hosts Fireworks models under deployment ids like FW-Kimi-K3; rewriting
those yields 404 DeploymentNotFound.
Leave names that already start with FW- unchanged. Native Fireworks
short names still get the accounts/ path.
Co-authored-by: Cursor <cursoragent@cursor.com>
A transient DB error during the spend log flush dropped that batch's guardrail
metrics and usage unit rows for good. Retry only the rows that failed, up to 3
times with 1s/2s/4s backoff, mirroring the daily spend writer, and inject the
sleep so tests stay fast. Lowers the lint budgets the refactor freed up
An empty pass no longer holds the daily lock, a False lock claim (held or redis
error) is retried on the next 30 second poll instead of sleeping a day, and a
sent alert is stamped in the shared cache for a day so sibling pods and restarts
stay quiet
The flush and the usage endpoints summed units with a scan per distinct key,
quadratic in rows times keys; group sorted rows instead. Skip payloads without
a request_id like the metrics path, type the flush key as a NamedTuple, and drop
the (guardrail_id, date) index that the primary key already covers
* fix(shadow_eval): copy messages before router call and raise judge output cap
* fix(shadow_eval): lead failure detail with location and pin post-failure continuation
Config-driven pass_through_endpoints pointed at a comprehendmedical.*.amazonaws.com
target were being claimed by the Comprehend Medical logging handler through the
hostname arm, which overrode their operator-set cost_per_request and relabeled
their spend rows. Only the built-in /comprehendmedical routes tag the provider,
so match on that alone.
Also mirror /comprehendmedical into the helm ingress and terraform gateway
prefix lists that hand-copy gateway/routes/allowlist.py
Regenerates the guardrails and policy_engine fragments of the lazy OpenAPI
snapshot for the usage-unit fields, regenerates schema.d.ts from it, and
exports DailyGuardrailUsageUnitsRepository next to its sibling repositories
Per-row guards in the daily metrics and usage unit flush so a single DB error no longer drops the rest of the batch, plus removal of narrating comments flagged in review
Both loading tests mock searchToolQueryCall as a promise that resolves on a
timer, assert the loading affordance, then return with that promise still in
flight. When the worker outlives the file's jsdom environment, the component's
setIsLoading(false) runs against a torn-down window, and React reports
"ReferenceError: window is not defined" as an unhandled rejection. Vitest counts
that as an error, so ui-unit-tests fails the job while reporting every one of
its 7351 tests as passed.
Awaiting the settled state keeps both assertions and leaves nothing pending at
teardown.
The Noma guardrail sends the conversation to the scanner in `inputs`. It
also forwarded `request_data` whole, which repeats that same conversation
under `messages` (or `input` on the responses API), and attached
`logging_obj.model_call_details`, which repeats it a third time.
For image-heavy calls that duplication is most of the request. A
production scan of a request carrying base64 images measured 100MB total,
of which 94.8MB was `request_data` against 5.1MB of `inputs` - the proxy
was uploading ~95% redundant bytes, and paying to serialize them.
Drop `messages` and `input` from `request_data` and from
`model_call_details`. This is a denylist rather than an allowlist on
purpose: every other key is still forwarded untouched, so a scanner-side
change that starts reading a new `request_data` key needs no matching
release of this hook. The removed keys are ones the scanner never reads -
it takes context only from metadata, litellm_metadata,
provider_specific_header, litellm_session_id/trace_id/call_id, stream,
response/responses ids, and litellm_logging_obj.complete_streaming_response,
all of which still pass through.
The conversation still reaches the scanner in full via `inputs`, so no
detection coverage changes.
Trimming happens before serialization, so the duplicate is never encoded.
Existing payload tests asserted the duplication; they now assert the trim
while keeping what they originally guarded - deep-copy semantics and the
unpicklable-object (uvloop.Loop) regression.
