Classification timing and session affinity are the same operator question,
so Advanced: Classification Method now carries a single "How often to
classify" radio: every request, every new user message, or once per session.
The session choice writes session_affinity and stays disabled on custom tier
sets, where the backend rejects it. Advanced: Affinity keeps the deployment
switch alone.
The serializer always writes classification_mode, matching session_affinity
on the line below it, so an explicitly stored every_request survives an
untouched save instead of being dropped back to the backend default.
max_budget and reset_at already live on the matching budget_limits entry, so
repeating them (as budget_limit and reset_at) only invited confusion about which
copy is authoritative.
budget_limits now comes back exactly as stored on /key/info and /v2/key/info.
The per-window usage moves to a sibling budget_limits_usage field keyed by
budget_duration (current_spend, budget_limit, reset_at), mirroring
model_max_budget_usage, so the stored shape that /key/update accepts never
carries a computed field.
* feat(ui): auto-router controls for context-window escalation
Adds an Advanced: Context Window Escalation section to the auto-router
form, both create and edit arms, with the toggle for
enable_context_window_escalation and a clamped decimal input for
context_window_escalation_buffer. An untouched control keeps both keys
out of the payload so the router tracks the backend defaults; an
explicit opt-out (false) survives the edit round-trip through the
managed-keys projection and the hydrator, and preset prefill maps both
keys straight through so a preset cannot silently drop them
Resolves LIT-6601
* fix(ui): clearing the context-window buffer removes it from the payload
Both review bots converged on the same defect: an emptied buffer field
early-returned in commitBuffer, the draft was discarded on blur, and the
stale number reappeared and stayed in the saved config, contradicting
the copy that an empty field tracks the backend default. An empty commit
now removes the key, which the managed-keys projection propagates as a
real deletion on edit. Also trims the narrative comments the review
flagged as restating behavior
* fix(ui): keep litellm_credential_name from LiteLLM Params JSON when no credential is selected
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(ui): drop null litellm_credential_name from AddModelPanel payload fixture
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Shadow eval jobs previously targeted only virtual keys, so deployments on
pure JWT auth (which present no key at all) could never sample their
traffic. Jobs now carry a typed (target_type, target_id) pair covering
keys, teams, and users; sampling matches the identity every request
resolves to at auth time, so team and user jobs cover JWT traffic with
no client changes.
Resolves LIT-6578
* feat(complexity_router): escalate oversized prompts to a tier that fits before dispatch
The classifier scores complexity and never prompt size, so a long agentic
session whose newest ask is trivial classifies SIMPLE onto a small-window
tier and the provider rejects it with a context-window 400 that nothing
retries. The gate runs after classification on every decision path
(classify tail and session-affinity pin), estimates prompt tokens
including the out-of-band carriers (top-level system, tools,
instructions), and when the decided tier provably cannot hold the prompt
moves the request to the lowest configured tier with a model whose
declared window fits, restricting the pick to fitting models when the
decided tier can keep it. Models with no resolvable window are never
escalated away from or onto, escalated decisions are never written as
session pins, and the decision records context_escalated plus the
original tier in spend logs.
Resolves LIT-6503
* fix(complexity_router): judge groups by smallest window, bound skips by bytes, filter adaptive picks
Review-round rework, one mechanism per finding. A group is judged by its
smallest resolvable deployment window, since the core router picks within
a group with no fit check. The counting skip is gated on UTF-8 byte
length, which BPE token counts can never exceed, so token-dense scripts
cannot slip past it; only a real tokenizer count ever moves a request and
a failed count leaves the placement alone. The fit facts now filter every
adaptive phase including cold start and the tier fallbacks. Window
questions adopt the declared provider and never resolve authenticating
providers, and a router instance without get_model_list degrades the gate
to a no-op. Tests rebuilt on real Router instances resolving deployment
model_info end to end, plus a full-path test through
async_get_available_deployment
The UI e2e suite had a class of assertions that pin how the dashboard is
built rather than what it does, so an ordinary refactor turns them red
without any user-visible change.
Geometry. The auto-router template select had two tests made of pixel
arithmetic plus a data-side="bottom" check, which is Base UI's own
positioner signal. The regression they guard (#38554) is a popup opening
on top of the control that spawned it, so both cases collapse to one
invariant: the options never cover the trigger. It now runs at both
viewport heights and reads the popup as role=listbox. The models header
test compared the tabs and refresh centers within 2px, which a padding
change flips; it now asserts the two share a row.
Structure. The logs drawer test walked xpath=../../.. from a text node
and read collapsed state off chevron icon classes. SectionHeader now
renders a real disclosure button with aria-expanded, and its two copy
buttons carry distinct names instead of both being "Copy". Sidebar group
toggles expose aria-expanded too, so the migration spec can ask for a
collapsed group by state rather than by nesting depth.
Positional lookups. keyRow.locator("button").first(), row.locator("td")
.first() and getByTestId(grid).locator("div").first() all named a
position where they meant an action; they now name the control. Table
scoping moves from "table tbody" to role=row.
Timing. Nine waitForTimeout calls are gone. Every assertion that followed
them already retried to its own timeout, so the sleeps only slowed the
run down.
Both files under tests/users/ were wrapped in test.skip("...", () => {}),
which registers one skipped test and never runs the body, so the four
tests inside had never executed and were written against a UI that has
since changed (the search placeholder is "Search by email…", the ID
filters moved into a drawer, pagination is labelled "Go to previous
page"). Rewritten against the current surface: the suite goes from 104
collected tests to 107.
Left in place deliberately: the chip and dialog-footer data-slot
selectors, because the accessible names they work around live in
components/ui/, which is shadcn CLI-managed and not hand-edited.
* feat(spend_tracking): persist router metadata in spend logs for internal router models
* test(spend_tracking): expect router_metadata key in exact-payload tests, type the routed-kwargs helper
* test(newrelic): cover static default_team_settings per-team routing
The dynamic POST /team/{team_id}/callback path for New Relic is tested, but
the static default_team_settings twin had no regression coverage. Add a test
that drives default_team_settings -> add_team_based_callbacks_from_config and
asserts the resolved trusted vars dispatch to BOTH the per-team metrics logger
(cost/usage) and the trace logger (LLM/agent spans), so a config-file customer
gets the same per-team routing as the API customer.
Also correct the /team/callback docstring: callback_name is a str validated
against the credential-capable callbacks, not a fixed langfuse/langsmith/gcs
Literal, and document the newrelic_api_key / newrelic_region vars.
* chore(ui): sync schema.d.ts with the /team/callback docstring
Regenerate the dashboard OpenAPI types for the add_team_callbacks description
change: callback_name is a validated str (not a langfuse/langsmith/gcs
Literal) and the newrelic_api_key / newrelic_region vars are documented.
* docs(newrelic): note LITELLM_OTEL_V2 prerequisite, trim test comments
Address review: team-scoped New Relic config is rejected with a 400 unless the
proxy runs with LITELLM_OTEL_V2=true, so document that in the /team/callback
endpoint and sync schema.d.ts. Drop the narrative setup comments in the new
test per the repo comment convention; the test name and docstring already say why.
* fix(proxy): deliver budget alerts on webhook-only alerting and accept ALERTING_WEBHOOK_URL
ProxyLogging.budget_alerts forwarded to the alerting pipeline only when
'slack' was in general_settings.alerting, so alerting: ['webhook'] plus
WEBHOOK_URL silently never delivered a budget alert (the config
/health/services?service=webhook exists to test). Forward when 'webhook'
is present too; SlackAlerting.send_alert already fans out per channel.
Also accept a provider-neutral ALERTING_WEBHOOK_URL env fallback for the
Slack-format channel (any Slack-compatible receiver works), mark it as a
sensitive var, and de-brand the admin UI alerting copy.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): format settings.tsx with prettier
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(ui): regenerate schema.d.ts for updated alerting description
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* ci: retrigger checks after ALERTING_WEBHOOK_URL docs merged
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The Add Model page's connection test dialog rendered a literal ", ]" beside the Close button, left over from converting an array of footer buttons into JSX children.
An Admin Viewer and a group whose name contains a slash both reached an
enabled Set budget action that could only ever come back 403 or 404. Gate
the row actions on proxy admin and on the name being addressable, with the
reason in the tooltip.
Model access group budgets shipped API-only, so the only way to give a group a
budget was a curl. Adds an Access Group Budgets tab under Models & Endpoints
that lists every group with the spend drawn against its shared pool, and a
modal to set, edit or clear the budget.
/access_group/list now carries each group's budget and spend inline, so the
table renders from one read instead of one follow-up request per row.
The paginated search select diffs the input against the selected option's
label to recover what the user just typed, so a query started from a picked
value searches for the new text rather than the label plus the new text.
That diff can only express an insertion: it walks a common prefix and a
common suffix and returns what sits between them. A deletion leaves nothing
between them, so every deletion-only edit returned the empty string.
Backspacing once in a field showing a selected label therefore threw the
edit away. The empty result was stored as the query, the controlled input
re-rendered blank, and the server was asked for the unfiltered page instead
of the text the user left in the box.
An edit that yields no insertion but did change the value is a deletion, and
there the remaining text is the query the user means. Insertions and
whole-selection replacements are untouched.
Backspacing the last digit of Context Window Size instantly refilled the default (3), since onChange mapped empty input to null and the handler coalesced null back to the default. The same defect affected Timeout (ms) and Context Character Budget. Add per-field raw draft state so an empty or partial value stays visible while focused, commit only finite values (rounded, clamped to each field's minimum), and clear the draft on blur so an abandoned edit falls back to the committed value. 0 stays a valid committed value for both context controls. Add stable ids and label associations; update tests to query by label
* feat(proxy): CyberArk Conjur secret manager configuration via Admin UI
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(ui): mock networking base-url helpers in AdminPanel test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): restore deployment CyberArk env config on delete and roll back on persist failure
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): reinit env-configured hashicorp vault manager after cyberark persist rollback
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Regenerates the lazy OpenAPI snapshot and dashboard schema.d.ts from the merged tree.
Passes list/dict copies into NewMCPServerRequest from the connector conversion and switches the
server-managed NewMCPServerRequest fields to keyword defaults so the basedpyright budget gate stays flat.
A model access group could gate which models a caller reaches but never how
much that group of callers could spend in total. Capping a shared pool meant
setting a per-entity budget on every key by hand, which caps each key
separately and still leaves no way to read what the group cost.
Spend is attributed to a group only when the group's name appears on an
allowlist the caller was granted (key, team, team-member scope, project or
org) and that group serves the requested model. Asking for a model that
merely belongs to a group attributes nothing, because nothing about the
caller named the group. Levels are unioned rather than ranked, so a team
granted "*" whose member is scoped to one group still counts as gated by
that group.
Enforcement runs on both paths tags already use: a reservation counter on
the pre-call path and a read-time max_budget check inside the existing
concurrent budget gather, so the ceiling still holds under
disable_budget_reservation.
Adds LiteLLM_ModelAccessGroupBudgetTable, which is the only place a group is
ever a row: the groups themselves stay free-text strings in
model_info.access_groups, so a row exists only once someone gives that group
a budget. GET, PUT and DELETE /access_group/{name}/budget manage it, and
/access_group/{name}/info now carries the spend and budget alongside the
models.
Open, the trace sidebar is byte-identical to before: the toggle sits over its
header exactly where it did, and the header keeps the padding that makes room
for it. Collapsed, that button has nowhere to live, so the drawer header shows
one instead, on the model row or the request id row when the log names no
model.
Both come from SidebarToggle, so they cannot drift in design. The chevrons now
point the way the sidebar will move: right while it is open, left while it is
collapsed.
Putting the toggle in the drawer header unconditionally stranded it on its own
line: the model row renders empty for a log that names no model or provider, so
the chevron sat alone above the request id.
The sidebar keeps the toggle whenever it is open, in its own header. Collapsed,
the toggle moves into the drawer header and joins the model row, or the request
id row when there is no model to join. Shared between both through
SidebarToggle so the two call sites cannot drift.
