* feat(proxy): require trace IDs per team on LLM, MCP and agent requests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): assert trace ID authorization outcomes
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): clear trace ID type gate errors
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(proxy): format trace ID type check
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): validate trace ID metadata
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): use read-only trace metadata types
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): skip trace ID metadata validation for teams without dict metadata
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* perf(proxy): check team trace ID setting before route matching
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): simplify playground decision endpoint
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): expose both decision API shapes in playground
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): omit undiscovered decision models from examples
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): align decisions form and response views
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(guardrails): split logging_only_scope direction from input-failure policy
logging_only_scope is now direction-only (input, output, unset for both
directions). The new logging_only_continue_on_input_failure flag controls
whether a flagged or raising logging_only input scan still lets the
response be scanned. "both" stays accepted everywhere as a deprecated
alias for unset scope plus the flag set to true, normalized at callback
init while stored rows keep the written value.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(dashboard): narrow continue-on-input-failure update type and stay under guardrail_info max-lines
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(dashboard): keep custom-code guardrail controls readable with logging_only
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(dashboard): scope continue-on-input-failure to the default direction and restore verdict assertion
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(guardrails): remove the both value from logging_only_scope
logging_only_scope only shipped its both value in dev prereleases, so it is
dropped instead of kept as a deprecated alias. Leaving the scope unset and
setting logging_only_continue_on_input_failure gives the same behavior. A
stored both value is now treated like any other invalid scope: it is logged
and ignored on load, and rejected with 422 on create or update.
* fix(guardrails): tolerate an invalid logging_only_scope in the v1 guardrail list
A config.yaml guardrail whose logging_only_scope is no longer valid, such as
the removed both value, made GET /guardrails/list return 500 for every
guardrail. The v1 list now parses params with the same tolerant parser as
the v2 list, info and PATCH paths, so the bad value is logged and dropped.
* test(guardrails): drive continue-flag rejection tests through the real guardrail handler
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: yucheng-berri <293926326+yucheng-berri@users.noreply.github.com>
* feat(ui): list the decision providers from the cost map in the decision models banner
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): title-case decision providers the dashboard has no display name for
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(ui): wait for the catalog query to fail before asserting the banner fallback
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): replace Add a decision model with a Supported providers docs link in the decision models banner
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): drop the Add Model search hint from the decision models banner
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): call it the Decisions playground in the decision models banner
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): call the System One playground the Decisions playground across the dashboard
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): render the System One JSON editor with the stock One Dark theme
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): rename the Playground System One tab to Decisions and use the VS Code Dark+ JSON theme
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): keep the JSON editor header and issue list on the light surface
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(ui): drop the root block scanner test and the unused band styles
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): follow the dashboard theme in the JSON editor and drop the breakdown accents
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): round the Decisions score pill to two decimals
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): strip the Prism block backgrounds so the JSON editor theme swap stops the React style warning
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): let admins make per-model key/team rate limits a hard 429 instead of falling back
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): sync per-model rate limit API types
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): type per-model fallback setting access
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): keep the per-model hard cap inside the fallback loop and test it through the real limiter
* fix(proxy): list the per-model hard cap setting on the settings page and cover every per-model descriptor
* fix(proxy): read disable_fallbacks_on_per_model_rate_limits as a boolean value and cover the team-wide per-model cap
* test(proxy): cover per-model rate limit 429s skipping fallbacks on a live proxy
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(ui): build System One playground requests with a form
* test(ui): fill System One form fields with single change events
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(ui): announce decision model support in Models + Endpoints
* feat(ui): make decision models findable, callable, and testable from the Admin UI
Add Model provider search now matches decision model names and shows a note on where to call a decision model. The banner links to the docs and opens Add Model. The System One playground defaults to /v1/systemone with a picker of the proxy's decision models, and the model hub shows a /v1/systemone usage example for evaluation models.
* fix(ui): keep the admin signed in when a typed virtual key cannot list decision models
The System One playground looked up a typed virtual key's decision models through
the dashboard's shared request client, so an expired key signed the admin out, and
typing a key by hand sent one lookup per keystroke. The lookup now uses the same
client as Send, against the same proxy, and waits for typing to stop
* fix(ui): hide decision model Playground links from view-only sessions
A proxy admin viewer cannot open the Playground, so the Models + Endpoints
decision models banner and the AI Hub's Try it in the Playground link sent
them to an Access Denied page. Hide both for view-only sessions.
* fix(ui): send the AI Hub decision example's key in the proxy's configured header
A proxy with litellm_key_header_name set reads keys only from that header,
so the example's hardcoded Authorization header was rejected. Use the header
name the dashboard already reads from the session.
* fix(ui): stop the decision model notice from ruling out /chat/completions for dual-mode models
* fix(ui): point only users who can add models at Add Model in the decision models banner
---------
Co-authored-by: moyai-devin-berriai[bot] <336287033+moyai-devin-berriai[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(ui): resolve the provider dropdown key to the slug the backend declares
The Add Model form submits the `provider` field from /public/providers/fields
verbatim, but provider_map is keyed by hand on the frontend, so the two sides
disagree for 14 of the 124 providers the backend serves. Some differ only in
case ("MINIMAX" vs "MiniMax", "CURSOR" vs "Cursor"), others have no key at all
("MILVUS", "LANGFUSE"). getProviderModels looked the key up, got undefined,
matched no models, and the model field silently degraded from a dropdown into
a free-text input with no candidates
Both call sites now go through resolveLitellmProviderSlug, which prefers an
exact provider_map key and otherwise lowercases the value. That resolves every
provider in the catalog to the litellm_provider slug the backend declares for
it. Matching provider_map case-insensitively instead would look tempting and be
wrong: "SAGEMAKER" and "SageMaker" are two distinct providers with two distinct
slugs, and a case-insensitive lookup collapses both onto sagemaker_chat
The regression test drives the real provider_create_fields.json rather than a
fixture, so a provider added to the catalog with a mismatched key fails here
instead of reaching users as an empty dropdown
Co-authored-by: AaronHowell <237895480@qq.com>
* chore(ui): drop explanatory comments from provider slug fix
---------
Co-authored-by: AaronHowell <237895480@qq.com>
Co-authored-by: ryan-crabbe-berri <ryan@berri.ai>
* docs(agents): ban semicolons and colons in human-facing text
Tighten the writing rule so ";" and ":" are only allowed where the sentence would be nonsensical without them, rewrite the existing semicolons in AGENTS.md to follow it, and note that docs (litellm-docs) keep their trailing "."
* docs(agents): rewrite semicolons in nested AGENTS.md files
* docs(agents): hand-maintain the dashboard Next.js warning and sync e2e CONTRIBUTING lines
* docs(agents): point at the PR template path that exists
* docs(agents): restore the generated Next.js block in the dashboard AGENTS.md
* feat(decisions): add Databricks ai_decide as a /v1/decisions provider and auto-router decider
* fix(decisions): reject Databricks endpoint names carrying URL separators and keep the provider wiring under llms/
* fix(decisions): reject dot-segment Databricks endpoint names and add Databricks to the dashboard's OSS classifier editor
* fix(ui): name every OSS classifier provider in the classifier radio copy
* feat(decisions): register the Databricks OpenJev decider as an evaluation-mode cost-map entry
* fix(cost-map): declare zero cache rates on the databricks-openjev-qwen35-4b entry
* fix(proxy): reconcile every DB deployment when a registry miss lands on an auto router
A /model/new for an auto router lands on one replica. A request routed to
that auto router on a sibling replica missed it, the model read-through
loaded only the auto router's own row, and the pre-routing hook then picked
a tier the sibling had not loaded, so the caller got a 400 "no healthy
deployments for <tier>" until the next periodic reload. An auto router miss
now runs the full add_deployment reconcile, tiers included. Found by the
audit's two-worker rig on this PR's routing cells; CircleCI's one-worker
proxy never opens the window.
* test(integration): audit cells for Databricks decisions and the Databricks auto-router classifier
Wire cells for /v1/decisions with databricks/<endpoint> deployments, routing
cells for the complexity router's databricks classifier on all three
endpoints with the Postgres rows read back, management cells for the
classifier's spend writes and for the evaluation-mode health check of a
Databricks serving endpoint.
* feat(microsoft_365_copilot): add Microsoft 365 Copilot chat provider with OAuth token exchange
* fix(ui): create a credential for credential-only auth types in Add Model
* fix(microsoft_365_copilot): accept max_tokens and list the chat model
* fix(microsoft_365_copilot): register provider model set
* fix(ui): hide other auth types' fields when credential-only types are filtered
* fix(health): do not inherit stored auth settings when a connection test brings its own api_key
* docs(health): restore connection-test inheritance docstrings
* fix(lint): suppress justified provider boundary casts
* fix(lint): satisfy M365 type-discipline gate
* fix(types): eliminate type-check gate regressions
* style: shorten suppression reasons so ruff format leaves them on one line
* refactor(types): drop shared-file type widening unrelated to the Copilot provider
* fix(health): pass caller headers as secret_fields to connection-test probes
* fix(types): type connection-test probe and token counter locally
* fix(lint): drop cast import from connection-test header getter
* chore: remove explanatory comments flagged by review
* fix(proxy): gate OAuth client credentials behind proxy admin
* fix(router): skip cooldown on caller-scoped OAuth auth failures
* fix(router): skip fallback cooldown on caller-scoped OAuth auth failures
* fix(router): type caller-scoped cooldown lookups
* fix(ci): include Microsoft 365 Copilot tests in unit shard
* fix(proxy): keep saved endpoint when test_connection retests a deployment by id
* fix(m365): collapse doubled Graph replies
* fix(m365): preserve trailing system messages
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Moves failure analytics off the cost overview into a dedicated admin-only
Errors tab: error rate with 4xx/5xx/429/401-403 callouts, error rate over
time, failed requests per day stacked by HTTP status code with a legend,
failures by status code, and a Failures by identity table switchable between
virtual keys, teams, users and models with error rate and top status per
row, linking to Logs. Reads GET /gateway/errors/activity. The Overview keeps
its plain ok/failed counts.
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Counts failed gateway requests by HTTP status at the ASGI edge
(LiteLLM_DailyGatewayFailedRequests) and rolls failures up per virtual key,
team, user and model group with the status the logging callbacks recorded
(LiteLLM_DailyRequestErrors, fed from the spend writer, flushed like the
gateway counters, directly or through Redis with a pod lease, and on
shutdown). GET /gateway/daily/activity gains by_status_code; the new
admin-only GET /gateway/errors/activity returns per-day totals with
per-status counts, failures by status code, and keys, teams, users and
model groups ranked by failures with their top status. Dashboard schema.d.ts
regenerated.
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(teams): let proxy admins grant team admins the right to raise their team budget
Adds a raise_max_budget entry to team_admin_editable_team_fields. With max_budget alone a team admin can still only keep or lower the team budget. Adding raise_max_budget lets them raise it, capped by the organization's max_budget when the team belongs to one, while removing the budget stays with proxy admins. The entry is rejected without max_budget, /team/info reports may_raise_max_budget to the caller, and the Admin UI nests the new checkbox under Max Budget with a tooltip and shows the team admin a hint under the budget field.
* fix(ui): shorten the raise_max_budget tooltip and link it to the docs
* fix(teams): pin the organization in the guarded team budget write
A granted raise is checked against the team's organization at read time, so the write now also requires organization_id to be unchanged. A concurrent move into a budgeted org returns 409 instead of landing an uncapped raise. Also types the new test helpers.
* test(teams): type the team row store methods and the touched race test arguments
* test(teams): annotate the last fixture and parametrize arguments in the raise_max_budget tests
* fix(ui): show team alias in key edit Team dropdown and allow clearing it
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): show alias-less teams by ID once in the key edit Team dropdown
* fix(ui): keep the key organization when its team is cleared
* fix(ui): show a dash for teams without an alias, keeping the ID underneath
---------
Co-authored-by: yuneng <yuneng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): show the public JWKS for LiteLLM-signed Anthropic credentials
The edit modal of a saved Anthropic credential whose identity source is the
LiteLLM internal issuer now fetches GET /credentials/{name}/jwks and shows the
document with a copy button, so an admin can register it in the Claude Console
without curl. Picking the internal issuer before the credential is saved shows
a hint to reopen it once saved
* fix(ui): wait for a fresh JWKS before offering one cached from an earlier open
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(ui): title usage overview chart "Daily usage" and label the model table "Top models"
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): clarify usage chart subtitle as top 8 models plus Other
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Krrish Dholakia <krrishdholakia@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: wait on conditions instead of wall-clock in guardrail parallelism and scope-option tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: annotate the guardrail barrier locals as Final
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): configure OpenAI workload identity federation from the LLM Credentials and Add Model forms
* fix(ui): let a federated OpenAI credential omit the service account when the proxy env provides it
* fix(ui): clear the federation API Base error when the admin switches the credential to an API key
* fix(ui): let ModelSelect deselect selections that are no longer offered
A selected model that is no longer served (removed from config, deleted,
or outside the org ceiling) never appeared in the dropdown, so once it sat
past the fifth chip nothing could remove it. List such selections in an
Unavailable group ahead of the offered options so they can be found and
unchecked.
* fix(ui): skip the Unavailable group when no live models were loaded
A failed or empty model list made every selection look unavailable. Only
flag selections as unavailable when there is a live model list to compare
against, and cover ordering with several unavailable selections.
* fix(ui): only suppress the Unavailable group when the model list failed to load
Keying the guard on the context-filtered list hid the group when an
organization ceiling excluded every selection, which brought the dead end
back for team forms. Key it on the proxy model list having loaded.
* fix(ui): keep other selections when removing one beside a special option
Removing an unavailable model while a special option stayed selected
collapsed the whole selection to the special option, dropping the other
saved values. Collapse only when a special option is newly picked. Also
skip the Unavailable group while an org team's model ceiling is unknown,
since the offered list is empty for that reason alone.
* feat(credentials): add credential_alias and make credential_name immutable on PATCH
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(ui): query credential rows by role to stay under the no-node-access budget
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): expect credential_alias in load_credential_list dump
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(ui): drive the credential SearchSelect by placeholder and option roles
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): pass the decrypted CredentialItem to update_db_credential during master key rotation
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): trim credential_alias in CredentialModal so whitespace-only input clears the alias
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(credentials): replace credential_alias with display_name and reject edits to config credentials
credential_name stays the immutable reference key. display_name is a nullable, trimmed, max 255
character label set on POST or PATCH (omit keeps it, null clears it, blank is a 400). Reads return
display_name plus an in-memory source tag (db or config), and PATCH or DELETE on a config-defined
credential answers 400 instead of a misleading 404
* feat(ui): show credential display names everywhere and lock config credentials
The credentials table shows the display name with the credential name beneath it, badges config
credentials and disables their edit and delete actions. The edit modal adds an editable display
name next to the read-only credential name and only sends it when it changed. Every credential
picker and reference (add model, model info, models table, vector store form and info) shows the
label while still submitting credential_name
* feat(cli): set display names on credentials and show them in the list
* refactor(credentials): keep the new credential types inside the lint gate budgets
* refactor(cli): print credential command JSON through one helper
* fix(credentials): treat an empty credential_name on PATCH as omitted and pin the 404 for vanished rows
A blank credential_name never renamed anything, so PATCH accepts it again instead of answering 400. New tests pin a trimmed display_name on PATCH, the repository carrying display_name, and a 404 (not the config-owned 400) for a DB credential another worker already deleted. The hydration helper no longer copies display_name, since none of its callers read it, and the CLI update passes display_name straight through because the exactly-one check already makes it None when clearing.
* fix(ui): keep a model's credential when its picker text is emptied and skip the admin-only list for other roles
Clearing the search text in the model edit form's credential picker used to submit null, silently detaching the model's credential on save. None is now the only way to clear it. The models table fetches /credentials only for proxy admins, since everyone else got a 403 on each page load and falls back to the raw name anyway. Also fixes a type error in the re-use credential dialog and adds tests for the display name surviving a provider switch, the 255 character limit, the request payload, and the vector store None choice.
* fix(client): percent-encode the credential name when updating its display name
A name with ? or # was cut short in the URL, so the PATCH landed on a different credential or 404'd.
* fix(credentials): answer 405 with Allow: GET for PATCH and DELETE on config credentials
A config-defined credential exists (GET returns it) but is read-only through the API, which is what 405 Method Not Allowed means. 400 described a malformed request, and 404 would claim the credential does not exist.
* fix(proxy): ignore a display_name set on config credential_list entries so non-string values cannot fail boot
---------
Co-authored-by: yuneng <yuneng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* ci: restructure lint, unit and smoke workflows
* ci: preserve source formatting in tier workflows
* ci: retire per-shard coverage flags and tighten the tier workflow guards
* test(ci): freeze the workflow startup safety models
* ci: keep the current required check names running until the ruleset moves to the tier collectors
---------
Co-authored-by: yuneng <yuneng@berri.ai>
* fix(ui): link model access group chips to the access group filter
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(ui): format access group chip link changes
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(ui): mock access group hook in affected suites
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): keep access group chips unlinked until the group lookup resolves
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): keep cached access group names when a refetch fails
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: nate <nate@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): show the Add Model picker once the model catalog loads after a provider is picked
* fix(ui): show the Add Model picker when a name typed before the catalog loaded was cleared
* feat(spend_logs): configure which metadata fields are stored in LiteLLM_SpendLogs
Adds general_settings.spend_logs_metadata_fields with mutually exclusive include and exclude lists. The filter runs on a copy of the row right before it is queued for Postgres, so daily spend rollups, budgets and callbacks still see every key.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(spend_logs): keep excluded auto-router savings keys out of published spend log metadata
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(spend_logs): cover metadata retention across endpoints, failures, cache hits, batches, runtime updates and auto-router publication
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(ui): regenerate schema.d.ts for spend_logs_metadata_fields
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(spend_logs): filter metadata at DB write so guardrail usage sees full rows
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(spend_logs): poll guardrail daily metrics instead of reading once
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(spend_logs): drop timeout comment
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(spend_logs): read spend_logs_metadata_fields through typed general settings
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: mrinal <mrinal@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(guardrails): run each mode only on streaming, non-streaming, or both
Add stream_scope so a rail can target streaming inference, non-streaming inference, or both per pre, during, and post mode
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(guardrails): honor stream_scope in pipelines and dashboard types
Pipeline steps skipped the stream_scope filter, direct construction ignored mixed-case maps, and schema.d.ts was stale.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(guardrails): skip unmatched stream_scope steps instead of allowing
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(ui): keep stored stream_scope keys for modes not on screen
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(ci): format stream_scope helpers and update fork MCP unit tests
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(mcp): keep hang cancellation tests from timing out during setup
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(guardrails): honor path-defined streaming for stream_scope
Passthrough routes like Gemini streamGenerateContent decide streaming from the URL, so stamp that onto hook data before guardrails run.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(rust): copy AnthropicModelCapabilities instead of cloning
Clippy treats clone-on-Copy as an error, which failed rust-lint on the messages request tests.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(guardrails): trust only server stream classification
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(guardrails): keep streaming marker through deepcopy
scan_raw_request snapshots copy each field, so a plain object() marker would lose identity and skip streaming-only rails.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(cost-map): drop duplicate perceptron-mk1.5 row
Two main cost-map PRs both added the OpenRouter model, so the merge left a second key that CI rejects.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(rust): expect native transcription 429 as RustUpstreamError
HTTP status errors from native routes map through route_error_to_pyerr, so the wheel SIGINT child was dying on an outdated RuntimeError check and never reached the hang probe.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(tests): follow Google Interactions OpenAPI without hardcoded names
The live spec dropped CreateModelInteractionParams and renamed the item path to {interactionsId}. Misc CI failed because the compliance tests still looked those names up as literals.
* test(guardrails): reproduce stream_scope bedrock and passthrough field gaps
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(guardrails): cover invalid stored stream_scope reads
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(guardrails): classify bedrock stream actions and keep caller is_streaming_request
Co-authored-by: Shivi Jain <mobile.350017@gmail.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(guardrails): make stream_scope_allows public and drop mutable builds
Co-authored-by: Shivi Jain <mobile.350017@gmail.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(guardrails): classify pass-through and Bedrock stream scopes
Co-authored-by: Shivi Jain <mobile.350017@gmail.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(guardrails): harden stream classification and validation
Co-authored-by: Shivi Jain <mobile.350017@gmail.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(guardrails): add stream scope integration audit
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): avoid mutating passthrough custom body
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(guardrails): run stream scope audit without enterprise license
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(guardrails): keep stored scope restart cell on one worker
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(guardrails): scope stream scope audit sink assertions to the rail under test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(guardrails): assert logging_only scope absence behind an ordered barrier rail
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(guardrails): assert one logging_only scan per phase after the barrier rail
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(guardrails): set request scope on pass-through stream fixtures
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(guardrails): tolerate invalid YAML stream_scope in v1 guardrails list
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore: merge main into litellm_guardrail_stream_scope_fixes
Update pass-through pre-call test callbacks for main's endpoint_type argument
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(guardrails): add request paths to pass-through fixtures
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(guardrails): cover websocket pass-through stream scope and LIT-9050 outage spend rows
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(lint): remove unused type discipline suppressions
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(guardrails): script the vertex live upstream in the websocket stream scope test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(guardrails): keep stream marker json-serializable and restore pass-through helper names
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(lint): allow required Bedrock action re-export
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(guardrails): strip the stream marker from pass-through payloads
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(guardrails): drop stream marker by value in scans and snapshots, plain-tuple stream scope state
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): render tag-scoped guardrail modes read-only in the custom code editor
A guardrail whose litellm_params.mode is the tag-scoped dict {tags, default}
crashed the Custom Code editor on open: normalizeMode wrapped the dict into
the mode array and StreamScopeFields rendered it as a React child (error #31,
whole dashboard unmounted). Treat a non-string non-array mode as no editable
modes, show formatGuardrailMode(mode) in a disabled input (read-only, matching
the guardrail info view), and keep mode/stream_scope out of the update payload
for such guardrails.
* fix(ui): resolve merge fallout in guardrails components
Deduplicate toModeArray import after the merge, and move the read-only
guardrail details block back into GuardrailReadOnlyDetails (now rendering
the shared mode/logging-only rows plus the stream-scope detail) so
guardrail_info.tsx stays under the 800-line lint budget. Guardrails UI
suite: 298 passed.
* fix(ui): drop duplicate toModeArray import reintroduced by merge
* chore(pass-through): document the deliberate in-place marker strip as mutable-ok
The clear/update on _parsed_body is load-bearing: rebinding to a fresh
mapping instead breaks 75 pass-through tests because the marker-free body
must propagate through the caller's request dict so downstream guardrail
scans and snapshots never observe the server streaming marker.
* fix(guardrails): move stream_scope after timeout in CustomGuardrail init
Inserting stream_scope before the existing timeout parameter shifted the
positional slot of timeout, so positional callers constructed with their
timeout bound to stream_scope (ValueError) and timeout silently None.
Restores the base parameter order; keyword callers are unaffected.
* fix(guardrails): typing pass for the lint gates
stream_scope leaves the declared constructor parameters (restoring the
base positional surface; keyword construction unchanged), unknown config
values crossing the new stream-scope code paths get typed locals or
cast-ok boundaries, and the passthrough payload literals are annotated.
All three lint gates pass against current main; the guardrail suites are
unchanged (236+5019 passing; one known anyio-driver failure pre-existing
on base).
* style: sort cast imports for the ruff gate
* chore(pass-through): drop unused BEDROCK_STREAMING_ACTIONS re-export
The streaming check now uses is_bedrock_streaming_endpoint; nothing in
the repo imports the name from this module.
---------
Co-authored-by: Shivi Jain <mobile.350017@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: gabriele <gabriele@berri.ai>
* feat(voyage): rebrand to VoyageAI by MongoDB, refresh model list, support both contextual input shapes
* fix(voyage): send auto-chunking params for flat contextual inputs
A flat list[str] or bare str for the contextualized embeddings endpoint is
only valid as documents with enable_auto_chunking=True and input_type=document,
or as queries with input_type=query. Default those params for non-query flat
inputs so the request matches the live API contract, while letting caller-set
values win. Nested list[list[str]] still passes through unchanged.
Tests now assert the auto-chunking params instead of only echoing inputs.
* feat(voyage): route MongoDB-issued keys to ai.mongodb.com
Mirror voyageai.util.get_default_base_url from the official SDK: a key with
the `al-` prefix is issued by MongoDB and is only valid on ai.mongodb.com,
every other key on api.voyageai.com. The choice now lives in one shared
helper that the embedding, contextual, multimodal, and rerank configs all
use for both the default base URL and the Authorization header, so the host
and the key always agree.
Also drops the voyage-4-nano and voyage-multilingual-2 model map entries:
voyage-4-nano is not served on the Voyage API and voyage-multilingual-2 is
an older model, so neither belongs in this change.
* fix(voyage): restore MongoDB routing on contextual endpoint and re-add voyage-4-nano after upstream merge
The upstream merge replaced the contextual config's get_complete_url with a
Voyage-only host and dropped voyage-4-nano from the model map. Reapply the
al- key routing via get_default_base_url and add voyage-4-nano (open-weight,
per docs.voyageai.com) so the branch keeps its task changes on top of upstream.
* fix(voyage): drop voyage-4-nano and the contextual tests upstream already covers
voyage-4-nano is not served on the Voyage API, so it does not belong in the
model map. The contextual input tests duplicate
tests/test_litellm/llms/voyage/test_voyage_contextual_embedding.py, which
landed upstream with the auto-chunking fix this branch was carrying.
* test(voyage): drop sys.path.insert from the voyage common utils test
* fix(voyage): route rerank on the key it authenticates with
get_complete_url picked the rerank host from the environment while the auth
header carried the request key, so an explicit MongoDB-issued key was posted
to api.voyageai.com. validate_environment now hands the resolved key to the
config instance and get_complete_url reads it back, so the host and the
credential always come from one key. The config is built per request, so
nothing carries over between them.
* test: allow ultrafast pricing keys in model prices schema
* test: drop ultrafast schema keys now added upstream
* test(integration): prove voyage key-based host routing on the wire
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(fireworks_ai): forward the LiteLLM user id as user behind fireworks_forward_user_id
* fix(fireworks_ai): keep the forwarded user id when the caller sends extra_body.user
The Add Model selector read the merged runtime cost map, so deployment
ids and provider-prefixed backend keys registered for proxy deployments
showed up as duplicate model choices. The public cost map endpoint now
accepts catalog_only=true to return the catalog as loaded, and the Add
Model panel requests that view. The default response is unchanged.
Resolves LIT-9263
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(lens): reject feedback writes from callers with no team or key
* test(lens): teamless callers cannot overwrite feedback
* test(lens): expect lens_feedback retention during schema setup
* refactor(lens): flatten feedback summaries without a stacked comprehension
* chore(lens-ui): drop routine comment on the feedback panel
* chore(lens-ui): drop routine comment on the feedback view
* chore(lens-ui): drop routine comment on the low feedback check
* chore(lens-ui): drop routine comment on the post stub
* feat(mcp): support upstream OAuth client metadata identities
* fix(mcp): load client metadata on cold OAuth requests
* fix(mcp): discover client metadata with configured OAuth endpoints
* fix(mcp): preserve configured endpoints when discovery fails
* fix(mcp): retain CIMD identities across refresh and catalog reloads
* fix(mcp): reuse saved CIMD identity for token endpoint refresh
* fix(mcp): keep dynamic client registration ahead of CIMD unless the deployment opts in
A provider that advertises both a registration endpoint and client ID
metadata documents now gets the registration flow the gateway used
before, so a gateway on a private network keeps working against it;
the metadata document identity applies when the provider offers no
registration, or when general_settings.mcp_prefer_client_id_metadata_document
is true. The saved CIMD refresh identity compares tokens as bytes so a
non-ASCII refresh token cannot crash the token route.
* chore(ui): regenerate dashboard API types for the new MCP general setting
---------
Co-authored-by: Joshua Valluru <326636767+joshua-berri@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(ui): add TypeSafe and Strands Decider to the Add Model provider list
* test(public_endpoints): type the provider fields helper against the route's model
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(decisions): serve System One format at /v1/systemone and OpenAI format at /v1/decisions
The System One request format moves to /v1/systemone and /systemone. /v1/decisions and
/decisions now accept the OpenAI Decisions API format, translate it into a System One
request, route it through the same pipeline, and translate the answers back.
* fix(decisions): import assert_never from typing_extensions for Python 3.10
* fix(decisions): cap OpenAI-format questions at the System One limit
/v1/decisions accepted up to 200 questions, but the System One request it translates to takes at most 128, so 129 to 200 questions failed with an internal validation error. Both now share MAX_DECISION_QUESTIONS.
* fix(decisions): return 400 for bodies that are not JSON
* test(decisions): send System One bodies to /v1/systemone in integration tests
* fix(decisions): return guardrail_information on OpenAI-format decisions when requested
* refactor(decisions): validate proxy request data before reading guardrail settings
* feat(lens): add lens_feedback ClickHouse table for human trace scores
ReplacingMergeTree(UpdatedAt, IsDeleted) keyed like agent_traces_by_key so
feedback joins traces in sort order, with a bloom filter on TraceId, a score
CHECK, and the same retention as traces
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* feat(lens): add scoped ClickHouse reads for trace feedback
feedback_target resolves a visible trace's team, key and trace_ref before a
write; feedback lists the latest live entry per author; feedback_summary
returns count, average and lowest score for a batch of traces
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(lens): pin the Claude session to trace id hash shared with feedback
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* feat(lens): expose feedback queries to the Python trace bridge
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* feat(lens): add typed feedback models and ClickHouse feedback store
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* feat(lens): add /lens/feedback API for 0-10 human scores on traces
PUT saves or replaces the caller's score and comment, GET lists every
entry, DELETE removes the caller's, POST /summary feeds the trace list.
Callers can target a trace_id or a Claude Code session_id
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* feat(lens-ui): add typed feedback calls to the traces API
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* feat(lens-ui): flag rated traces and filter the list by feedback
A Feedback column shows each run's average score and rating count, marked
red when any score is 4 or below, and a filter narrows to rated or
low-score runs through the URL
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* feat(lens-ui): show and edit human feedback inside a trace
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* feat(lens): let app keys post end-user feedback on their own traces
Any key can now PUT/DELETE feedback on traces its team or key sent, naming
the end user in an optional user field. Reading feedback stays with Lens
admins
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* feat(lens-ui): show end-user feedback first in a run and flag low-scored rows
Replace the edit popover and feedback filter with a read-only view: the
list shows each run's score in a fixed-width badge and tints the row red
when a user scored it 4 or below, and opening a run shows what users said
with their score right under the header
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* feat(lens): add who started the run to RunSource
* feat(lens): carry agent.source.user on trace span rows
* feat(lens): resolve the run source user from the span row
* feat(lens): read agent.source.user in trace_spans query
* feat(lens): read agent.source.user in trace_page_spans query
* feat(lens): read agent.source.user in trace_span_batch query
* feat(lens): read agent.source.user in trace_list_span_batch query
* feat(lens): decode the source user from clickhouse span rows
* test(lens): add source user to trace cache test rows
* test(lens): add source user to trace cache read fixtures
* test(lens): add source user to trace cache snapshot fixtures
* test(lens): add source user to capture fixture rows
* test(lens): round-trip source user on span row contract
* test(lens): cover run source user resolution
* chore(lens): regenerate python trace types with source user
* chore(lens): regenerate trace json schema with source user
* chore(lens): regenerate trace page json schema with source user
* chore(ui): add source user to api types
* feat(lens): add run user chip and slack thread chip
* feat(lens): put agent, user and thread on the first row of the run header
* test(lens): cover the run header identity row and compact totals
* feat(lens): start demo runs from a slack thread
* fix(lens-ui): open a trace on its top-level step instead of the first failed one
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(lens-ui): pin that failed runs open on the root step
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(lens-ui): drop the unused firstErrorSpan helper
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(lens-ui): stop testing the removed firstErrorSpan helper
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(lens-ui): stop showing the model name as a run's input when none was recorded
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(lens-ui): cover the trace list row with no recorded input
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
