* fix(databricks): call the ai_decide AI Function route for decisions and the auto-router
Databricks serves ai_decide at POST <host>/api/2.0/ai-functions/ai-decide, not
on a serving endpoint's /invocations route. The decisions provider and the
auto-router classifier now accept only the ai_decide model, post to that route
without a model field, unwrap the response envelope, and map a noul answer's
probability to the System One noul field.
* test(databricks): bound the megabyte model name test by scaling, not wall time
* test(databricks): assert the whole OpenAI-format decision and floor the liveliness bound
* test(decisions): move the Databricks OpenAI-format wire row to the ai_decide route
* fix(databricks): keep serving endpoint decisions beside the ai_decide route
* test(databricks): assert the whole classifier request body in the test routing case
* test(databricks): correct System One spend call type
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): restyle Decisions playground after OpenRouter's model playground
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): replace the Decisions JSON editor with a read-only cURL tab
The form is the only way to edit the request. The cURL tab shows the exact curl to the proxy with a $LITELLM_API_KEY placeholder and a copy button, so nothing can be typed into a broken payload
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): shell-quote the proxy URL in the Decisions cURL preview
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): send the Decisions cURL under the configured auth header and preview the validated payload
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(ui): name the cURL builder inputs to stay inside the lint budget
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): keep the Decisions cURL key placeholder shell-expandable
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): match the Decisions cURL URL to the send path and block copying an invalid payload
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(ui): share the Decisions error count helper and cover the cURL preview
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): load the Home page What's new launches from /public/whats_new at runtime
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): show the three newest What's new launches
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* build: ship whats_new.json in the wheel
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(decisions): rename the evaluation model mode to decisions
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(decisions): canonicalize the health check mode without rebinding
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(decisions): replace canonical_model_mode with is_decisions_model_mode
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(decisions): preserve evaluation mode compatibility
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore: drop the rebuilt dashboard bundle from this PR
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(dashboard): format decision mode helper
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): add a Home page as the dashboard root with usage summary, recent launches, blog updates and a model leaderboard
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): fall back to spend-log request counts on Home when the gateway has recorded none
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(ui): cover the Home page totals, launches, leaderboard and updates feed
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): render Home Updates as rows and return up to 6 blog posts from /public/litellm_blog_posts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): filter Home Updates by the new-model and api-change blog tags
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(blog): import NotRequired from typing_extensions for py3.10 and test the endpoint passes every fetched post through
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* revert(ui): drop the Updates filter chips, keep Updates as a plain row list
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(ui): restore generated schema.d.ts to main
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): label the Home key button Create New Virtual Key
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(ui): drop the What's new sublabel on Home
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): show an error with retry when Home usage fails to load and move the Home test to the integration tier
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(dd_span_tagger): emit litellm_user_email span tag for JWT-authenticated requests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit b4f9e14a44)
* refactor(dd_span_tagger): use dotted litellm.user_email tag for consistency
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit 819faa01c3)
* feat(model_hub): surface model_info.description in model group info and Model Hub UI
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit b224b15b9d)
* refactor(router): aggregate model group description without in-place mutation
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit 342b93fe0a)
* test(dd_span_tagger): assert the user_email tag on the active span instead of on the patched setter
The two ported tests only inspected the mock that replaced the tag setter, which the test quality gate counts as two new mock-echo tests over main. They now read the tags that land on the active span
* refactor(router): read the model group description through the typed model_info reader
The ported lookup read model_info off the untyped deployment dict, which added three basedpyright errors over main. It now goes through the existing validated model_info reader
---------
Co-authored-by: milan <milan@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: ryan-crabbe-berri <ryan@berri.ai>
* fix(guardrails): scan or refuse attachments in the bedrock guardrail
* fix(guardrails): cover native pre_call and computer screenshots in the bedrock attachment scan
* fix(guardrails): classify every attachment shape the bedrock guardrail can receive
* test(guardrails): type the attachment-scanning test guardrail
* fix(guardrails): keep base behavior for Bedrock subclasses and malformed messages
- skip the attachment scan when a subclass overrides apply_guardrail
- ignore non-string message role/type instead of raising
- treat a null Converse document/video/audio block as absent
- keep the base request-body debug log for text-only calls
* fix(guardrails): let text documents, unpadded base64 images and request-override subclasses through as before
* fix(guardrails): let Converse text documents through and refuse documents that carry images
* fix(guardrails): scan images inside content documents instead of refusing them
* fix(guardrails): scan text document content with the Bedrock guardrail
* fix(guardrails): scan document title, context and nested documents
* Expand nested content documents without recursion
* Keep attachment scan within the type-discipline budget
* Take the image format and pixel size from the image bytes
* chore(guardrails): satisfy type-discipline gate after merging main
* chore(guardrails): drop docstrings added by the attachment scan
* fix(guardrails): scan Converse text documents and guardContent images
A Converse document whose source is text (or a list of text parts) now
goes through the Bedrock text scan instead of being refused as
unscannable, so the pass-through request that worked before this PR
keeps working. A Converse guardContent image is now scanned like any
other image. Byte and S3 document sources stay unscannable.
Adds the integration test that drives every scanned endpoint through a
real proxy against a scripted Bedrock ApplyGuardrail stub, and retypes
the touched hook signatures for the type gates.
* test(guardrails): cover Bedrock checks mode in the attachment integration test
Adds a checks-mode proxy to the rig: images are refused without calling Bedrock on every route, and a Converse text document is sent to InvokeGuardrailChecks.
* fix(guardrails): scan a Converse document's name and context whatever its source
A Converse PDF or S3 document let through by skip_unscannable_attachments now still has its name and context scanned as text, the same as the Anthropic document path.
---------
Co-authored-by: yucheng-berri <293926326+yucheng-berri@users.noreply.github.com>
* fix(credentials): keep credential labels and edits consistent across UI, API, and replicas
The credential modal flattened stored credential values into the same form object as the
credential's own name, label, and provider, so a stored value keyed credential_name or
display_name replaced the real one and the edit went to the wrong credential or relabeled it.
The modal now submits the top-level fields and credential_values separately
PATCH /credentials now reads the row it merges onto from the writer, so an edit inside the read
replica's lag window no longer writes back an older label or key
POST /credentials answers 409 for a name defined in config instead of creating a DB copy that
shadows it
display_name rules live on the server only. It rejects control, formatting, and line separator
characters, and the UI stops enforcing its own length check and shows the server's error
The models table, model detail view, and model edit picker show the credential's technical name
under its label whenever the two differ
* fix(ui): show credential source in its own column like models
The config marker sat under the credential name. The credentials table now has a Source column
with a Config or DB status badge, matching the Source column on the models table
* test(credentials): unit-test label and prefill helpers and keep the full credential name on hover
Moves displayNameChange and initialFormValues out of the modal so their rules are unit tested,
keeps only the wiring cases in the modal's integration test, makes the label acceptance test
write through the real credentials repository, and adds the full credential name as a hover
title where the models pages truncate it
* feat(guardrails): add decision_model guardrail for decisions-API models
Adds a new guardrail provider that scores request and response text with a
decisions-API model (e.g. Jev) against named predicate checks. Each check
carries its own threshold and action (block or log); a flagged block check
rejects the request with a 400 naming the flagged checks, and call failures
honor unreachable_fallback (fail_open or fail_closed).
Ships four preset checks for the Admin UI (prompt injection, jailbreak,
system prompt extraction, data exfiltration) served from
/guardrails/ui/add_guardrail_settings, plus a Decision Model step in the
add-guardrail form filtered to evaluation-mode models.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(guardrails): fail closed on verdict errors and drop comments in decision_model
* feat(guardrails): cap decision_model input with head and tail elision
* fix(guardrails): list only block checks in decision_model 400s and tighten inputs
* feat(guardrails): honor timeout, cap checks at MAX_DECISION_QUESTIONS, and reject duplicate check names
* fix(guardrails): decorate decision_model apply_guardrail with log_guardrail_information
* fix(guardrails): suppress untyped-decorator pyright error on log_guardrail_information import
* feat(guardrails): screen each text in overlapping chunks with a shared decisions-call semaphore
Replace head/tail elision with per-text screening: each non-empty text is
sent to the decision model on its own, split into overlapping chunks when
it exceeds max_input_chars, and all calls run under a per-event-loop
semaphore shared across requests (max_concurrent_decision_calls,
default 8). Check verdicts aggregate the max probability across calls; a
missing predicate answer on a Block check blocks under fail_closed and
only warns under fail_open, while log-only checks are never flagged for
it. The 400 detail now lists flagged check names only. The provider
combobox also matches Decision Model on jev, typesafe, prompt injection,
jailbreak, and decision
* fix(ui): align decision model custom check rows with preset rows
* fix(ui): move decision model custom check remove button to the row end
* style(ui): run prettier on decision model check files
* fix(ui): inline decision model evaluation-mode filter after lens module removal
* fix(guardrails): keep decision model detections when a chunk call fails
* fix(ui): require an enabled decision model check before create
* fix(guardrails): screen decision model texts sequentially like presidio
* feat(guardrails): replace decision model presets with provider picker and per-question test
* fix(ui): render decision model question test as a full-width row
* feat(ui): add run-history test section to decision model form
* fix(ui): prevent threshold overlap and show test shortcut
* feat(guardrails): default decision model threshold to 0.7
* test(ui): expect decision model default threshold 0.7 in characterization payload
* feat(guardrails): screen tool call arguments with the decision model
* refactor(guardrails): freeze the decision model config model
* feat(guardrails): list only evaluation-mode deployments in the decision model picker
* feat(guardrails): hide the decision model questions table until a question exists
* style(ui): format DecisionModelFields
* style(ui): format decisionModelQuestion tests
* refactor(guardrails): type the tool call function extraction
* refactor(guardrails): build tool call texts without a mutable accumulator
* refactor(ui): share isDecisionMode for the decision model picker
* fix(guardrails): screen prior Responses function call arguments in the decision model guardrail
The Responses input handler leaves function_call arguments out of texts and tool_calls, so a prompt injection in a prior call's arguments passed unscreened. Assistant tool calls from structured messages are now screened too, deduplicated against the request's tool calls
* test(integration): send instructions with the decision model guardrail check
Checks require instructions, so creating the guardrail returned 422 and the test never sent a chat request
* fix(ui): score the decision model test preview the way the guardrail does
An unanswered block question now shows Block, disabled questions no longer count, and a run is hidden once the model or a question's text changes since it was scored
* fix(ui): clear the decision model error on pick and list questions under Add a question
Picking a model after a failed Create left "Select a decision model" on screen, because the form only validates on submit. The field now revalidates when it already shows an error. The Questions table and its heading appear only once a question exists, below the Add a question box
* test(ui): move the decision model form tests into an integration test file
They render the whole AddGuardrailForm, so they belong in a .integration.test.tsx file. Plain fields are now filled with fireEvent.change, and the two existing form test files go back to main apart from the modelHubCall mock the form now needs
* test(guardrails): build the decision model test params without mutating a local
* fix(guardrails): reject list checks outside decision_model and honor streaming scan settings
- LitellmParams now rejects a list `checks` for any guardrail other than
decision_model, so a Bedrock guardrail with list checks gets 422 again
instead of being stored and failing every request
- The decision model guardrail reads streaming_end_of_stream_only and
streaming_sampling_rate, so a streamed post_call response can be scored
once at end of stream instead of every 5 chunks
- max_concurrent_decision_calls is described as a per-worker limit
* fix(ui): match guardrail providers like the default picker filter
- The provider search uses the Base UI collator filter on the label and
the aliases, so searches such as presidio_pii or llm_as_a_judge match
again, as they did before the alias filter
- Model groups from /model_group/info are parsed defensively and after the
providers are populated, so a malformed response cannot empty the
provider list
* fix(ui): edit decision model questions in place and show passing test chips in green
Questions start as an Add question button. Each click adds a blank question
card with its name, text, action and threshold, editable until the guardrail
is created, mirroring the per-model budget editor. Create and Next reject a
blank or duplicate enabled question. Test run verdicts and chips use the
success tone for Pass, the error tone for Block and the warning tone for a
flagged Log only question
* fix(ui): drop the per-question enable checkbox from the decision model form
A question is either in the list or removed with its X, so every listed
question is sent and scored. The card now labels Name and Question the way
it labels Action and Threshold
* fix(ui): give the question name field the full card width
* fix(ui): show the provider logo on the selected decision provider
* fix(ui): show the full threshold slider track on decision model questions
* feat(ui): add decision model cards to the guardrail garden
Adds a Decision Model section to the guardrail garden with TypeSafe Jev,
Perplexity, Microsoft, OpenAI and Databricks cards. Each card opens the
create form scoped to that provider's decision models.
The create form drops the Decision Provider picker and lists every
decision model on the proxy with its provider logo, prompts to add a
model when none is available, and says that test runs are billed and
logged.
* fix(ui): shorten the decision model test hint to one line
* fix(ui): prefill the decision model from the garden card's provider
* chore(ui): regenerate the dashboard API types after merging main
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: yucheng-berri <293926326+yucheng-berri@users.noreply.github.com>
* feat(sdk): add litellm.systemone and trim litellm.decisions to the OpenAI shape
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* ci: run tests/unit/systemone in the core-utils unit shard
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(sdk): keep litellm.decisions(state=...) working with a DeprecationWarning
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: make tests/unit/systemone a package
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(sdk): surface the litellm.decisions(state=...) deprecation as a UserWarning
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): label the System One yes / no question type noul and explain it on hover
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): capitalize the Noul answer type label
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* Revert "feat(ui): capitalize the Noul answer type label"
This reverts commit e79a40fce2.
* test(ui): cover the noul banner tooltip and the answer type option title
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): replace Reset example with an Example preset dropdown and a Clear button on the Decisions tab
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): match Decisions presets by content and keep an omitted model out of Clear and preset picks
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* docs(proxy): describe /v1/decisions and /v1/systemone in Swagger UI
Add summaries, descriptions, inlined request body schemas with examples and typed 200 responses to the
decision routes and their aliases, so the OpenAPI document and the lazy snapshot show both request formats
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): cap _inlined schema walk depth for the recursion check
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(otel): let a team strip captured message content from its own OTel v2 destination
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(proxy): split the shared callback option conflict check and sync dashboard API types
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(otel): address review feedback on team capture_message_content
Classify capture_message_content as NotSecret in the credential slot registry, inject test exporters through register_exporter_factory instead of patching _exporter_from_spec, type the test helpers, and stop reassigning processor in build_tracer_provider
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(otel): add integration cells for team capture_message_content
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(otel): cover weave_otel and newrelic team destinations in capture_message_content cells
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(otel): stop rejecting client-sent capture_message_content at the request body boundary
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(otel): let a team capture_message_content override the global setting in both directions
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(otel): keep a team's capture_message_content on its own OTel v2 destination
The setting no longer rides the flattened team callback vars, so a sibling destination and a routed provider follow the global mode. Request bodies get the base 400 back, a failure-only entry rejects the setting, and entries for one backend must agree
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(otel): type the capture_message_content callback validation tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(otel): type the remaining capture_message_content test signatures
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(otel): mark the new capture_message_content test locals Final
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(otel): default tenant capture to callback_settings.otel and keep CaptureMessageContent formatting bare
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(otel): keep CaptureMessageContent a plain str like main
The str-to-Enum switch changed the public type: the config field's type,
repr and model_dump, a constructor that now raised on unknown values, and
unknown env values coerced to no_content. CaptureMessageContent is the plain
str subclass main ships again, the field is a str, and a
captures_span_content helper replaces the enum property. A test pins that a
published preset's capture mode stays the default for an unowned
destination, which is what main forwards.
* test(otel): drop the module-level CaptureMessageContent import the rewritten test no longer uses
* fix(lint): keep the merged branch within main's type budgets
Freeze the capture-mode settings model (LIT015) and read stored logging
entries through one typed sequence helper, so the conflicting-capture checks
add no unknown types.
* fix(otel): reject a destination capture_message_content while OTel v2 is off
The legacy OTel callbacks never read the per-destination setting, so a team
could register no_content and still export prompts and replies under the
global capture mode. Registration now refuses the var unless the proxy runs
with LITELLM_OTEL_V2=true, the same rule the Arize transport and New Relic
routing vars follow.
* test(otel): type the OTel v2 fixtures and the v2-off regression test
* fix(otel): accept only no_content and span_only on a team or key destination
OTel v2 writes captured content only to span attributes, so event_only
behaved like no_content and span_and_event like span_only on a
destination. The proxy-wide setting keeps all four values
* docs(otel): say an omitted capture setting follows the proxy's own callback for that backend
* test(otel): describe the general capture fallback in the unowned destination test
---------
Co-authored-by: mrinal <mrinal@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: yucheng-berri <yucheng@berri.ai>
Co-authored-by: yucheng-berri <293926326+yucheng-berri@users.noreply.github.com>
* feat(proxy): require trace IDs per team on LLM, MCP and agent requests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): assert trace ID authorization outcomes
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): clear trace ID type gate errors
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(proxy): format trace ID type check
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): validate trace ID metadata
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): use read-only trace metadata types
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): skip trace ID metadata validation for teams without dict metadata
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* perf(proxy): check team trace ID setting before route matching
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): simplify playground decision endpoint
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): expose both decision API shapes in playground
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): omit undiscovered decision models from examples
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): align decisions form and response views
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(guardrails): split logging_only_scope direction from input-failure policy
logging_only_scope is now direction-only (input, output, unset for both
directions). The new logging_only_continue_on_input_failure flag controls
whether a flagged or raising logging_only input scan still lets the
response be scanned. "both" stays accepted everywhere as a deprecated
alias for unset scope plus the flag set to true, normalized at callback
init while stored rows keep the written value.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(dashboard): narrow continue-on-input-failure update type and stay under guardrail_info max-lines
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(dashboard): keep custom-code guardrail controls readable with logging_only
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(dashboard): scope continue-on-input-failure to the default direction and restore verdict assertion
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(guardrails): remove the both value from logging_only_scope
logging_only_scope only shipped its both value in dev prereleases, so it is
dropped instead of kept as a deprecated alias. Leaving the scope unset and
setting logging_only_continue_on_input_failure gives the same behavior. A
stored both value is now treated like any other invalid scope: it is logged
and ignored on load, and rejected with 422 on create or update.
* fix(guardrails): tolerate an invalid logging_only_scope in the v1 guardrail list
A config.yaml guardrail whose logging_only_scope is no longer valid, such as
the removed both value, made GET /guardrails/list return 500 for every
guardrail. The v1 list now parses params with the same tolerant parser as
the v2 list, info and PATCH paths, so the bad value is logged and dropped.
* test(guardrails): drive continue-flag rejection tests through the real guardrail handler
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: yucheng-berri <293926326+yucheng-berri@users.noreply.github.com>
* feat(ui): list the decision providers from the cost map in the decision models banner
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): title-case decision providers the dashboard has no display name for
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(ui): wait for the catalog query to fail before asserting the banner fallback
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): replace Add a decision model with a Supported providers docs link in the decision models banner
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): drop the Add Model search hint from the decision models banner
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): call it the Decisions playground in the decision models banner
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): call the System One playground the Decisions playground across the dashboard
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): render the System One JSON editor with the stock One Dark theme
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): rename the Playground System One tab to Decisions and use the VS Code Dark+ JSON theme
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): keep the JSON editor header and issue list on the light surface
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(ui): drop the root block scanner test and the unused band styles
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): follow the dashboard theme in the JSON editor and drop the breakdown accents
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): round the Decisions score pill to two decimals
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): strip the Prism block backgrounds so the JSON editor theme swap stops the React style warning
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): let admins make per-model key/team rate limits a hard 429 instead of falling back
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): sync per-model rate limit API types
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): type per-model fallback setting access
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): keep the per-model hard cap inside the fallback loop and test it through the real limiter
* fix(proxy): list the per-model hard cap setting on the settings page and cover every per-model descriptor
* fix(proxy): read disable_fallbacks_on_per_model_rate_limits as a boolean value and cover the team-wide per-model cap
* test(proxy): cover per-model rate limit 429s skipping fallbacks on a live proxy
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(ui): build System One playground requests with a form
* test(ui): fill System One form fields with single change events
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(ui): announce decision model support in Models + Endpoints
* feat(ui): make decision models findable, callable, and testable from the Admin UI
Add Model provider search now matches decision model names and shows a note on where to call a decision model. The banner links to the docs and opens Add Model. The System One playground defaults to /v1/systemone with a picker of the proxy's decision models, and the model hub shows a /v1/systemone usage example for evaluation models.
* fix(ui): keep the admin signed in when a typed virtual key cannot list decision models
The System One playground looked up a typed virtual key's decision models through
the dashboard's shared request client, so an expired key signed the admin out, and
typing a key by hand sent one lookup per keystroke. The lookup now uses the same
client as Send, against the same proxy, and waits for typing to stop
* fix(ui): hide decision model Playground links from view-only sessions
A proxy admin viewer cannot open the Playground, so the Models + Endpoints
decision models banner and the AI Hub's Try it in the Playground link sent
them to an Access Denied page. Hide both for view-only sessions.
* fix(ui): send the AI Hub decision example's key in the proxy's configured header
A proxy with litellm_key_header_name set reads keys only from that header,
so the example's hardcoded Authorization header was rejected. Use the header
name the dashboard already reads from the session.
* fix(ui): stop the decision model notice from ruling out /chat/completions for dual-mode models
* fix(ui): point only users who can add models at Add Model in the decision models banner
---------
Co-authored-by: moyai-devin-berriai[bot] <336287033+moyai-devin-berriai[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(ui): resolve the provider dropdown key to the slug the backend declares
The Add Model form submits the `provider` field from /public/providers/fields
verbatim, but provider_map is keyed by hand on the frontend, so the two sides
disagree for 14 of the 124 providers the backend serves. Some differ only in
case ("MINIMAX" vs "MiniMax", "CURSOR" vs "Cursor"), others have no key at all
("MILVUS", "LANGFUSE"). getProviderModels looked the key up, got undefined,
matched no models, and the model field silently degraded from a dropdown into
a free-text input with no candidates
Both call sites now go through resolveLitellmProviderSlug, which prefers an
exact provider_map key and otherwise lowercases the value. That resolves every
provider in the catalog to the litellm_provider slug the backend declares for
it. Matching provider_map case-insensitively instead would look tempting and be
wrong: "SAGEMAKER" and "SageMaker" are two distinct providers with two distinct
slugs, and a case-insensitive lookup collapses both onto sagemaker_chat
The regression test drives the real provider_create_fields.json rather than a
fixture, so a provider added to the catalog with a mismatched key fails here
instead of reaching users as an empty dropdown
Co-authored-by: AaronHowell <237895480@qq.com>
* chore(ui): drop explanatory comments from provider slug fix
---------
Co-authored-by: AaronHowell <237895480@qq.com>
Co-authored-by: ryan-crabbe-berri <ryan@berri.ai>
* docs(agents): ban semicolons and colons in human-facing text
Tighten the writing rule so ";" and ":" are only allowed where the sentence would be nonsensical without them, rewrite the existing semicolons in AGENTS.md to follow it, and note that docs (litellm-docs) keep their trailing "."
* docs(agents): rewrite semicolons in nested AGENTS.md files
* docs(agents): hand-maintain the dashboard Next.js warning and sync e2e CONTRIBUTING lines
* docs(agents): point at the PR template path that exists
* docs(agents): restore the generated Next.js block in the dashboard AGENTS.md
* feat(decisions): add Databricks ai_decide as a /v1/decisions provider and auto-router decider
* fix(decisions): reject Databricks endpoint names carrying URL separators and keep the provider wiring under llms/
* fix(decisions): reject dot-segment Databricks endpoint names and add Databricks to the dashboard's OSS classifier editor
* fix(ui): name every OSS classifier provider in the classifier radio copy
* feat(decisions): register the Databricks OpenJev decider as an evaluation-mode cost-map entry
* fix(cost-map): declare zero cache rates on the databricks-openjev-qwen35-4b entry
* fix(proxy): reconcile every DB deployment when a registry miss lands on an auto router
A /model/new for an auto router lands on one replica. A request routed to
that auto router on a sibling replica missed it, the model read-through
loaded only the auto router's own row, and the pre-routing hook then picked
a tier the sibling had not loaded, so the caller got a 400 "no healthy
deployments for <tier>" until the next periodic reload. An auto router miss
now runs the full add_deployment reconcile, tiers included. Found by the
audit's two-worker rig on this PR's routing cells; CircleCI's one-worker
proxy never opens the window.
* test(integration): audit cells for Databricks decisions and the Databricks auto-router classifier
Wire cells for /v1/decisions with databricks/<endpoint> deployments, routing
cells for the complexity router's databricks classifier on all three
endpoints with the Postgres rows read back, management cells for the
classifier's spend writes and for the evaluation-mode health check of a
Databricks serving endpoint.
* feat(microsoft_365_copilot): add Microsoft 365 Copilot chat provider with OAuth token exchange
* fix(ui): create a credential for credential-only auth types in Add Model
* fix(microsoft_365_copilot): accept max_tokens and list the chat model
* fix(microsoft_365_copilot): register provider model set
* fix(ui): hide other auth types' fields when credential-only types are filtered
* fix(health): do not inherit stored auth settings when a connection test brings its own api_key
* docs(health): restore connection-test inheritance docstrings
* fix(lint): suppress justified provider boundary casts
* fix(lint): satisfy M365 type-discipline gate
* fix(types): eliminate type-check gate regressions
* style: shorten suppression reasons so ruff format leaves them on one line
* refactor(types): drop shared-file type widening unrelated to the Copilot provider
* fix(health): pass caller headers as secret_fields to connection-test probes
* fix(types): type connection-test probe and token counter locally
* fix(lint): drop cast import from connection-test header getter
* chore: remove explanatory comments flagged by review
* fix(proxy): gate OAuth client credentials behind proxy admin
* fix(router): skip cooldown on caller-scoped OAuth auth failures
* fix(router): skip fallback cooldown on caller-scoped OAuth auth failures
* fix(router): type caller-scoped cooldown lookups
* fix(ci): include Microsoft 365 Copilot tests in unit shard
* fix(proxy): keep saved endpoint when test_connection retests a deployment by id
* fix(m365): collapse doubled Graph replies
* fix(m365): preserve trailing system messages
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Moves failure analytics off the cost overview into a dedicated admin-only
Errors tab: error rate with 4xx/5xx/429/401-403 callouts, error rate over
time, failed requests per day stacked by HTTP status code with a legend,
failures by status code, and a Failures by identity table switchable between
virtual keys, teams, users and models with error rate and top status per
row, linking to Logs. Reads GET /gateway/errors/activity. The Overview keeps
its plain ok/failed counts.
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Counts failed gateway requests by HTTP status at the ASGI edge
(LiteLLM_DailyGatewayFailedRequests) and rolls failures up per virtual key,
team, user and model group with the status the logging callbacks recorded
(LiteLLM_DailyRequestErrors, fed from the spend writer, flushed like the
gateway counters, directly or through Redis with a pod lease, and on
shutdown). GET /gateway/daily/activity gains by_status_code; the new
admin-only GET /gateway/errors/activity returns per-day totals with
per-status counts, failures by status code, and keys, teams, users and
model groups ranked by failures with their top status. Dashboard schema.d.ts
regenerated.
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(teams): let proxy admins grant team admins the right to raise their team budget
Adds a raise_max_budget entry to team_admin_editable_team_fields. With max_budget alone a team admin can still only keep or lower the team budget. Adding raise_max_budget lets them raise it, capped by the organization's max_budget when the team belongs to one, while removing the budget stays with proxy admins. The entry is rejected without max_budget, /team/info reports may_raise_max_budget to the caller, and the Admin UI nests the new checkbox under Max Budget with a tooltip and shows the team admin a hint under the budget field.
* fix(ui): shorten the raise_max_budget tooltip and link it to the docs
* fix(teams): pin the organization in the guarded team budget write
A granted raise is checked against the team's organization at read time, so the write now also requires organization_id to be unchanged. A concurrent move into a budgeted org returns 409 instead of landing an uncapped raise. Also types the new test helpers.
* test(teams): type the team row store methods and the touched race test arguments
* test(teams): annotate the last fixture and parametrize arguments in the raise_max_budget tests
* fix(ui): show team alias in key edit Team dropdown and allow clearing it
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): show alias-less teams by ID once in the key edit Team dropdown
* fix(ui): keep the key organization when its team is cleared
* fix(ui): show a dash for teams without an alias, keeping the ID underneath
---------
Co-authored-by: yuneng <yuneng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): show the public JWKS for LiteLLM-signed Anthropic credentials
The edit modal of a saved Anthropic credential whose identity source is the
LiteLLM internal issuer now fetches GET /credentials/{name}/jwks and shows the
document with a copy button, so an admin can register it in the Claude Console
without curl. Picking the internal issuer before the credential is saved shows
a hint to reopen it once saved
* fix(ui): wait for a fresh JWKS before offering one cached from an earlier open
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(ui): title usage overview chart "Daily usage" and label the model table "Top models"
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): clarify usage chart subtitle as top 8 models plus Other
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Krrish Dholakia <krrishdholakia@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: wait on conditions instead of wall-clock in guardrail parallelism and scope-option tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: annotate the guardrail barrier locals as Final
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): configure OpenAI workload identity federation from the LLM Credentials and Add Model forms
* fix(ui): let a federated OpenAI credential omit the service account when the proxy env provides it
* fix(ui): clear the federation API Base error when the admin switches the credential to an API key
* fix(ui): let ModelSelect deselect selections that are no longer offered
A selected model that is no longer served (removed from config, deleted,
or outside the org ceiling) never appeared in the dropdown, so once it sat
past the fifth chip nothing could remove it. List such selections in an
Unavailable group ahead of the offered options so they can be found and
unchecked.
* fix(ui): skip the Unavailable group when no live models were loaded
A failed or empty model list made every selection look unavailable. Only
flag selections as unavailable when there is a live model list to compare
against, and cover ordering with several unavailable selections.
* fix(ui): only suppress the Unavailable group when the model list failed to load
Keying the guard on the context-filtered list hid the group when an
organization ceiling excluded every selection, which brought the dead end
back for team forms. Key it on the proxy model list having loaded.
* fix(ui): keep other selections when removing one beside a special option
Removing an unavailable model while a special option stayed selected
collapsed the whole selection to the special option, dropping the other
saved values. Collapse only when a special option is newly picked. Also
skip the Unavailable group while an org team's model ceiling is unknown,
since the offered list is empty for that reason alone.
* feat(credentials): add credential_alias and make credential_name immutable on PATCH
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(ui): query credential rows by role to stay under the no-node-access budget
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): expect credential_alias in load_credential_list dump
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(ui): drive the credential SearchSelect by placeholder and option roles
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): pass the decrypted CredentialItem to update_db_credential during master key rotation
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): trim credential_alias in CredentialModal so whitespace-only input clears the alias
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(credentials): replace credential_alias with display_name and reject edits to config credentials
credential_name stays the immutable reference key. display_name is a nullable, trimmed, max 255
character label set on POST or PATCH (omit keeps it, null clears it, blank is a 400). Reads return
display_name plus an in-memory source tag (db or config), and PATCH or DELETE on a config-defined
credential answers 400 instead of a misleading 404
* feat(ui): show credential display names everywhere and lock config credentials
The credentials table shows the display name with the credential name beneath it, badges config
credentials and disables their edit and delete actions. The edit modal adds an editable display
name next to the read-only credential name and only sends it when it changed. Every credential
picker and reference (add model, model info, models table, vector store form and info) shows the
label while still submitting credential_name
* feat(cli): set display names on credentials and show them in the list
* refactor(credentials): keep the new credential types inside the lint gate budgets
* refactor(cli): print credential command JSON through one helper
* fix(credentials): treat an empty credential_name on PATCH as omitted and pin the 404 for vanished rows
A blank credential_name never renamed anything, so PATCH accepts it again instead of answering 400. New tests pin a trimmed display_name on PATCH, the repository carrying display_name, and a 404 (not the config-owned 400) for a DB credential another worker already deleted. The hydration helper no longer copies display_name, since none of its callers read it, and the CLI update passes display_name straight through because the exactly-one check already makes it None when clearing.
* fix(ui): keep a model's credential when its picker text is emptied and skip the admin-only list for other roles
Clearing the search text in the model edit form's credential picker used to submit null, silently detaching the model's credential on save. None is now the only way to clear it. The models table fetches /credentials only for proxy admins, since everyone else got a 403 on each page load and falls back to the raw name anyway. Also fixes a type error in the re-use credential dialog and adds tests for the display name surviving a provider switch, the 255 character limit, the request payload, and the vector store None choice.
* fix(client): percent-encode the credential name when updating its display name
A name with ? or # was cut short in the URL, so the PATCH landed on a different credential or 404'd.
* fix(credentials): answer 405 with Allow: GET for PATCH and DELETE on config credentials
A config-defined credential exists (GET returns it) but is read-only through the API, which is what 405 Method Not Allowed means. 400 described a malformed request, and 404 would claim the credential does not exist.
* fix(proxy): ignore a display_name set on config credential_list entries so non-string values cannot fail boot
---------
Co-authored-by: yuneng <yuneng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* ci: restructure lint, unit and smoke workflows
* ci: preserve source formatting in tier workflows
* ci: retire per-shard coverage flags and tighten the tier workflow guards
* test(ci): freeze the workflow startup safety models
* ci: keep the current required check names running until the ruleset moves to the tier collectors
---------
Co-authored-by: yuneng <yuneng@berri.ai>
