* feat(telemetry): count Admin UI page views and tab clicks through the proxy
* fix(ui): share one telemetry settings save across both tabs so a second toggle waits for the first
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(telemetry): let every dashboard role reach the UI event routes
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): fail with the denial detail when a dashboard role cannot reach UI event routes
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(telemetry): document POST /telemetry/ui_events with a description and a 200 response schema
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): keep the test-quality reason on the line pytest's checker reads
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): pick the telemetry destination on the Telemetry page and warn when none is chosen
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(ui): send the destination along with groups in the telemetry settings hook test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): add a top-level telemetry switch and gate turning telemetry on behind an enterprise modal
The page can only turn telemetry off. Turning it on from the switch, a group
or a first destination opens a modal: enterprise proxies are told to contact
their LiteLLM enterprise representative, others that telemetry is an
enterprise-only feature. Group switches and the destination stay visible but
disabled while telemetry is off.
---------
Co-authored-by: nate <nate@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(telemetry): keep reports in a local table with a persistent instance id and an admin export route
* feat(telemetry): key deployment hashes from a random secret stored in the proxy DB
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(telemetry): retry a locally stored window under the same id so it is never counted twice
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* docs(telemetry): state the overlap and dedupe contract for paging stored reports
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(ui): regenerate dashboard API types for the telemetry reports paging note
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(telemetry): require an explicit LITELLM_TELEMETRY_DESTINATION or saved destination before recording
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(ui): regenerate dashboard API types for the explicit telemetry destination
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(telemetry): register the attempt logger and create its hash secret only once a sink is built
A proxy with a database that never opts in now carries no telemetry callback
and writes no deployment hash secret to LiteLLM_Config.
---------
Co-authored-by: nate <nate@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(telemetry): record proxy requests and provider attempts into the telemetry sink
* fix(telemetry): validate env settings, fix the window and only record the request's own provider calls
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(telemetry): bind the call id from a typed local
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(telemetry): record a finished request into the sink current when it finishes
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(telemetry): record every failed provider attempt, including retries and fallbacks on a shared logging object
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(telemetry): report timeouts and connection failures as their own provider status classes
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(telemetry): record failed attempts from the per-deployment failure hook instead of a new core logging hook
Reverts the async_log_attempt_failure_event hook added to CustomLogger and
litellm_logging in 3421b38685. The router already calls
async_post_call_failure_deployment_hook once per failed attempt, retries and
fallbacks included, so every failed fallback still gets its own row without
touching the shared logging object.
* fix(telemetry): register the attempt logger only once a telemetry group is turned on
A proxy that never opts in now carries no telemetry callback. Once registered
the logger stays, returning early while every group is off.
---------
Co-authored-by: nate <nate@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(typing): validate provider payloads at the boundary to clear 464 basedpyright Any errors
Replaces untyped reads of decoded JSON in 48 provider, guardrail and proxy
modules with one pydantic validation at the point the payload enters the
module, so every value read afterwards carries a real type.
reportAny 6111 -> 5663 and reportExplicitAny 1416 -> 1400. No basedpyright,
ruff-strict or type-discipline rule is higher repo-wide or in any single
file, and no cast, ignore or suppression marker was added.
* chore(typing): replace 47 Any annotations that every caller already satisfies with object
Each annotation here is a parameter or return type whose value is only
stored, compared, logged or forwarded, so object describes it exactly and
no function body changes. A site was kept only if no basedpyright,
ruff-strict or type-discipline rule rose in any file that references it.
reportExplicitAny 1400 -> 1353 and reportAny 5663 -> 5655.
* chore(typing): type upstream responses and loose annotations across providers, proxy and integrations
Second pass of the Any reduction. Decoded upstream JSON is validated once at
the provider boundary and read through typed views afterwards, annotations
that every caller already satisfies move from Any to a concrete type, and the
nine search transformations validate each result item on its own so error
text no longer carries the item index.
No cast, no type: ignore, no suppression marker and no budget file edit
* ci(unit): run the brave, google_pse, nlp_cloud and opensandbox tests in the providers shard
A JSON-RPC tools/call on /{server_name}/mcp (or /mcp with x-mcp-servers naming one server) that uses the upstream's bare tool name answered "Tool not found" on every proxy process that had not yet listed that server, because the per-process tool-name map stays empty for a server whose upstream only lists tools with the caller's own credentials, and mcp SDK 2 no longer lists before an uncached call. call_mcp_tool now hands the single scoped server to execute_mcp_tool as requested_server_id, so the existing REST contract lists the server with the caller's headers and routes the bare name to it, and a bare name shared by two servers goes to the one the path names
Co-authored-by: aayushbaluni <73417844+aayushbaluni@users.noreply.github.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): broadcast team cache invalidation on team writes
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): cover cross-instance team cache invalidation
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): source Home What's new from day 0 support blog posts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): Home cards read card_title and card_summary from blog tags with provider logo
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(unit): wait on the work itself instead of fixed sleeps in 21 flaky files
Replace sleeps that raced background logging, cache writes, refreshes and
thread work with either awaiting that work directly (logging worker flush,
the batch logger's flush_queue, the tasks the call spawned, thread joins) or
an Event the test's own fake sets when it runs. Each wait keeps a 5s hang
guard that never decides pass or fail. Tests that need a fake clock, real
sockets, or measure latency are left unchanged, and no production code moves
* test(unit): await the tasks each completion spawns instead of yielding once
The logging tests yielded once with asyncio.sleep(0) so the task that
enqueues the success log could run before the worker flush. That relied
on the enqueue happening before the task's first await. Snapshot the
running tasks around each acompletion call and await the ones it spawned,
so the guarantee holds however many steps sit before the enqueue
* fix(proxy): return provider-matching 4xx for the LIT-8062 status code follow-ups
- OCR multipart errors are 400 invalid_request_error on the file param
- a missing model is a 400 on chat, responses and messages even with a wildcard deployment
- responses max_output_tokens <= 0 reaches the provider instead of being raised to 16
- daily activity rejects end_date before start_date with a 400
- vector store file attributes are sent in full so the provider enforces its key limit
- router-wide defaults no longer satisfy a param the Router takes positionally, or a model the router does not serve
- a JSON null user_config is treated as absent
* chore: keep the type gates at their merge-base counts
* fix(proxy): keep fixed-target wildcards serving a missing model and close router-default gaps
- a missing model is rejected only when the wildcard would forward it into the target
- moderation only counts router defaults for models get_model_list lists
- user_config defaults and deployment params never satisfy a positionally bound param
* test(integration): pin the fixed-target wildcard path and no provider call on positional-param 400s
* chore: validate pattern-matched deployments instead of reading untyped dicts
* fix(proxy): keep router defaults counting for a2a agent models
* test: drop the signature-walk test and type the new test helpers
* test(integration): audit matrix for the status-code follow-ups
Drives every mode the fix branches on through a real proxy, Postgres, Redis and
the scripted upstream: positional params against router, deployment and
user_config defaults; moderation listed vs unlisted; unknown and a2a models;
every user_config shape on chat, responses and messages, streaming and SDK
clients; forwarding vs fixed-target wildcards for missing, null and empty
models; Responses max_output_tokens edges; OCR parse errors; vector store file
attributes; every daily activity route; and a provider outage mid burst plus a
killed worker on a two-worker proxy. Product gaps found are skipped as BUG.
* style: ruff format the audit test files
* test: name the auth pre-check as the source of the JSON array 500
* test: build the audit helpers without mutation and share one sentinel
* test: build every audit request body as one literal
* fix(proxy): keep router defaults and model-less dispatch for requests a fallback chain serves
* fix(proxy): count request, key and team fallbacks when an unlisted model inherits router defaults
* fix(proxy): skip key and team fallbacks on the client-credential dispatch path
* fix(proxy): keep router defaults when the router's own * fallback serves an unlisted model
* fix(proxy): count client-style fallbacks, forward an empty model, report invalid usage chat dates
- The router-default pre-check now treats client-style request, key and team fallbacks
([{"model": ...}]) as serving an unlisted model, as the router does
- The forwarding-wildcard gate only rejects a missing or null model; an empty model
reaches the provider as before (vLLM serves its loaded model for it)
- The /usage/ai/chat tools hand the model the date-range error instead of an empty
dataset or a generic fetch error
* test(usage): type the invalid-date tool test
* fix(proxy): keep rejecting an empty model behind a forwarding wildcard
Key and team model checks skip an empty model, so forwarding it let a key
restricted to another model reach a server's default model (vLLM serves
its loaded model for ""). Pin that with a restricted-key integration cell.
* fix(usage): report the query layer's date 400 to the usage chat model
Checking dates before the fetch refused formats the global usage tool's query
accepts (2026/10/01 returned spend on main). Catch the daily activity query's
own 400 instead, so each tool rejects exactly what it rejected before and only
the message changes.
* fix(proxy): restore three model-less and unlisted-model paths main served
An empty model is looked up as sent, so next to a forwarding */* wildcard it
still reaches the fixed * deployment it matches. A null model with a prompt_id
is left to the prompt manager, which may pick the model. A Claude Code
subagent turn on a session-bound auto/complexity router counts as served, so
the router default max_tokens still applies.
* fix(proxy): only skip the model check for a prompt_id when a prompt manager is registered
Without a callback prompt manager nothing can pick the model, so a null model
with a prompt_id behind a forwarding wildcard gets the same 400 as any other
null model.
* fix(proxy): leave a model-less Claude Code subagent turn to its session router
A subagent turn on a session bound to an auto, complexity, adaptive or
quality router is answered by that router whatever model it names, so a
null or missing model behind a forwarding wildcard is no longer rejected.
An empty model stays rejected.
* fix(proxy): skip the wildcard model check only for a bound Claude Code session
Read the session's router binding, as the router does, instead of only
checking that the request could resume one, so a subagent turn without a
model on an unbound session is still rejected before a forwarding
wildcard sends it upstream.
* refactor(proxy): stop predicting routing in the proxy pre-checks
- drop the wildcard missing-model gate: a model-less request behind a forwarding wildcard goes to the router and gets the provider's own status, as on main
- drop the "model the router serves" check: router defaults count for any named model again, except moderation, where the Router merges them only for listed models
- keep the positional-param, moderation and null user_config fixes
* test(ocr): cover the form, size, empty body, invalid JSON and file-document 400s
---------
Co-authored-by: yucheng-berri <293926326+yucheng-berri@users.noreply.github.com>
test_cold_concurrent_schema_validation_accepts_valid_arguments pins the
process to a single CPU core and fans out 8 concurrent calls through
anyio.to_process's subprocess worker pool, each needing to cold-spawn a
fresh interpreter and import jsonschema. Under a slow or contended CI
runner that routinely exceeds the production 30-second budget before any
actual validation work runs, failing the test on a genuine timeout rather
than a validation bug.
_tool_argument_validation_error now takes an optional timeout_seconds,
defaulting to the existing constant, so production behavior is
unchanged. The test passes a timeout sized for its own single-core,
cold-start stress scenario, matching the margin the companion
integration test (test_schema_validation_stops_expensive_work_and_recovers)
already grants a similar concurrency=8 case.
* feat(guardrails): add Wingback guardrail integration
Mirrored from #38685 at d73b66544d (net diff against its merge base, applied on main)
* fix(guardrails): dispatch Wingback through the unified guardrail
The proxy only runs a unified guardrail when apply_guardrail is defined on
the guardrail's own class, so WingbackGuardrail now binds the parent's
decorated method in its class body. Chat, messages and responses pre, post
and during call checks reach Wingback again, and each check logs once.
Also types the initializer and config model for the lint gates and uses
monkeypatch.setenv in the unit test
* chore(guardrails): drop the Wingback dispatch comment
The unit and integration hook tests already fail when the apply_guardrail binding is removed, so the comment is redundant under the repo comment rule
---------
Co-authored-by: bsk-wb <bharath@wingback.ai>
Co-authored-by: yucheng-berri <293926326+yucheng-berri@users.noreply.github.com>
* feat(otel): let a team or key excluded_services override the global setting for its OTel v2 destination
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style: ruff format otel excluded_services changes
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: use raw regex for excluded_services match
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(otel): reject non-string destination callback vars
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(otel): harden destination callback var validation
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(otel): compare parsed values in callback var conflict check
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* Add user field to AI Guard execute-policy payload for dashboard attribution
Send the top-level OpenAI request `user` field in the execute-policy
JSON payload so AI Guard can surface usernames on its dashboard. This is
independent of the existing user_api_key_* headers which remain unchanged.
The upstream guardrail framework already passes the original request_data
(containing the `user` field) through to response-path guardrail calls,
so this works for both IN and OUT directions including streaming.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Apply suggestion from @greptile-apps[bot]
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* Add type annotations and tests for user field in execute-policy payload
- Add Final annotation to user_for_payload
- Add str | None type to user parameter in make_zscaler_ai_guard_api_call
- Add 3 tests: user included when present, omitted when absent,
and user in/out of JSON body sent to AI Guard
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* test(zscaler_ai_guard): inject the http client in user field tests
Pass an AsyncHTTPHandler into ZscalerAIGuard so the user field tests read the
real execute-policy body off a MockTransport instead of patching class methods,
and drop the comments that restated the lookup
* fix(zscaler_ai_guard): forward only non-empty string users
Read the request user through a typed helper and type the execute-policy call
params so basedpyright stays under the merge-base count, and skip non-string
values the vendor rejects with a 500
* test(zscaler_ai_guard): drop the test-only http_client
Run the user field tests against an in-process aiohttp server through api_base
so the guardrail keeps its original constructor
---------
Co-authored-by: Syamala Keerthana Chegu <schegu@zscaler.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: yucheng-berri <293926326+yucheng-berri@users.noreply.github.com>
* fix(proxy): keep the personal budget on a proxy-admin user object
A token carrying the proxy_admin role (JWT admin scope, SSO) replaced the fetched LiteLLM_UserTable row with a bare admin object, so max_budget never reached common_checks and admins spent past their personal budget on /chat/completions. Keep the row and force only user_role.
Co-authored-by: L4XB <lukas.buck@e-mail.de>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(e2e): budget denial for a proxy-admin JWT is the 422 ExceededBudget response
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): carry only the personal budget onto a forced proxy-admin user object
Keeping the whole DB row also carried its models list, so a proxy admin
whose row had no-default-models or a restricted list got 403 on personal
requests. Build the admin object as before and copy max_budget and
user_email across. The built-in admin identity (the master key and the UI
username/password login) keeps ignoring its row's budget.
* test(proxy): pin the contrast case in the admin model-list and admin-name budget tests
Each test now also shows the same cached row refusing a non-exempt token, so the
pass case cannot go green on a row the gate never read.
* test(e2e): drive the personal-budget case with a proxy_admin user row and a plain JWT
The Buildkite stack keeps the default admin_allowed_routes, so an admin-scope JWT
gets 403 on /chat/completions before any budget check and the case could never
pass there. A plain token whose sub resolves to a proxy_admin row goes through the
same forced-admin rebuild. Drop the stage-mirror admin_allowed_routes edit, which
the deployed stack never read.
* ci: rerun e2e after an unrelated Gemini streaming spend-row timeout
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: L4XB <lukas.buck@e-mail.de>
Co-authored-by: yucheng-berri <293926326+yucheng-berri@users.noreply.github.com>
cost_discount_config keys may now be <provider>/<model-pattern> with fnmatch globs, so a discount can target a subset of a provider's models. An exact pattern beats a glob, the glob with the longest literal wins, and the bare provider key is the fallback. PATCH /config/cost_discount_config validates the provider half only, and the admin UI cost tracking page gains a Model pattern field and a Models column
Resolves LIT-8465
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(decisions): bill decisions and systemone from <provider>/decisions/<model> cost map rows
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(decisions): mark decisions pricing rows with mode decisions
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): sync generated API types for decisions mode
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(decisions): update pricing expectations in proxy tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(decisions): price classifier calls from decision rows
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(health): drop redundant evaluation handler after main merge
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(decisions): inject pricing row for deployed classifier
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(decisions): ship the decisions pricing lookup without cost map rows
Restore both cost maps to main. Tests inject their own
<provider>/decisions/<model> rows. The Add Model dropdown hides those
pricing keys and offers mode decisions instead of evaluation.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(ui): expect the Decisions mode option in the Add Model form
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(decisions): bill decisions and systemone from _decisions cost map fields
Replace the provider/decisions/model pricing rows with <field>_decisions
fields on the model's own row. cost_per_token overlays them for the
decisions, adecisions, systemone and asystemone call types and prices
through generic_cost_per_token; chat, responses and batch keep the plain
fields. gpt-6-luna gains its Decisions rates (input 1e-07, output 0,
cached and cache-write input 1e-07, 2x above 272k tokens).
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(decisions): satisfy type gate, schema sync and test_utils schema for _decisions fields
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(decisions): import cost_calculator module to keep basedpyright unknown-variable count flat
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(decisions): drop stray fireworks import from __init__ and annotate test cost rows for the type gate
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): cover gpt-6-luna decisions and systemone above the 272k tier
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(decisions): keep the decisions pricing block clear of main's stale azure comment
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(cost_calculator): drop stale azure models doc comment
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): skip failure rows for expired Admin UI session keys on dashboard info routes
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): mark DB-boundary write assertion as test-quality-ok
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(decisions): support Clef-omni audio and video
* fix(decisions): accept Clef-omni media types with parameters and cover the proxy routes
The clip patterns now mirror Cloudflare's input schema, so a data URL or
content_type carrying parameters like audio/webm;codecs=opus is accepted.
The four proxy routes get media tests of their own, the backup cost map
is synced with the new Clef-omni rows, and the docs page moves to the
docs repo as a follow-up.
* fix(decisions): refuse parameterized media-type data URLs and refresh the OpenAPI snapshot
Cloudflare answers 422 to a data URL whose media type carries parameters even though its schema only requires the data:audio/ prefix, so LiteLLM now refuses those with a 400 before the call. The object form keeps passing parameters through, which Cloudflare accepts. The lazy OpenAPI snapshot and the dashboard API types are regenerated for the new audio and videos fields
* fix(decisions): gate media by cost-map flags, treat null clips as none, drop unsupported clips before validation
* fix(decisions): keep the SDK's positional order and gate media flags on evaluation-mode models
* fix(decisions): gate media on both decisions model modes and move the audit cells onto the harness helpers
* test(decisions): cover the Fireworks System One router in the media refusal cells
---------
Co-authored-by: moyai-devin-berriai[bot] <336287033+moyai-devin-berriai[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* test(managed_files): pin ownership checks on batch and fine-tuning create
Tenant B can create a batch or fine-tuning job from tenant A's managed
file id today. These fail on main for acreate_batch input_file_id and
acreate_fine_tuning_job training_file and validation_file
* fix(managed_files): check input file ownership on batch and fine-tuning create
Batch create and fine-tuning create only mapped the managed input file to
provider ids and never checked who owned it, so any key with access to the
model could build a batch or job on another tenant's file and then own the
outputs. Both now run check_file_ids_access like chat and responses do,
which 403s a non-owner and 404s a managed id with no ownership row.
Fine-tuning also checks validation_file, which was never looked at
* test(batches): pin that a pre-call ownership denial stops every batch create branch
LIT-4797 noted an endpoint-level check missed the load-balanced branch
because it dispatches first. The ownership check lives in the pre-call
hook, which runs before branch selection, so a 403 there must keep both
the load-balanced and unified-file branches from reaching the router
* test(integration): pin managed-file ownership on batch and fine-tuning create
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(managed_files): check ownership before the batch rate limiter reads the input file and keep key-owned resources across key regenerate
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: migrate proxy security, schema migration and MCP legacy tests
* test: run the MCP proxy e2e upstream under the SDK1 peer in integration-mcp
* test: warm the baseline store before timing the locked projection
* test(e2e): hold the stack exclusively while the semantic filter flag is flipped
* test(mcp): scope the in-process proxy e2e fixtures to their module
The session-scoped autouse fixture deleted DATABASE_URL and overrode LITELLM_MASTER_KEY for the rest of the worker, which broke every later test in the integration mcp suite
* test(e2e): assert the semantic filter keeps listed tools instead of a ranking
Which Datadog tool an OpenAI embedding ranks first is not something litellm owns, so the test now checks that the kept tools are a non-empty subset of the key's listing within top_k
* test(mcp): keep the in-process proxy e2e tests on one worker
loadgroup spread the file's tests across workers and interleaved them with other files, so the module-scoped in-process proxy was torn down and restarted between tests and requests timed out
* test(mcp): name the upstream requests that carried the wrong token
The bare all() assertion hid which request broke the forwarded token, which made an intermittent CircleCI failure undiagnosable
---------
Co-authored-by: yuneng <yuneng@berri.ai>
* refactor(rust_bridge): group the Python modules Rust imports by name under rust_bridge/host
Rust reads Python-owned state (legacy callback registries, model capabilities) through
modules it imports by path. Put them in one folder with an AGENTS.md that states the
contract, and update the module path constants on the Rust side.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* refactor(rust_bridge): route Python state reads through host accessors
* docs(rust_bridge): point test_support at the moved host contract test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(rust_bridge): move secret manager, transport and timeout accessors under host
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(greptile): register rust_bridge/host AGENTS.md in files.json
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: migrate tests/proxy_behavior into integration and unit tiers (partial)
* test(integration): run settings-dependent management tests on an owned proxy
* test(integration): assert stored values in the rewritten management rows
* test(integration): pin sibling team allocations against the org guaranteed throughput cap
* test(integration): keep Prisma clients out of the shared integration conftest import path
---------
Co-authored-by: yuneng <yuneng@berri.ai>
* ci: run every CircleCI check from the integration workflow
* test(ci): keep the startup test CircleCI-owned
* test(ui): find reset log rows by request id so concurrent workers cannot page them out
* test(ui): retry the key budget window spec until the draft refresh fix lands
* ci(ui): set Playwright workers and retries on the CircleCI command only
The shared playwright.config.ts also drives the Buildkite UI suite, where 3 workers
ran out of memory on one of two runs. Buildkite keeps main's 1 worker and 2 retries
---------
Co-authored-by: yuneng <yuneng@berri.ai>
* test: migrate legacy unit dirs and unrun folders into tests/unit, tests/integration and tests/e2e
* test: fix disk cache collection, ungate saturation check, drop RSS memory tests
* test: assert one delivered log per completion in Datadog and LangSmith checks
* test: drop SDK-direct e2e copies and align cost-map path filter cases
---------
Co-authored-by: yuneng <yuneng@berri.ai>
* fix(home): serve the bundled whats_new.json by default so a release shows only its own launches
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit 4dd96800e8)
* style: annotate whats_new_url with str | None
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit d1ae7ea085)
* test(home): pin whats_new_url to the env var and regenerate the dashboard API types
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit 4bfec53865)
* test(public_endpoints): prove the /public/whats_new route serves the bundled list when no URL is configured
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(public_endpoints): drop the unused os import
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(home): pin whats_new_url to the env var so a baked-in default URL fails the suite
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(tests): sort the import blocks ruff flagged in test_public_endpoints
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(home): let the route test read the real whats_new_url default so a baked-in URL fails it
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: move guardrails, image, audio, batches, search legacy tests to unit
* test: restore the custom guardrail and remaining chunk 1 unit rows
* test: move the ocr matrix to e2e
* test: drop comments from moved tests and fix vertex ocr credentials
* test: move litellm utils and unified google legacy tests to their tiers
* test: move logging callback and pass-through unit legacy tests to their tiers
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: move store_model_in_db legacy tests and start the admin ui migration
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: finish the proxy job rows and drop their CircleCI jobs
Moves the remaining multi-instance, openai endpoint, otel, pass-through and
top-level proxy tests into tests/integration with scripted upstreams, rewrites
the weak replacements to assert the outbound wire and exact responses, and
removes the build_and_test job group this PR owns from CircleCI
* test: tighten the key generation and error log assertions, rename migrated unit files to their mapped names
* test: shard the pagerduty unit tests and record the rust harness file on the coverage allowlist
* test: drive the SSO sign-in tests through a scripted IdP and isolate the cost center stand-in
* test: type the e2e beta header, prompt cache and secret manager values
* test: assert the missing model error message on the OpenAI routes
* test: drop the deleted otel base class import and restore mcp redaction and exact langfuse usage checks
* test: make the langfuse fixture directory a package for the layout check
* test: restore collected no-match cases, real langfuse and gcs payload checks, and gate the aws secret lane
* test: keep the ocr e2e pdf fixture next to the test
* test: put meta below covers on the streaming tool search e2e
* test: run the OCR matrix against a scripted upstream and fix two runner-only e2e failures
* test: count only OCR calls in the OCR matrix recorder
* test: pin OCR parse bodies and billed cost, group new e2e cases into classes
* ci: restore the python image anchor dropped in the main sync
---------
Co-authored-by: yuneng <yuneng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(databricks): call the ai_decide AI Function route for decisions and the auto-router
Databricks serves ai_decide at POST <host>/api/2.0/ai-functions/ai-decide, not
on a serving endpoint's /invocations route. The decisions provider and the
auto-router classifier now accept only the ai_decide model, post to that route
without a model field, unwrap the response envelope, and map a noul answer's
probability to the System One noul field.
* test(databricks): bound the megabyte model name test by scaling, not wall time
* test(databricks): assert the whole OpenAI-format decision and floor the liveliness bound
* test(decisions): move the Databricks OpenAI-format wire row to the ai_decide route
* fix(databricks): keep serving endpoint decisions beside the ai_decide route
* test(databricks): assert the whole classifier request body in the test routing case
* test(databricks): correct System One spend call type
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): load the Home page What's new launches from /public/whats_new at runtime
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): show the three newest What's new launches
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* build: ship whats_new.json in the wheel
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(decisions): rename the evaluation model mode to decisions
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(decisions): canonicalize the health check mode without rebinding
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(decisions): replace canonical_model_mode with is_decisions_model_mode
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(decisions): preserve evaluation mode compatibility
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore: drop the rebuilt dashboard bundle from this PR
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(dashboard): format decision mode helper
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(headroom): drop null tool_calls before calling /v1/compress
headroom-ai 0.27.0 through 0.30.0 iterate msg.get("tool_calls", []) and fail
with 'NoneType' object is not iterable when an assistant row carries an explicit
tool_calls: null, which the fail-closed guardrail then surfaces as a 502. Strip
the null key from rows sent to the compression service; real tool_calls lists
are forwarded unchanged
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(headroom): pin that SDK-echoed null tool_calls never reach /v1/compress
Main now withholds every assistant row from /v1/compress (#39974), so the
null tool_calls strip added here never runs on a row headroom-ai 0.27-0.30
fails on. Drop the strip and pin the customer's history shape through the
public OpenAI chat translation handler instead.
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: yucheng-berri <293926326+yucheng-berri@users.noreply.github.com>
* feat(ui): add a Home page as the dashboard root with usage summary, recent launches, blog updates and a model leaderboard
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): fall back to spend-log request counts on Home when the gateway has recorded none
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(ui): cover the Home page totals, launches, leaderboard and updates feed
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): render Home Updates as rows and return up to 6 blog posts from /public/litellm_blog_posts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): filter Home Updates by the new-model and api-change blog tags
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(blog): import NotRequired from typing_extensions for py3.10 and test the endpoint passes every fetched post through
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* revert(ui): drop the Updates filter chips, keep Updates as a plain row list
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(ui): restore generated schema.d.ts to main
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): label the Home key button Create New Virtual Key
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(ui): drop the What's new sublabel on Home
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): show an error with retry when Home usage fails to load and move the Home test to the integration tier
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(dd_span_tagger): emit litellm_user_email span tag for JWT-authenticated requests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit b4f9e14a44)
* refactor(dd_span_tagger): use dotted litellm.user_email tag for consistency
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit 819faa01c3)
* feat(model_hub): surface model_info.description in model group info and Model Hub UI
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit b224b15b9d)
* refactor(router): aggregate model group description without in-place mutation
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit 342b93fe0a)
* test(dd_span_tagger): assert the user_email tag on the active span instead of on the patched setter
The two ported tests only inspected the mock that replaced the tag setter, which the test quality gate counts as two new mock-echo tests over main. They now read the tags that land on the active span
* refactor(router): read the model group description through the typed model_info reader
The ported lookup read model_info off the untyped deployment dict, which added three basedpyright errors over main. It now goes through the existing validated model_info reader
---------
Co-authored-by: milan <milan@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: ryan-crabbe-berri <ryan@berri.ai>
* fix(guardrails): scan or refuse attachments in the bedrock guardrail
* fix(guardrails): cover native pre_call and computer screenshots in the bedrock attachment scan
* fix(guardrails): classify every attachment shape the bedrock guardrail can receive
* test(guardrails): type the attachment-scanning test guardrail
* fix(guardrails): keep base behavior for Bedrock subclasses and malformed messages
- skip the attachment scan when a subclass overrides apply_guardrail
- ignore non-string message role/type instead of raising
- treat a null Converse document/video/audio block as absent
- keep the base request-body debug log for text-only calls
* fix(guardrails): let text documents, unpadded base64 images and request-override subclasses through as before
* fix(guardrails): let Converse text documents through and refuse documents that carry images
* fix(guardrails): scan images inside content documents instead of refusing them
* fix(guardrails): scan text document content with the Bedrock guardrail
* fix(guardrails): scan document title, context and nested documents
* Expand nested content documents without recursion
* Keep attachment scan within the type-discipline budget
* Take the image format and pixel size from the image bytes
* chore(guardrails): satisfy type-discipline gate after merging main
* chore(guardrails): drop docstrings added by the attachment scan
* fix(guardrails): scan Converse text documents and guardContent images
A Converse document whose source is text (or a list of text parts) now
goes through the Bedrock text scan instead of being refused as
unscannable, so the pass-through request that worked before this PR
keeps working. A Converse guardContent image is now scanned like any
other image. Byte and S3 document sources stay unscannable.
Adds the integration test that drives every scanned endpoint through a
real proxy against a scripted Bedrock ApplyGuardrail stub, and retypes
the touched hook signatures for the type gates.
* test(guardrails): cover Bedrock checks mode in the attachment integration test
Adds a checks-mode proxy to the rig: images are refused without calling Bedrock on every route, and a Converse text document is sent to InvokeGuardrailChecks.
* fix(guardrails): scan a Converse document's name and context whatever its source
A Converse PDF or S3 document let through by skip_unscannable_attachments now still has its name and context scanned as text, the same as the Anthropic document path.
---------
Co-authored-by: yucheng-berri <293926326+yucheng-berri@users.noreply.github.com>
* fix(credentials): keep credential labels and edits consistent across UI, API, and replicas
The credential modal flattened stored credential values into the same form object as the
credential's own name, label, and provider, so a stored value keyed credential_name or
display_name replaced the real one and the edit went to the wrong credential or relabeled it.
The modal now submits the top-level fields and credential_values separately
PATCH /credentials now reads the row it merges onto from the writer, so an edit inside the read
replica's lag window no longer writes back an older label or key
POST /credentials answers 409 for a name defined in config instead of creating a DB copy that
shadows it
display_name rules live on the server only. It rejects control, formatting, and line separator
characters, and the UI stops enforcing its own length check and shows the server's error
The models table, model detail view, and model edit picker show the credential's technical name
under its label whenever the two differ
* fix(ui): show credential source in its own column like models
The config marker sat under the credential name. The credentials table now has a Source column
with a Config or DB status badge, matching the Source column on the models table
* test(credentials): unit-test label and prefill helpers and keep the full credential name on hover
Moves displayNameChange and initialFormValues out of the modal so their rules are unit tested,
keeps only the wiring cases in the modal's integration test, makes the label acceptance test
write through the real credentials repository, and adds the full credential name as a hover
title where the models pages truncate it
* feat(guardrails): add decision_model guardrail for decisions-API models
Adds a new guardrail provider that scores request and response text with a
decisions-API model (e.g. Jev) against named predicate checks. Each check
carries its own threshold and action (block or log); a flagged block check
rejects the request with a 400 naming the flagged checks, and call failures
honor unreachable_fallback (fail_open or fail_closed).
Ships four preset checks for the Admin UI (prompt injection, jailbreak,
system prompt extraction, data exfiltration) served from
/guardrails/ui/add_guardrail_settings, plus a Decision Model step in the
add-guardrail form filtered to evaluation-mode models.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(guardrails): fail closed on verdict errors and drop comments in decision_model
* feat(guardrails): cap decision_model input with head and tail elision
* fix(guardrails): list only block checks in decision_model 400s and tighten inputs
* feat(guardrails): honor timeout, cap checks at MAX_DECISION_QUESTIONS, and reject duplicate check names
* fix(guardrails): decorate decision_model apply_guardrail with log_guardrail_information
* fix(guardrails): suppress untyped-decorator pyright error on log_guardrail_information import
* feat(guardrails): screen each text in overlapping chunks with a shared decisions-call semaphore
Replace head/tail elision with per-text screening: each non-empty text is
sent to the decision model on its own, split into overlapping chunks when
it exceeds max_input_chars, and all calls run under a per-event-loop
semaphore shared across requests (max_concurrent_decision_calls,
default 8). Check verdicts aggregate the max probability across calls; a
missing predicate answer on a Block check blocks under fail_closed and
only warns under fail_open, while log-only checks are never flagged for
it. The 400 detail now lists flagged check names only. The provider
combobox also matches Decision Model on jev, typesafe, prompt injection,
jailbreak, and decision
* fix(ui): align decision model custom check rows with preset rows
* fix(ui): move decision model custom check remove button to the row end
* style(ui): run prettier on decision model check files
* fix(ui): inline decision model evaluation-mode filter after lens module removal
* fix(guardrails): keep decision model detections when a chunk call fails
* fix(ui): require an enabled decision model check before create
* fix(guardrails): screen decision model texts sequentially like presidio
* feat(guardrails): replace decision model presets with provider picker and per-question test
* fix(ui): render decision model question test as a full-width row
* feat(ui): add run-history test section to decision model form
* fix(ui): prevent threshold overlap and show test shortcut
* feat(guardrails): default decision model threshold to 0.7
* test(ui): expect decision model default threshold 0.7 in characterization payload
* feat(guardrails): screen tool call arguments with the decision model
* refactor(guardrails): freeze the decision model config model
* feat(guardrails): list only evaluation-mode deployments in the decision model picker
* feat(guardrails): hide the decision model questions table until a question exists
* style(ui): format DecisionModelFields
* style(ui): format decisionModelQuestion tests
* refactor(guardrails): type the tool call function extraction
* refactor(guardrails): build tool call texts without a mutable accumulator
* refactor(ui): share isDecisionMode for the decision model picker
* fix(guardrails): screen prior Responses function call arguments in the decision model guardrail
The Responses input handler leaves function_call arguments out of texts and tool_calls, so a prompt injection in a prior call's arguments passed unscreened. Assistant tool calls from structured messages are now screened too, deduplicated against the request's tool calls
* test(integration): send instructions with the decision model guardrail check
Checks require instructions, so creating the guardrail returned 422 and the test never sent a chat request
* fix(ui): score the decision model test preview the way the guardrail does
An unanswered block question now shows Block, disabled questions no longer count, and a run is hidden once the model or a question's text changes since it was scored
* fix(ui): clear the decision model error on pick and list questions under Add a question
Picking a model after a failed Create left "Select a decision model" on screen, because the form only validates on submit. The field now revalidates when it already shows an error. The Questions table and its heading appear only once a question exists, below the Add a question box
* test(ui): move the decision model form tests into an integration test file
They render the whole AddGuardrailForm, so they belong in a .integration.test.tsx file. Plain fields are now filled with fireEvent.change, and the two existing form test files go back to main apart from the modelHubCall mock the form now needs
* test(guardrails): build the decision model test params without mutating a local
* fix(guardrails): reject list checks outside decision_model and honor streaming scan settings
- LitellmParams now rejects a list `checks` for any guardrail other than
decision_model, so a Bedrock guardrail with list checks gets 422 again
instead of being stored and failing every request
- The decision model guardrail reads streaming_end_of_stream_only and
streaming_sampling_rate, so a streamed post_call response can be scored
once at end of stream instead of every 5 chunks
- max_concurrent_decision_calls is described as a per-worker limit
* fix(ui): match guardrail providers like the default picker filter
- The provider search uses the Base UI collator filter on the label and
the aliases, so searches such as presidio_pii or llm_as_a_judge match
again, as they did before the alias filter
- Model groups from /model_group/info are parsed defensively and after the
providers are populated, so a malformed response cannot empty the
provider list
* fix(ui): edit decision model questions in place and show passing test chips in green
Questions start as an Add question button. Each click adds a blank question
card with its name, text, action and threshold, editable until the guardrail
is created, mirroring the per-model budget editor. Create and Next reject a
blank or duplicate enabled question. Test run verdicts and chips use the
success tone for Pass, the error tone for Block and the warning tone for a
flagged Log only question
* fix(ui): drop the per-question enable checkbox from the decision model form
A question is either in the list or removed with its X, so every listed
question is sent and scored. The card now labels Name and Question the way
it labels Action and Threshold
* fix(ui): give the question name field the full card width
* fix(ui): show the provider logo on the selected decision provider
* fix(ui): show the full threshold slider track on decision model questions
* feat(ui): add decision model cards to the guardrail garden
Adds a Decision Model section to the guardrail garden with TypeSafe Jev,
Perplexity, Microsoft, OpenAI and Databricks cards. Each card opens the
create form scoped to that provider's decision models.
The create form drops the Decision Provider picker and lists every
decision model on the proxy with its provider logo, prompts to add a
model when none is available, and says that test runs are billed and
logged.
* fix(ui): shorten the decision model test hint to one line
* fix(ui): prefill the decision model from the garden card's provider
* chore(ui): regenerate the dashboard API types after merging main
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: yucheng-berri <293926326+yucheng-berri@users.noreply.github.com>
* fix(proxy/auth): expand all-team-models sentinel for project model access checks
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(e2e): use CI-served models and tighten project all-team-models denial check
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(e2e): declare coverage registry cells for project all-team-models tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* ci: retrigger e2e after ephemeral stack deploy timeout
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(e2e): wait for team allowlist propagation before project denial check
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(e2e): annotate written_at as Final per LIT010
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: jesus <jesus@berri.ai>
