* feat(otel): let a team or key excluded_services override the global setting for its OTel v2 destination
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style: ruff format otel excluded_services changes
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: use raw regex for excluded_services match
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(otel): reject non-string destination callback vars
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(otel): harden destination callback var validation
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(otel): compare parsed values in callback var conflict check
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* Add user field to AI Guard execute-policy payload for dashboard attribution
Send the top-level OpenAI request `user` field in the execute-policy
JSON payload so AI Guard can surface usernames on its dashboard. This is
independent of the existing user_api_key_* headers which remain unchanged.
The upstream guardrail framework already passes the original request_data
(containing the `user` field) through to response-path guardrail calls,
so this works for both IN and OUT directions including streaming.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Apply suggestion from @greptile-apps[bot]
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* Add type annotations and tests for user field in execute-policy payload
- Add Final annotation to user_for_payload
- Add str | None type to user parameter in make_zscaler_ai_guard_api_call
- Add 3 tests: user included when present, omitted when absent,
and user in/out of JSON body sent to AI Guard
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* test(zscaler_ai_guard): inject the http client in user field tests
Pass an AsyncHTTPHandler into ZscalerAIGuard so the user field tests read the
real execute-policy body off a MockTransport instead of patching class methods,
and drop the comments that restated the lookup
* fix(zscaler_ai_guard): forward only non-empty string users
Read the request user through a typed helper and type the execute-policy call
params so basedpyright stays under the merge-base count, and skip non-string
values the vendor rejects with a 500
* test(zscaler_ai_guard): drop the test-only http_client
Run the user field tests against an in-process aiohttp server through api_base
so the guardrail keeps its original constructor
---------
Co-authored-by: Syamala Keerthana Chegu <schegu@zscaler.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: yucheng-berri <293926326+yucheng-berri@users.noreply.github.com>
* fix(proxy): keep the personal budget on a proxy-admin user object
A token carrying the proxy_admin role (JWT admin scope, SSO) replaced the fetched LiteLLM_UserTable row with a bare admin object, so max_budget never reached common_checks and admins spent past their personal budget on /chat/completions. Keep the row and force only user_role.
Co-authored-by: L4XB <lukas.buck@e-mail.de>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(e2e): budget denial for a proxy-admin JWT is the 422 ExceededBudget response
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): carry only the personal budget onto a forced proxy-admin user object
Keeping the whole DB row also carried its models list, so a proxy admin
whose row had no-default-models or a restricted list got 403 on personal
requests. Build the admin object as before and copy max_budget and
user_email across. The built-in admin identity (the master key and the UI
username/password login) keeps ignoring its row's budget.
* test(proxy): pin the contrast case in the admin model-list and admin-name budget tests
Each test now also shows the same cached row refusing a non-exempt token, so the
pass case cannot go green on a row the gate never read.
* test(e2e): drive the personal-budget case with a proxy_admin user row and a plain JWT
The Buildkite stack keeps the default admin_allowed_routes, so an admin-scope JWT
gets 403 on /chat/completions before any budget check and the case could never
pass there. A plain token whose sub resolves to a proxy_admin row goes through the
same forced-admin rebuild. Drop the stage-mirror admin_allowed_routes edit, which
the deployed stack never read.
* ci: rerun e2e after an unrelated Gemini streaming spend-row timeout
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: L4XB <lukas.buck@e-mail.de>
Co-authored-by: yucheng-berri <293926326+yucheng-berri@users.noreply.github.com>
cost_discount_config keys may now be <provider>/<model-pattern> with fnmatch globs, so a discount can target a subset of a provider's models. An exact pattern beats a glob, the glob with the longest literal wins, and the bare provider key is the fallback. PATCH /config/cost_discount_config validates the provider half only, and the admin UI cost tracking page gains a Model pattern field and a Models column
Resolves LIT-8465
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(decisions): bill decisions and systemone from <provider>/decisions/<model> cost map rows
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(decisions): mark decisions pricing rows with mode decisions
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): sync generated API types for decisions mode
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(decisions): update pricing expectations in proxy tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(decisions): price classifier calls from decision rows
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(health): drop redundant evaluation handler after main merge
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(decisions): inject pricing row for deployed classifier
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(decisions): ship the decisions pricing lookup without cost map rows
Restore both cost maps to main. Tests inject their own
<provider>/decisions/<model> rows. The Add Model dropdown hides those
pricing keys and offers mode decisions instead of evaluation.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(ui): expect the Decisions mode option in the Add Model form
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(decisions): bill decisions and systemone from _decisions cost map fields
Replace the provider/decisions/model pricing rows with <field>_decisions
fields on the model's own row. cost_per_token overlays them for the
decisions, adecisions, systemone and asystemone call types and prices
through generic_cost_per_token; chat, responses and batch keep the plain
fields. gpt-6-luna gains its Decisions rates (input 1e-07, output 0,
cached and cache-write input 1e-07, 2x above 272k tokens).
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(decisions): satisfy type gate, schema sync and test_utils schema for _decisions fields
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(decisions): import cost_calculator module to keep basedpyright unknown-variable count flat
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(decisions): drop stray fireworks import from __init__ and annotate test cost rows for the type gate
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): cover gpt-6-luna decisions and systemone above the 272k tier
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(decisions): keep the decisions pricing block clear of main's stale azure comment
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(cost_calculator): drop stale azure models doc comment
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): skip failure rows for expired Admin UI session keys on dashboard info routes
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): mark DB-boundary write assertion as test-quality-ok
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(decisions): support Clef-omni audio and video
* fix(decisions): accept Clef-omni media types with parameters and cover the proxy routes
The clip patterns now mirror Cloudflare's input schema, so a data URL or
content_type carrying parameters like audio/webm;codecs=opus is accepted.
The four proxy routes get media tests of their own, the backup cost map
is synced with the new Clef-omni rows, and the docs page moves to the
docs repo as a follow-up.
* fix(decisions): refuse parameterized media-type data URLs and refresh the OpenAPI snapshot
Cloudflare answers 422 to a data URL whose media type carries parameters even though its schema only requires the data:audio/ prefix, so LiteLLM now refuses those with a 400 before the call. The object form keeps passing parameters through, which Cloudflare accepts. The lazy OpenAPI snapshot and the dashboard API types are regenerated for the new audio and videos fields
* fix(decisions): gate media by cost-map flags, treat null clips as none, drop unsupported clips before validation
* fix(decisions): keep the SDK's positional order and gate media flags on evaluation-mode models
* fix(decisions): gate media on both decisions model modes and move the audit cells onto the harness helpers
* test(decisions): cover the Fireworks System One router in the media refusal cells
---------
Co-authored-by: moyai-devin-berriai[bot] <336287033+moyai-devin-berriai[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* test(managed_files): pin ownership checks on batch and fine-tuning create
Tenant B can create a batch or fine-tuning job from tenant A's managed
file id today. These fail on main for acreate_batch input_file_id and
acreate_fine_tuning_job training_file and validation_file
* fix(managed_files): check input file ownership on batch and fine-tuning create
Batch create and fine-tuning create only mapped the managed input file to
provider ids and never checked who owned it, so any key with access to the
model could build a batch or job on another tenant's file and then own the
outputs. Both now run check_file_ids_access like chat and responses do,
which 403s a non-owner and 404s a managed id with no ownership row.
Fine-tuning also checks validation_file, which was never looked at
* test(batches): pin that a pre-call ownership denial stops every batch create branch
LIT-4797 noted an endpoint-level check missed the load-balanced branch
because it dispatches first. The ownership check lives in the pre-call
hook, which runs before branch selection, so a 403 there must keep both
the load-balanced and unified-file branches from reaching the router
* test(integration): pin managed-file ownership on batch and fine-tuning create
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(managed_files): check ownership before the batch rate limiter reads the input file and keep key-owned resources across key regenerate
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: migrate proxy security, schema migration and MCP legacy tests
* test: run the MCP proxy e2e upstream under the SDK1 peer in integration-mcp
* test: warm the baseline store before timing the locked projection
* test(e2e): hold the stack exclusively while the semantic filter flag is flipped
* test(mcp): scope the in-process proxy e2e fixtures to their module
The session-scoped autouse fixture deleted DATABASE_URL and overrode LITELLM_MASTER_KEY for the rest of the worker, which broke every later test in the integration mcp suite
* test(e2e): assert the semantic filter keeps listed tools instead of a ranking
Which Datadog tool an OpenAI embedding ranks first is not something litellm owns, so the test now checks that the kept tools are a non-empty subset of the key's listing within top_k
* test(mcp): keep the in-process proxy e2e tests on one worker
loadgroup spread the file's tests across workers and interleaved them with other files, so the module-scoped in-process proxy was torn down and restarted between tests and requests timed out
* test(mcp): name the upstream requests that carried the wrong token
The bare all() assertion hid which request broke the forwarded token, which made an intermittent CircleCI failure undiagnosable
---------
Co-authored-by: yuneng <yuneng@berri.ai>
* refactor(rust_bridge): group the Python modules Rust imports by name under rust_bridge/host
Rust reads Python-owned state (legacy callback registries, model capabilities) through
modules it imports by path. Put them in one folder with an AGENTS.md that states the
contract, and update the module path constants on the Rust side.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* refactor(rust_bridge): route Python state reads through host accessors
* docs(rust_bridge): point test_support at the moved host contract test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(rust_bridge): move secret manager, transport and timeout accessors under host
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(greptile): register rust_bridge/host AGENTS.md in files.json
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: migrate tests/proxy_behavior into integration and unit tiers (partial)
* test(integration): run settings-dependent management tests on an owned proxy
* test(integration): assert stored values in the rewritten management rows
* test(integration): pin sibling team allocations against the org guaranteed throughput cap
* test(integration): keep Prisma clients out of the shared integration conftest import path
---------
Co-authored-by: yuneng <yuneng@berri.ai>
* ci: run every CircleCI check from the integration workflow
* test(ci): keep the startup test CircleCI-owned
* test(ui): find reset log rows by request id so concurrent workers cannot page them out
* test(ui): retry the key budget window spec until the draft refresh fix lands
* ci(ui): set Playwright workers and retries on the CircleCI command only
The shared playwright.config.ts also drives the Buildkite UI suite, where 3 workers
ran out of memory on one of two runs. Buildkite keeps main's 1 worker and 2 retries
---------
Co-authored-by: yuneng <yuneng@berri.ai>
* test: migrate legacy unit dirs and unrun folders into tests/unit, tests/integration and tests/e2e
* test: fix disk cache collection, ungate saturation check, drop RSS memory tests
* test: assert one delivered log per completion in Datadog and LangSmith checks
* test: drop SDK-direct e2e copies and align cost-map path filter cases
---------
Co-authored-by: yuneng <yuneng@berri.ai>
* fix(home): serve the bundled whats_new.json by default so a release shows only its own launches
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit 4dd96800e8)
* style: annotate whats_new_url with str | None
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit d1ae7ea085)
* test(home): pin whats_new_url to the env var and regenerate the dashboard API types
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit 4bfec53865)
* test(public_endpoints): prove the /public/whats_new route serves the bundled list when no URL is configured
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(public_endpoints): drop the unused os import
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(home): pin whats_new_url to the env var so a baked-in default URL fails the suite
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(tests): sort the import blocks ruff flagged in test_public_endpoints
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(home): let the route test read the real whats_new_url default so a baked-in URL fails it
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: move guardrails, image, audio, batches, search legacy tests to unit
* test: restore the custom guardrail and remaining chunk 1 unit rows
* test: move the ocr matrix to e2e
* test: drop comments from moved tests and fix vertex ocr credentials
* test: move litellm utils and unified google legacy tests to their tiers
* test: move logging callback and pass-through unit legacy tests to their tiers
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: move store_model_in_db legacy tests and start the admin ui migration
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: finish the proxy job rows and drop their CircleCI jobs
Moves the remaining multi-instance, openai endpoint, otel, pass-through and
top-level proxy tests into tests/integration with scripted upstreams, rewrites
the weak replacements to assert the outbound wire and exact responses, and
removes the build_and_test job group this PR owns from CircleCI
* test: tighten the key generation and error log assertions, rename migrated unit files to their mapped names
* test: shard the pagerduty unit tests and record the rust harness file on the coverage allowlist
* test: drive the SSO sign-in tests through a scripted IdP and isolate the cost center stand-in
* test: type the e2e beta header, prompt cache and secret manager values
* test: assert the missing model error message on the OpenAI routes
* test: drop the deleted otel base class import and restore mcp redaction and exact langfuse usage checks
* test: make the langfuse fixture directory a package for the layout check
* test: restore collected no-match cases, real langfuse and gcs payload checks, and gate the aws secret lane
* test: keep the ocr e2e pdf fixture next to the test
* test: put meta below covers on the streaming tool search e2e
* test: run the OCR matrix against a scripted upstream and fix two runner-only e2e failures
* test: count only OCR calls in the OCR matrix recorder
* test: pin OCR parse bodies and billed cost, group new e2e cases into classes
* ci: restore the python image anchor dropped in the main sync
---------
Co-authored-by: yuneng <yuneng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(databricks): call the ai_decide AI Function route for decisions and the auto-router
Databricks serves ai_decide at POST <host>/api/2.0/ai-functions/ai-decide, not
on a serving endpoint's /invocations route. The decisions provider and the
auto-router classifier now accept only the ai_decide model, post to that route
without a model field, unwrap the response envelope, and map a noul answer's
probability to the System One noul field.
* test(databricks): bound the megabyte model name test by scaling, not wall time
* test(databricks): assert the whole OpenAI-format decision and floor the liveliness bound
* test(decisions): move the Databricks OpenAI-format wire row to the ai_decide route
* fix(databricks): keep serving endpoint decisions beside the ai_decide route
* test(databricks): assert the whole classifier request body in the test routing case
* test(databricks): correct System One spend call type
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): load the Home page What's new launches from /public/whats_new at runtime
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): show the three newest What's new launches
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* build: ship whats_new.json in the wheel
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(decisions): rename the evaluation model mode to decisions
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(decisions): canonicalize the health check mode without rebinding
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(decisions): replace canonical_model_mode with is_decisions_model_mode
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(decisions): preserve evaluation mode compatibility
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore: drop the rebuilt dashboard bundle from this PR
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(dashboard): format decision mode helper
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(headroom): drop null tool_calls before calling /v1/compress
headroom-ai 0.27.0 through 0.30.0 iterate msg.get("tool_calls", []) and fail
with 'NoneType' object is not iterable when an assistant row carries an explicit
tool_calls: null, which the fail-closed guardrail then surfaces as a 502. Strip
the null key from rows sent to the compression service; real tool_calls lists
are forwarded unchanged
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(headroom): pin that SDK-echoed null tool_calls never reach /v1/compress
Main now withholds every assistant row from /v1/compress (#39974), so the
null tool_calls strip added here never runs on a row headroom-ai 0.27-0.30
fails on. Drop the strip and pin the customer's history shape through the
public OpenAI chat translation handler instead.
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: yucheng-berri <293926326+yucheng-berri@users.noreply.github.com>
* feat(ui): add a Home page as the dashboard root with usage summary, recent launches, blog updates and a model leaderboard
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): fall back to spend-log request counts on Home when the gateway has recorded none
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(ui): cover the Home page totals, launches, leaderboard and updates feed
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): render Home Updates as rows and return up to 6 blog posts from /public/litellm_blog_posts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): filter Home Updates by the new-model and api-change blog tags
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(blog): import NotRequired from typing_extensions for py3.10 and test the endpoint passes every fetched post through
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* revert(ui): drop the Updates filter chips, keep Updates as a plain row list
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(ui): restore generated schema.d.ts to main
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): label the Home key button Create New Virtual Key
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(ui): drop the What's new sublabel on Home
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): show an error with retry when Home usage fails to load and move the Home test to the integration tier
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(dd_span_tagger): emit litellm_user_email span tag for JWT-authenticated requests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit b4f9e14a44)
* refactor(dd_span_tagger): use dotted litellm.user_email tag for consistency
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit 819faa01c3)
* feat(model_hub): surface model_info.description in model group info and Model Hub UI
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit b224b15b9d)
* refactor(router): aggregate model group description without in-place mutation
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit 342b93fe0a)
* test(dd_span_tagger): assert the user_email tag on the active span instead of on the patched setter
The two ported tests only inspected the mock that replaced the tag setter, which the test quality gate counts as two new mock-echo tests over main. They now read the tags that land on the active span
* refactor(router): read the model group description through the typed model_info reader
The ported lookup read model_info off the untyped deployment dict, which added three basedpyright errors over main. It now goes through the existing validated model_info reader
---------
Co-authored-by: milan <milan@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: ryan-crabbe-berri <ryan@berri.ai>
* fix(guardrails): scan or refuse attachments in the bedrock guardrail
* fix(guardrails): cover native pre_call and computer screenshots in the bedrock attachment scan
* fix(guardrails): classify every attachment shape the bedrock guardrail can receive
* test(guardrails): type the attachment-scanning test guardrail
* fix(guardrails): keep base behavior for Bedrock subclasses and malformed messages
- skip the attachment scan when a subclass overrides apply_guardrail
- ignore non-string message role/type instead of raising
- treat a null Converse document/video/audio block as absent
- keep the base request-body debug log for text-only calls
* fix(guardrails): let text documents, unpadded base64 images and request-override subclasses through as before
* fix(guardrails): let Converse text documents through and refuse documents that carry images
* fix(guardrails): scan images inside content documents instead of refusing them
* fix(guardrails): scan text document content with the Bedrock guardrail
* fix(guardrails): scan document title, context and nested documents
* Expand nested content documents without recursion
* Keep attachment scan within the type-discipline budget
* Take the image format and pixel size from the image bytes
* chore(guardrails): satisfy type-discipline gate after merging main
* chore(guardrails): drop docstrings added by the attachment scan
* fix(guardrails): scan Converse text documents and guardContent images
A Converse document whose source is text (or a list of text parts) now
goes through the Bedrock text scan instead of being refused as
unscannable, so the pass-through request that worked before this PR
keeps working. A Converse guardContent image is now scanned like any
other image. Byte and S3 document sources stay unscannable.
Adds the integration test that drives every scanned endpoint through a
real proxy against a scripted Bedrock ApplyGuardrail stub, and retypes
the touched hook signatures for the type gates.
* test(guardrails): cover Bedrock checks mode in the attachment integration test
Adds a checks-mode proxy to the rig: images are refused without calling Bedrock on every route, and a Converse text document is sent to InvokeGuardrailChecks.
* fix(guardrails): scan a Converse document's name and context whatever its source
A Converse PDF or S3 document let through by skip_unscannable_attachments now still has its name and context scanned as text, the same as the Anthropic document path.
---------
Co-authored-by: yucheng-berri <293926326+yucheng-berri@users.noreply.github.com>
* fix(credentials): keep credential labels and edits consistent across UI, API, and replicas
The credential modal flattened stored credential values into the same form object as the
credential's own name, label, and provider, so a stored value keyed credential_name or
display_name replaced the real one and the edit went to the wrong credential or relabeled it.
The modal now submits the top-level fields and credential_values separately
PATCH /credentials now reads the row it merges onto from the writer, so an edit inside the read
replica's lag window no longer writes back an older label or key
POST /credentials answers 409 for a name defined in config instead of creating a DB copy that
shadows it
display_name rules live on the server only. It rejects control, formatting, and line separator
characters, and the UI stops enforcing its own length check and shows the server's error
The models table, model detail view, and model edit picker show the credential's technical name
under its label whenever the two differ
* fix(ui): show credential source in its own column like models
The config marker sat under the credential name. The credentials table now has a Source column
with a Config or DB status badge, matching the Source column on the models table
* test(credentials): unit-test label and prefill helpers and keep the full credential name on hover
Moves displayNameChange and initialFormValues out of the modal so their rules are unit tested,
keeps only the wiring cases in the modal's integration test, makes the label acceptance test
write through the real credentials repository, and adds the full credential name as a hover
title where the models pages truncate it
* feat(guardrails): add decision_model guardrail for decisions-API models
Adds a new guardrail provider that scores request and response text with a
decisions-API model (e.g. Jev) against named predicate checks. Each check
carries its own threshold and action (block or log); a flagged block check
rejects the request with a 400 naming the flagged checks, and call failures
honor unreachable_fallback (fail_open or fail_closed).
Ships four preset checks for the Admin UI (prompt injection, jailbreak,
system prompt extraction, data exfiltration) served from
/guardrails/ui/add_guardrail_settings, plus a Decision Model step in the
add-guardrail form filtered to evaluation-mode models.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(guardrails): fail closed on verdict errors and drop comments in decision_model
* feat(guardrails): cap decision_model input with head and tail elision
* fix(guardrails): list only block checks in decision_model 400s and tighten inputs
* feat(guardrails): honor timeout, cap checks at MAX_DECISION_QUESTIONS, and reject duplicate check names
* fix(guardrails): decorate decision_model apply_guardrail with log_guardrail_information
* fix(guardrails): suppress untyped-decorator pyright error on log_guardrail_information import
* feat(guardrails): screen each text in overlapping chunks with a shared decisions-call semaphore
Replace head/tail elision with per-text screening: each non-empty text is
sent to the decision model on its own, split into overlapping chunks when
it exceeds max_input_chars, and all calls run under a per-event-loop
semaphore shared across requests (max_concurrent_decision_calls,
default 8). Check verdicts aggregate the max probability across calls; a
missing predicate answer on a Block check blocks under fail_closed and
only warns under fail_open, while log-only checks are never flagged for
it. The 400 detail now lists flagged check names only. The provider
combobox also matches Decision Model on jev, typesafe, prompt injection,
jailbreak, and decision
* fix(ui): align decision model custom check rows with preset rows
* fix(ui): move decision model custom check remove button to the row end
* style(ui): run prettier on decision model check files
* fix(ui): inline decision model evaluation-mode filter after lens module removal
* fix(guardrails): keep decision model detections when a chunk call fails
* fix(ui): require an enabled decision model check before create
* fix(guardrails): screen decision model texts sequentially like presidio
* feat(guardrails): replace decision model presets with provider picker and per-question test
* fix(ui): render decision model question test as a full-width row
* feat(ui): add run-history test section to decision model form
* fix(ui): prevent threshold overlap and show test shortcut
* feat(guardrails): default decision model threshold to 0.7
* test(ui): expect decision model default threshold 0.7 in characterization payload
* feat(guardrails): screen tool call arguments with the decision model
* refactor(guardrails): freeze the decision model config model
* feat(guardrails): list only evaluation-mode deployments in the decision model picker
* feat(guardrails): hide the decision model questions table until a question exists
* style(ui): format DecisionModelFields
* style(ui): format decisionModelQuestion tests
* refactor(guardrails): type the tool call function extraction
* refactor(guardrails): build tool call texts without a mutable accumulator
* refactor(ui): share isDecisionMode for the decision model picker
* fix(guardrails): screen prior Responses function call arguments in the decision model guardrail
The Responses input handler leaves function_call arguments out of texts and tool_calls, so a prompt injection in a prior call's arguments passed unscreened. Assistant tool calls from structured messages are now screened too, deduplicated against the request's tool calls
* test(integration): send instructions with the decision model guardrail check
Checks require instructions, so creating the guardrail returned 422 and the test never sent a chat request
* fix(ui): score the decision model test preview the way the guardrail does
An unanswered block question now shows Block, disabled questions no longer count, and a run is hidden once the model or a question's text changes since it was scored
* fix(ui): clear the decision model error on pick and list questions under Add a question
Picking a model after a failed Create left "Select a decision model" on screen, because the form only validates on submit. The field now revalidates when it already shows an error. The Questions table and its heading appear only once a question exists, below the Add a question box
* test(ui): move the decision model form tests into an integration test file
They render the whole AddGuardrailForm, so they belong in a .integration.test.tsx file. Plain fields are now filled with fireEvent.change, and the two existing form test files go back to main apart from the modelHubCall mock the form now needs
* test(guardrails): build the decision model test params without mutating a local
* fix(guardrails): reject list checks outside decision_model and honor streaming scan settings
- LitellmParams now rejects a list `checks` for any guardrail other than
decision_model, so a Bedrock guardrail with list checks gets 422 again
instead of being stored and failing every request
- The decision model guardrail reads streaming_end_of_stream_only and
streaming_sampling_rate, so a streamed post_call response can be scored
once at end of stream instead of every 5 chunks
- max_concurrent_decision_calls is described as a per-worker limit
* fix(ui): match guardrail providers like the default picker filter
- The provider search uses the Base UI collator filter on the label and
the aliases, so searches such as presidio_pii or llm_as_a_judge match
again, as they did before the alias filter
- Model groups from /model_group/info are parsed defensively and after the
providers are populated, so a malformed response cannot empty the
provider list
* fix(ui): edit decision model questions in place and show passing test chips in green
Questions start as an Add question button. Each click adds a blank question
card with its name, text, action and threshold, editable until the guardrail
is created, mirroring the per-model budget editor. Create and Next reject a
blank or duplicate enabled question. Test run verdicts and chips use the
success tone for Pass, the error tone for Block and the warning tone for a
flagged Log only question
* fix(ui): drop the per-question enable checkbox from the decision model form
A question is either in the list or removed with its X, so every listed
question is sent and scored. The card now labels Name and Question the way
it labels Action and Threshold
* fix(ui): give the question name field the full card width
* fix(ui): show the provider logo on the selected decision provider
* fix(ui): show the full threshold slider track on decision model questions
* feat(ui): add decision model cards to the guardrail garden
Adds a Decision Model section to the guardrail garden with TypeSafe Jev,
Perplexity, Microsoft, OpenAI and Databricks cards. Each card opens the
create form scoped to that provider's decision models.
The create form drops the Decision Provider picker and lists every
decision model on the proxy with its provider logo, prompts to add a
model when none is available, and says that test runs are billed and
logged.
* fix(ui): shorten the decision model test hint to one line
* fix(ui): prefill the decision model from the garden card's provider
* chore(ui): regenerate the dashboard API types after merging main
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: yucheng-berri <293926326+yucheng-berri@users.noreply.github.com>
* fix(proxy/auth): expand all-team-models sentinel for project model access checks
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(e2e): use CI-served models and tighten project all-team-models denial check
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(e2e): declare coverage registry cells for project all-team-models tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* ci: retrigger e2e after ephemeral stack deploy timeout
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(e2e): wait for team allowlist propagation before project denial check
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(e2e): annotate written_at as Final per LIT010
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: jesus <jesus@berri.ai>
* fix(proxy): only pass the JSON-RPC method to MCP admission auth
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): expect method-only MCP admission body
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* docs(proxy): describe /v1/decisions and /v1/systemone in Swagger UI
Add summaries, descriptions, inlined request body schemas with examples and typed 200 responses to the
decision routes and their aliases, so the OpenAPI document and the lazy snapshot show both request formats
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): cap _inlined schema walk depth for the recursion check
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(types): move LitellmUserRoles and KeyManagementRoutes into litellm.types.proxy.auth.user_roles
Part of LIT-9423
* refactor(types): move is_jwt into litellm.types.proxy.auth.user_api_key_auth
Part of LIT-9423
* refactor(auth): alias is_jwt import so JWTHandler.is_jwt is not read as recursive
* refactor(types): type UserAPIKeyAuth, its token view and hash_token in place
Part of LIT-9423
* refactor(types): move UserAPIKeyAuth, its token view and hash_token into litellm.types.proxy.auth.user_api_key_auth
Part of LIT-9423
* fix(types): keep agent, budget snapshot and is_jwt names importable from proxy._types
Part of LIT-9423
* refactor(core_utils): move resource_ownership out of the proxy
Part of LIT-9423
* fix(types): restore main's _types.py content alongside the user_api_key_auth move
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(sdk): resolve litellm.proxy lazily on attribute access
The layer cleanup stopped import litellm from loading any proxy module, so litellm.proxy was no longer bound and litellm.proxy._types.X raised AttributeError. The module __getattr__ now imports litellm.proxy on first access, which keeps import litellm free of proxy._types and fixes the docs and code-quality checks on main
* fix(sdk): resolve litellm.proxy lazily on attribute access
The layer cleanup stopped import litellm from loading any proxy module, so litellm.proxy was no longer bound and litellm.proxy._types.X raised AttributeError. The module __getattr__ now imports litellm.proxy on first access, which keeps import litellm free of proxy._types and fixes the docs and code-quality checks on main
* style(types): drop blank lines between proxy._types re-export imports
Part of LIT-9423
* refactor(proxy): keep only user_api_key_auth re-exports in _types.py
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(proxy): carry main's UserAPIKeyAuth field into moved file
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(types): drop blank lines between proxy._types re-export imports
Part of LIT-9423
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(types): move LitellmUserRoles and KeyManagementRoutes into litellm.types.proxy.auth.user_roles
Part of LIT-9423
* refactor(types): move is_jwt into litellm.types.proxy.auth.user_api_key_auth
Part of LIT-9423
* refactor(auth): alias is_jwt import so JWTHandler.is_jwt is not read as recursive
* refactor(types): type UserAPIKeyAuth, its token view and hash_token in place
Part of LIT-9423
* refactor(types): move UserAPIKeyAuth, its token view and hash_token into litellm.types.proxy.auth.user_api_key_auth
Part of LIT-9423
* fix(types): keep agent, budget snapshot and is_jwt names importable from proxy._types
Part of LIT-9423
* fix(types): restore main's _types.py content alongside the user_api_key_auth move
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(sdk): resolve litellm.proxy lazily on attribute access
The layer cleanup stopped import litellm from loading any proxy module, so litellm.proxy was no longer bound and litellm.proxy._types.X raised AttributeError. The module __getattr__ now imports litellm.proxy on first access, which keeps import litellm free of proxy._types and fixes the docs and code-quality checks on main
* style(types): drop blank lines between proxy._types re-export imports
Part of LIT-9423
* refactor(proxy): keep only user_api_key_auth re-exports in _types.py
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(proxy): carry main's UserAPIKeyAuth field into moved file
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(types): drop blank lines between proxy._types re-export imports
Part of LIT-9423
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(otel): let a team strip captured message content from its own OTel v2 destination
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(proxy): split the shared callback option conflict check and sync dashboard API types
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(otel): address review feedback on team capture_message_content
Classify capture_message_content as NotSecret in the credential slot registry, inject test exporters through register_exporter_factory instead of patching _exporter_from_spec, type the test helpers, and stop reassigning processor in build_tracer_provider
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(otel): add integration cells for team capture_message_content
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(otel): cover weave_otel and newrelic team destinations in capture_message_content cells
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(otel): stop rejecting client-sent capture_message_content at the request body boundary
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(otel): let a team capture_message_content override the global setting in both directions
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(otel): keep a team's capture_message_content on its own OTel v2 destination
The setting no longer rides the flattened team callback vars, so a sibling destination and a routed provider follow the global mode. Request bodies get the base 400 back, a failure-only entry rejects the setting, and entries for one backend must agree
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(otel): type the capture_message_content callback validation tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(otel): type the remaining capture_message_content test signatures
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(otel): mark the new capture_message_content test locals Final
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(otel): default tenant capture to callback_settings.otel and keep CaptureMessageContent formatting bare
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(otel): keep CaptureMessageContent a plain str like main
The str-to-Enum switch changed the public type: the config field's type,
repr and model_dump, a constructor that now raised on unknown values, and
unknown env values coerced to no_content. CaptureMessageContent is the plain
str subclass main ships again, the field is a str, and a
captures_span_content helper replaces the enum property. A test pins that a
published preset's capture mode stays the default for an unowned
destination, which is what main forwards.
* test(otel): drop the module-level CaptureMessageContent import the rewritten test no longer uses
* fix(lint): keep the merged branch within main's type budgets
Freeze the capture-mode settings model (LIT015) and read stored logging
entries through one typed sequence helper, so the conflicting-capture checks
add no unknown types.
* fix(otel): reject a destination capture_message_content while OTel v2 is off
The legacy OTel callbacks never read the per-destination setting, so a team
could register no_content and still export prompts and replies under the
global capture mode. Registration now refuses the var unless the proxy runs
with LITELLM_OTEL_V2=true, the same rule the Arize transport and New Relic
routing vars follow.
* test(otel): type the OTel v2 fixtures and the v2-off regression test
* fix(otel): accept only no_content and span_only on a team or key destination
OTel v2 writes captured content only to span attributes, so event_only
behaved like no_content and span_and_event like span_only on a
destination. The proxy-wide setting keeps all four values
* docs(otel): say an omitted capture setting follows the proxy's own callback for that backend
* test(otel): describe the general capture fallback in the unowned destination test
---------
Co-authored-by: mrinal <mrinal@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: yucheng-berri <yucheng@berri.ai>
Co-authored-by: yucheng-berri <293926326+yucheng-berri@users.noreply.github.com>
* refactor(anthropic): move prompt cache pricing helper out of the proxy package
Part of LIT-9423
* fix(sdk): resolve litellm.proxy lazily on attribute access
The layer cleanup stopped import litellm from loading any proxy module, so litellm.proxy was no longer bound and litellm.proxy._types.X raised AttributeError. The module __getattr__ now imports litellm.proxy on first access, which keeps import litellm free of proxy._types and fixes the docs and code-quality checks on main
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(types): move complexity router request models next to their base config
Part of LIT-9423
* test(types): cover the old auto-router import path for the moved request models
Part of LIT-9423
* refactor(anthropic): move prompt cache observation lookup out of the proxy hook
Part of LIT-9423
* test(internal_context): exempt moved prompt cache lookup helper from key-family check
* feat(proxy): require trace IDs per team on LLM, MCP and agent requests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): assert trace ID authorization outcomes
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): clear trace ID type gate errors
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(proxy): format trace ID type check
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): validate trace ID metadata
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): use read-only trace metadata types
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): skip trace ID metadata validation for teams without dict metadata
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* perf(proxy): check team trace ID setting before route matching
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
