* refactor(types): move LitellmUserRoles and KeyManagementRoutes into litellm.types.proxy.auth.user_roles
Part of LIT-9423
* refactor(types): move is_jwt into litellm.types.proxy.auth.user_api_key_auth
Part of LIT-9423
* refactor(auth): alias is_jwt import so JWTHandler.is_jwt is not read as recursive
* refactor(types): move complexity router request models next to their base config
Part of LIT-9423
* test(types): cover the old auto-router import path for the moved request models
Part of LIT-9423
* refactor(anthropic): move prompt cache observation lookup out of the proxy hook
Part of LIT-9423
* test(internal_context): exempt moved prompt cache lookup helper from key-family check
* feat(proxy): require trace IDs per team on LLM, MCP and agent requests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): assert trace ID authorization outcomes
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): clear trace ID type gate errors
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(proxy): format trace ID type check
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): validate trace ID metadata
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): use read-only trace metadata types
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): skip trace ID metadata validation for teams without dict metadata
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* perf(proxy): check team trace ID setting before route matching
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Each spend-log delete batch and the remaining-rows probe ran inside a
prisma interactive transaction with the default 5s timeout while setting a
30s Postgres statement and lock timeout. A batch that waited on a lock for
more than 5s was closed by the query engine with a 422 and rolled back, and
after three such batches the cleanup run aborted with nothing deleted.
Reuse the partition manager's bounded_tx, which sizes the prisma timeout to
the statement bound plus commit slack, for the delete batch and the probe.
Resolves LIT-8686
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): simplify playground decision endpoint
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): expose both decision API shapes in playground
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): omit undiscovered decision models from examples
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): align decisions form and response views
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* perf(ci): distribute unit cases and reuse security sweep connections
* revert(ci): reject first speed trial after new failures
* perf(ci): avoid repeated test discovery and parsing
* perf(ci): isolate parallel security workers
* revert(ci): restore measured security execution isolation
* perf(tests): reuse source scans and deduplicate cache cleanup
* perf(ci): restore parallel security with safe log batching
* test(ci): isolate fixture state and schema comparisons
* test(ci): make schema and scheduler fixtures deterministic
* test(ci): drain queued logging before fixture teardown
* test(ci): preserve logging delivery without idle worker restarts
* test(ci): keep guardrail sinks available during parallel bursts
* test(ci): isolate full proxy lifecycle from gateway imports
* test(ci): preserve first failure responses and sweep evidence
* test(ci): restore documentation URL settings between cases
* test(ci): isolate open MCP tools in private gateways
* test(ci): group private MCP cases and handle exited workers
* test(ci): retain proxy exits and MCP failure details
* test(ci): repair merge interactions with main
Key the schema shard's full clone on artifact-name, since main removed the
unit-flag input. Give each owned Redis its own log file again. Hand the
realtime transcription tests main added a logging object whose dispatch
handlers are awaitable, so the per-test logging drain no longer waits on a
MagicMock task forever
* perf(ci): shard owned integration cases and close lifecycle leaks
* fix(ci): qualify shard execution and harden interrupted cleanup
* fix(ci): bound worker supervision and own fault traffic
* perf(ci): reuse unparametrized fixtures across bounded case groups
* fix(ci): preserve boot fault allowances and complete exit records
* test(ci): tolerate startup retries and retain readiness details
* perf(ci): balance integration shards by recorded duration
* fix(ci): load migration database helpers only for workers
* test: move packaging and install tests to unit and integration
* test: move packaging and install checks to smoke and integration
* test: keep the boot check proxy on its scratch database in replica mode
* test: restore install imports and address review findings on packaging checks
---------
Co-authored-by: yuneng <yuneng@berri.ai>
* test: move routing, caching and callback tests out of local_testing
Moves the 37 routing, caching and callback files in tests/local_testing into mirrored tests/unit paths and tests/integration sdk and security groups per the migration manifest, and deletes the 8 rows it marks for deletion
* test: cover least-busy routing and tighten migrated router, cache and pass-through tests
Add router-level least-busy tests for the three legacy rows that had no
destination, move completion fallback and model alias tests to their mirror
files, move router batch tests next to the router, and strengthen tests
whose mutants survived (specific deployment, batch pair grouping, custom
cache key, router cache ttl). Add a unit test for configured pass-through
Authorization forwarding.
* test: tighten migrated callback, proxy config and cache ttl tests
Assert the full StandardLoggingPayload key set and the wire user on callback
tests, add an async embedding failure callback test, drive the prometheus
redis failure through a real RedisCache, use the real salt key in proxy
config tests, pin the redis ttl value and rename fallback test helpers
* test: fix semantic cache assertions and order-dependent router and callback tests
Count semantic cache embeddings on the route, drain async cache writes before
the second lookup, echo the requested model from the callback provider mock so
response_cost does not depend on model_cost entries other tests register, and
move the two-key authorization assertion to the auth failure test it belongs to
* test: cover model_list fan-out and the usage-based routing tpm limit lost in the move
* test: wait for the streamed cache entry in redis and drop deleted files from the hosted mock guard
* test: restore sync callback, audio redaction and langfuse export coverage, keep redis-compat filter on the moved file
* ci: run the remaining router-keyword legacy tests on one node
* test: refuse the redis write in-process instead of resolving an invalid host
---------
Co-authored-by: yuneng <yuneng@berri.ai>
* feat(guardrails): split logging_only_scope direction from input-failure policy
logging_only_scope is now direction-only (input, output, unset for both
directions). The new logging_only_continue_on_input_failure flag controls
whether a flagged or raising logging_only input scan still lets the
response be scanned. "both" stays accepted everywhere as a deprecated
alias for unset scope plus the flag set to true, normalized at callback
init while stored rows keep the written value.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(dashboard): narrow continue-on-input-failure update type and stay under guardrail_info max-lines
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(dashboard): keep custom-code guardrail controls readable with logging_only
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(dashboard): scope continue-on-input-failure to the default direction and restore verdict assertion
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(guardrails): remove the both value from logging_only_scope
logging_only_scope only shipped its both value in dev prereleases, so it is
dropped instead of kept as a deprecated alias. Leaving the scope unset and
setting logging_only_continue_on_input_failure gives the same behavior. A
stored both value is now treated like any other invalid scope: it is logged
and ignored on load, and rejected with 422 on create or update.
* fix(guardrails): tolerate an invalid logging_only_scope in the v1 guardrail list
A config.yaml guardrail whose logging_only_scope is no longer valid, such as
the removed both value, made GET /guardrails/list return 500 for every
guardrail. The v1 list now parses params with the same tolerant parser as
the v2 list, info and PATCH paths, so the bad value is logged and dropped.
* test(guardrails): drive continue-flag rejection tests through the real guardrail handler
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: yucheng-berri <293926326+yucheng-berri@users.noreply.github.com>
* fix(key mgmt): a project may only be attached to keys of its own team
A project is created under exactly one team, and its budget and models are
validated against that team's. Nothing checked that a key's team matched, so
`POST /key/generate` with `team_id: team-a` and a `project_id` owned by
`team-b` answered 200 and stored exactly that — a key belonging to team-a,
recorded under team-b's project and validated against its models and budget.
The same held with no team at all.
It needs no proxy-admin rights: an admin of team-a who is a member of no other
team can issue keys under another tenant's project with nothing but the project
id, and that tenant sees the spend without having granted anything.
`_check_project_key_limits` now takes the key's team and refuses a project
owned by a different one, before the model and budget checks so the refusal
reads as what it is. A project with no owning team is left alone — nobody owns
it, so there is no boundary to cross. `/key/update` passes the key's stored
team when the request does not carry one, and now runs the check whenever the
project itself is set or changed, which a request that moves only the project
previously skipped.
Two existing cells needed the project's team passed explicitly: they measure
the model allowlist, not tenancy, and would otherwise have been asserting model
behaviour on a request the new gate refuses. A third builds an unowned project
for the same reason.
Fixes#41089
* fix(key mgmt): check project ownership on every key mutation path
/key/update ran the project check only when project_id, models or
max_budget were supplied, so a request that changed team_id alone left a
foreign project attached. /key/regenerate never ran it at all. Both now
go through one helper that validates the key as the mutation leaves it.
On /key/generate the check moves after default_key_generate_params is
applied, because that can supply team_id; it was rejecting a valid key
whose team came from the defaults.
Tests move into the mapped test file per CLAUDE.md.
* test(key mgmt): read the project from the cache instead of patching the lookup
The test-quality gate rejected the new cells: 12 TQ008 for patching
`get_project_object`, an SDK internal, and 4 TQ001 for accept controls that
could only fail by raising.
The cells now seed `UserApiKeyCache` with the project, which is the idiom
the neighbouring project cells already use and removes the patching. The
accept controls are parametrised together with the rejecting ones, so each
test function carries a real assertion. The three regenerate seams that stay
stubbed each carry a reason.
* fix(proxy): prevent cross-team project keys
Co-authored-by: L4XB <lukas.buck@e-mail.de>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): validate bulk key team changes
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): use behavioral assertions in project ownership tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): read project owner from the database for key ownership checks
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): read project ownership from the primary database under the lookup deadline
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): check project moves against the primary database team
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): run ownership checks after existing validation without the lookup deadline
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): type ownership writer reads
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): make ownership tests independent of runner salt and clock
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): check key project ownership after existing validation
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): run key project ownership right before the key row write
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): reject key project ownership before permission and budget writes
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): keep project object permission validation on the stored payload
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(proxy): format key regeneration update payload
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): invalidate the written permission id on bulk update and regenerate
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): audit project team ownership across endpoints, bulk paths and chaos
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): drop budget and permission rows written for a rejected cross-team key
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(keys): require project-team membership when regenerate binds a key to a team-owned project
/key/{key}/regenerate and /key/regenerate accepted a team_id + project_id
pair without checking that the caller belongs to the destination team, so a
user with no team could regenerate their own key into another team and its
project. Regenerate now requires a proxy admin or a member/admin of the
project's team whenever the change lands the key on a team-owned project.
Also adapts tests to main's premium_user_check rename, drops a duplicate
import left by the merge, and replaces the created object-permission id cast
with an isinstance check.
* fix(keys): 404 a missing project before any key write, and match the project move refusal to its siblings
A missing project_id on /key/service-account/generate and on both regenerate
routes fell through to a foreign-key 500; regenerate had already written a
deleted-token history row, and service-account generate left its budget and
object-permission rows behind. The ownership check now answers 404 with the
same message /key/generate uses, before any write, and the generate rollback
covers it. The /project/update move refusal now uses ProxyException
(bad_request, param team_id) like the delete-with-keys refusal.
Tests: integration coverage for a team admin regenerating a team key into
another team's project (403), the missing-project 404s with no history rows,
and the unit test that never reached the ownership check is removed.
* test(keys): keep the existing-permission case in the regeneration cache eviction test
Parametrize the test over a key that already has an object_permission_id and one that mints a new row, so eviction under the key's existing permission id stays covered.
---------
Co-authored-by: L4XB <lukas.buck@e-mail.de>
Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): list the decision providers from the cost map in the decision models banner
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): title-case decision providers the dashboard has no display name for
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(ui): wait for the catalog query to fail before asserting the banner fallback
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): replace Add a decision model with a Supported providers docs link in the decision models banner
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): drop the Add Model search hint from the decision models banner
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): call it the Decisions playground in the decision models banner
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): call the System One playground the Decisions playground across the dashboard
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): render the System One JSON editor with the stock One Dark theme
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): rename the Playground System One tab to Decisions and use the VS Code Dark+ JSON theme
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): keep the JSON editor header and issue list on the light surface
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(ui): drop the root block scanner test and the unused band styles
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): follow the dashboard theme in the JSON editor and drop the breakdown accents
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): round the Decisions score pill to two decimals
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): strip the Prism block backgrounds so the JSON editor theme swap stops the React style warning
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): let admins make per-model key/team rate limits a hard 429 instead of falling back
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): sync per-model rate limit API types
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): type per-model fallback setting access
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): keep the per-model hard cap inside the fallback loop and test it through the real limiter
* fix(proxy): list the per-model hard cap setting on the settings page and cover every per-model descriptor
* fix(proxy): read disable_fallbacks_on_per_model_rate_limits as a boolean value and cover the team-wide per-model cap
* test(proxy): cover per-model rate limit 429s skipping fallbacks on a live proxy
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
