83 commits

Author	SHA1	Message	Date
Mateo Wang	e333ccabdb	fix(scim)!: delete a group through the team delete's bulk path instead of one member at a time (#47544) ... * fix(scim): write a group's roster in one locked transaction instead of one member at a time POST, PUT and PATCH /scim/v2/Groups resolve every member in one query and write the roster, the membership rows and the users' team tags in one transaction under the team's advisory lock, so a 500-member push runs a fixed number of statements instead of thousands. PUT now answers with the roster it just wrote * test(scim): assert the roster the IdP gets back and boot the bulk roster cell with two workers * fix(scim): skip the team lock for a rename-only PATCH and lock member rows in id order before the roster write A PATCH naming no members opened the roster transaction and took the team advisory lock for nothing. A SIGKILLed worker then left its orphaned query engine holding that lock until the 60 s transaction timeout, and /team/delete, which takes the same lock, timed out behind it. An empty delta is now answered from a plain team read, with no transaction and no lock Two groups trading members in parallel locked user rows in opposite orders, and one request died with a Postgres deadlock. Every user row a push touches is now locked with SELECT ... ORDER BY user_id FOR UPDATE before the first write, so the two requests serialize The bulk roster cell gets a 120 s client through a new owned_proxy_process client_timeout, so an unfixed push fails on the statement count instead of the harness clock, requires positive counts on both pushes, and gains a parallel roster-trade test that answered 500 before the lock * fix(scim): evict the cached team on a group rename and read a no-op roster from the writer A rename-only PUT or PATCH wrote the new alias and left the team cached under its id and old alias on every worker, so keys kept logging spend against the old team alias until the TTL expired. The group write now evicts the cached team (and broadcasts the eviction) right after the alias or metadata update. sync_team_roster reads the team from the writer when the plan names nobody, so a replica cannot hand back the pre-rename row; a plan already matching the roster returns inside the lock without writing, evicting, or emitting the members metric; the response is built from the written row, so lastModified reflects the roster write; and detaching a removed member bumps the user's updated_at like every other user write. * fix(scim): bump detached members' updated_at in UTC The detach UPDATE wrote updated_at = CURRENT_TIMESTAMP, which Postgres casts into Prisma's naive timestamp column in the session timezone, so on a database whose session is not UTC the removed member's row landed hours away from the UTC instants Prisma itself writes. Write NOW() AT TIME ZONE 'UTC' instead, and have the bulk roster cell assert the bumped row sits between two reads of the database's own UTC clock taken around the replace. * fix(scim): delete a group through the team delete's bulk path instead of one member at a time * fix(scim): chunk the group write's member lookups under the bind cap Every Prisma "in" filter the group write issues bound one parameter per member with nothing bounding the list, which check_unbounded_in_lists reports as 11 new findings. The single-field lookups now go through litellm.repositories.chunked_in (caller_transaction inside the roster transaction, per_chunk_ok for the role recompute), and the one cross-field OR read slices its values at IN_LIST_CHUNK_SIZE and records the bound. A push of IN_LIST_CHUNK_SIZE + 1 members now reads in two slices, and a member named only in the last slice still resolves. * fix(scim): demote a deleted admin group's members before the team delete so a failed delete stays retryable * fix(scim): recompute member roles again after the team delete so a concurrent group write cannot regrant admin * fix(proxy): send bulk cache evictions in slices so the broadcaster drops none Veria pointed out that the invalidation publisher caps its pending tasks at 1,024 and drops the excess with a warning, and that a group push now detaches every removed member's keys in one call instead of one member at a time, so a removal of more than roughly 340 members (three cache keys each) silently left stale key objects on the other replicas. The merge base never hit the cap because each member was a separate request. Bulk evictions (team, user, organization, and session key deletes, plus the roster settle) now publish in slices of 256 and wait, bounded at two seconds, for the backlog already handed to the event loop before each slice. A Redis that never answers costs a bulk caller at most that wait per slice instead of wedging the request, and single-key callers are untouched. Regression tests push 1,500 keys and a 500-member removal through a slow publisher and assert every eviction reaches Redis; both fail against the previous tip. * fix(proxy): clear this worker's cache before pacing bulk eviction broadcasts Greptile pointed out that pacing the broadcasts in slices also paced the local eviction: a bulk delete waited for the publish backlog before dropping the next slice from this worker's own memory, so on a Redis that accepts connections but never answers a key deleted from the database kept authenticating on the handling worker for up to two seconds per slice. The merge base cleared every key locally at once. Bulk evictions now drop every affected entry from this worker's memory first, through the cache's own partition lookup since key objects live apart from the other auth objects, and only then publish in paced slices. The roster settle does the same for the deleted tokens, the JWT mapping keys and every changed member's user and membership entries before any step that can wait. Regression tests run both paths against a publisher that never answers and check that nothing is still cached on this worker before the first bound elapses; both fail with the local eviction removed. * fix(scim): write recomputed member roles only where the membership still backs them * fix(team): report the users a team delete detaches so SCIM demotes late-added members * test(scim): audit cells for the bulk group delete Move the SCIM group helpers into tests/integration/_support/scim.py and add the delete cells: a member's other group survives, the admin group's members are demoted and the audit row names the SCIM caller, a member a concurrent PUT adds while the delete waits on the team lock is demoted too, unknown, repeated, non-admin and hostile ids, an empty group, the peer proxy, a 20-delete burst across both proxies, and a delete after a worker was SIGKILLed * fix(proxy): drop the Redis copies of bulk-evicted auth objects before pacing the broadcasts * docs(proxy): drop the module docstring that repeated the PR description * fix(team): sweep deleted team references only under the lock so a late-added member is demoted The unlocked sweep that ran before the team's advisory lock stripped a member a concurrent group write added after the roster was read, so neither the roster snapshot nor the locked sweep's RETURNING named them and the SCIM recompute left them proxy_admin after the admin group was gone. The locked sweep does the same strip and reports whom it stripped, so it is the only sweep now. The integration cell holds the delete on a member's user row so a PATCH lands between the roster read and the lock, then checks the late member is demoted. Both waiter queries in the SCIM support module return their count as text so the first poll parses. * chore(team): carry the unbounded-IN baseline entries to delete_validated_teams The delete pass moved from delete_team and the unlocked sweep into delete_validated_teams, so the checker's baseline names the function the two team_id IN lists now live in * test(scim): expect the master key alias in the audit row and send post-kill requests on a fresh connection * fix(scim): bound bulk cache evictions to one Redis round trip and close lock-order gaps Bulk key evictions now drop entries from memory, delete them from Redis in one chunked DEL, and broadcast every key in one pipelined publish, so a large roster removal no longer overflows the publisher's pending cap or costs a Redis round trip per member. A push waits for in-flight membership loads before evicting. User rows are locked in byte order to match the spend writer, a bulk user delete takes every team lock before its first user write, and a POST whose roster write fails deletes the empty team so the IdP retry creates it again. * test(team): observe delete broadcasts at the Redis layer * fix(scim): lock user rows in id order on every multi-row writer and create unprovisioned members concurrently * fix(team): chunk the email lookup behind the member row lock * fix(scim): create a member deleted mid-push again instead of failing the group write * fix(scim): refuse strict-mode mid-push deletes with 400 and check license seats once per batch A member deleted mid-push is created again only when scim_upsert_user allows it, under the verb's own created_via. New group members are checked against the license seat limit as one batch before any is created. Auth cache publish drops log the key count instead of the key. * fix(scim): lock member rows while recomputing admin roles after a group push * fix(users): re-read teams under the row locks in bulk user delete so a racing group push keeps no deleted member * fix(users): take /user/delete and SCIM DELETE /Users users off their current teams under the row locks so a racing group push keeps no deleted member * fix(scim): keep retrying a push while deletes race it, lock rows in member bulk delete, evict removed keys from Redis first A group push gave up after one retry when SCIM user deletes kept landing between attempts, answering 500 in 9 of 10 race rounds. It now retries while each attempt finds newly deleted members, and gives up only when a member it created again goes missing a second time. The team member bulk delete now locks the removed members' user rows in the shared order and takes the team off their teams arrays in one array_remove statement, so it no longer writes back a teams array read before a concurrent push added another team. A push that removes members now drops their deleted team keys from memory and Redis together with the members' other cache entries before the team eviction, so a removed key cannot refill from Redis mid-cleanup. * fix(user-delete): keep the team alias label on the team members gauge after removals * test(integration): count relay statements over plaintext so an SSL-enabled Postgres does not hide them * fix(user-delete): skip a team deleted while the user delete waited on its lock instead of answering 404 * fix(scim): answer 404 and roll back when the team is deleted during a roster write * test(integration): audit cells for SCIM bulk roster writes, user delete locking and the eviction broadcast * test(scim): cover deleting a SCIM group through the team delete's bulk path * fix(scim): evict a deleted group's member caches in bulk and drop cached roles after a recompute * fix(scim): answer 404 to a group delete that lost a race to another delete * fix(scim): chunk the locked team lookup and delete in the team delete transaction * fix(scim): keep service-account keys and team models on a group delete and write its records once under the lock * fix(scim): demote a deleted group's members inside the delete transaction The admin-group role write now runs on the team delete's transaction, over the roster read under the team lock, so a failed delete leaves roles untouched and a member added while the delete waited is demoted with the rest. The delete transaction also waits for a free connection as long as it may run, so a burst of deletes queues instead of failing after Prisma's two-second default. Every member in the 5-vs-500 statement test now holds a key. * fix(audit): write deleted keys' audit rows in one insert A delete that removes many keys wrote one audit insert task per key, and those tasks held pool connections the delete's follow-up writes waited on. The race test now waits on any lock wait behind the delete.	2026-10-21 00:59:23 +00:00
jesus-berri	c8f7e0de7d	feat(model hub): show admin-defined custom tags and metadata from model_info (#46064) ... * feat(model hub): show admin-defined custom tags and metadata from model_info Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(router): move model hub custom field merging into router_utils Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(model hub): share the custom tag badge list between the column and details Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(model hub): cover public hub, patch, null, hostile, forwarding, and killed-worker paths for custom fields Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(model hub): drop custom_metadata entries with non-string keys or non-finite numbers Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(model hub): use isinstance narrowing instead of TypeGuard for custom_metadata values Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(model hub): build custom tags and metadata in the proxy instead of the router Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(model hub): serve custom tags and metadata on model_group_alias rows Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * remove docstring from model_group_custom_fields.py * style(model hub): fix lint and format in custom metadata tests Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * style(model hub): drop leading blank line in custom fields helper Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-10-20 17:45:06 -07:00
devin-ai-integration[bot]	7e8be61fa0	test: fix flaky team delete, refusal sink, loop-blocking, Moyai tamper and model-cost leak tests (#47135) ... * test(integration): stabilize auth and team delete checks Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(unit): make loop-blocking and tampered-signature checks deterministic Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(unit): isolate model-cost state and synchronize loop progress Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(unit): fail loop-progress handshakes when the blocking call never starts Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-10-20 18:01:36 +00:00
Mateo Wang	047cb4a8e2	test(integration): expect the Exa search envelope's provider_metadata after #47423 (#47572) ... Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-10-20 15:29:30 +00:00
devin-ai-integration[bot]	1660c8ba1f	feat(key_management)!: apply default_key_generate_params.allowed_routes on key generation (#41236) ... * feat(key_management): apply default_key_generate_params.allowed_routes on key generation Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(key_management): avoid mutating default routes payload Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * feat(proxy): apply default allowed_routes to key_type presets it covers * fix(proxy): keep management and read_only presets, skip service account keys, and show the default routes to custom_key_policy * fix(proxy): keep Full Access keys out of the default allowed_routes, let owners re-pick a key type, and show custom_key_policy the member key's routes * fix(proxy): keep owners from changing routes a key got from default_key_generate_params * docs(proxy): describe key_type when default_key_generate_params.allowed_routes is set, and test custom_key_policy input with the default off * test(proxy): type the default_key_generate_params allowed_routes tests * fix(proxy): let key owners send unchanged allowed_routes back on /key/update * fix(proxy): ignore a route default with a non-string entry and match resent routes exactly A default_key_generate_params.allowed_routes list with a non-string entry made /key/generate fail at the database, so it is now ignored like a non-list. An owner's resent allowed_routes on /key/update skip the route check only when they equal the stored list in order, so reordered or duplicated lists are checked as a change. Adds integration cells for the default and the owner update paths. * test(proxy): cover a non-string allowed_routes default in the key integration tests * test(proxy): read key rows from the owned proxy's database in the default routes integration tests * fix(proxy): leave allowed_routes out of a key update that resends them unchanged --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>	2026-10-19 18:53:37 -07:00
hamzashah-dev	9b4f44135d	fix(proxy): refresh cached team on /team/block and /team/unblock (#35643) ... * fix(proxy): refresh cached team on /team/block and /team/unblock Both endpoints wrote `blocked` to Postgres and returned without touching the management-object cache. Auth resolves teams through `team_id:{team_id}` before reading the DB and `common_checks` enforces `blocked` off that cached object, so a team whose traffic had already warmed the cache kept serving requests after being blocked, and kept getting rejected after being unblocked, until the entry expired. `_refresh_cached_team` already exists for exactly this and its docstring says every endpoint mutating `litellm_teamtable` must call it; these two were the only mutating endpoints that didn't. The update now also asks for `object_permission` so the refreshed cache entry doesn't drop the relation, matching `team_model_add` and `team_model_delete`. * fix(proxy): keep a team block from being undone by in-flight spend writes update_cache read the cached team when a request finished and wrote that whole copy back in a background task, so a team blocked or invalidated while requests were in flight came back unblocked on every busy worker until its 60 s cache entry expired. The team spend now goes onto the team held in this worker's memory at write time, and a missing entry is left missing for the next auth read to load. Block and unblock responses keep litellm_model_table and object_permission null, as before this PR, while the cache still gets both relations. * test(proxy): type the team spend cache test parameters * test(proxy): hold a team block under in-flight team spend and keep team aliases routing after block and unblock --------- Co-authored-by: Hamza Shah <hamzasepal@gmail.com> Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>	2026-10-19 22:50:33 +00:00
yuneng-jiang	44d99475a5	test(integration): stop GET /__observations from draining the upstream log (#47256) ... * test(integration): stop GET /__observations from draining the upstream log GET /__observations used to empty the queue, so a second read in a test saw nothing and a "never reached upstream" check could pass whether or not the request was forwarded. GET now returns every request since the last reset and DELETE /__observations is the reset. An autouse fixture resets the worker upstream before each test, which also covers tests that read without the gateway fixture. Discarded reads used as resets become DELETEs, helpers that accumulated drained reads now read one snapshot, and tests with several phases reset between them. The three vacuous negative checks (rag allowlist, guardrail opt-out chaos, xAI batch output) now assert the absence on the same read that shows a request which did reach the upstream. * style(integration): space the observations unit test like its neighbours	2026-10-18 09:58:05 +00:00
yuneng-jiang	fc27dfdc7d	test(integration): give each owned proxy a private database and Redis by default (#47213) ... * test(integration): give each owned proxy a private database and Redis by default owned_proxy and owned_proxy_process now boot on a scratch database and a per-proxy redis-server unless the caller opts into the worker's state with state=OwnedState.SHARED_WITH_WORKER, passes its own DATABASE_URL, or strips DATABASE_URL. Under the replica profile the private database gets the writer and reader roles so the reader path stays covered. Gateway carries the database_url its proxy writes to, and the Scenario cleanup checks and delete_key_if_present read from it, so they keep checking something on a private database. The 189 test files with owned proxy sites that depend on the worker's database or Redis opt in with a shares_worker_state module marker, which a module-scoped conftest fixture turns into the shared state. The 7 helper modules with such sites opt in per call, since conftest fixtures and other test files reach them. * test(integration): reach session fixtures with the shared marker and keep the worker retry default The module-scoped fixture that applied shares_worker_state ran after session-scoped fixtures, so proxies those booted went private in marked files. The marker is now applied around each marked item's setup, call and teardown, which covers fixtures of every scope. Owned proxies on the worker database inherited the worker gateway's router_settings.num_retries of 0 from it. A private proxy now gets the same default in its config copy unless the config sets num_retries itself. Five providers files read the owned proxy's spend rows or response cache through the worker's database or Redis and opt into sharing.	2026-10-18 03:37:41 +00:00
devin-ai-integration[bot]	836f326611	test(integration): remove BUG skips for 2 tests that pass on main (#47142) ... Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-10-17 11:11:16 -07:00
Mateo Wang	4a58784fa2	fix(cost): price exact cost-map names before price-free capability rules, and fix SCIM and MCP integration tests on CircleCI (#47037) ... * fix(test-harness): preserve sibling auth modules in isolated worker configs Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): expect servers in MCP connect-flow descriptions Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): expect ptu_hours in daily activity metrics after #43043 Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): kill every serving worker once the SCIM push blocks on the row lock * fix(cost): price exact cost-map names before price-free capability rule matches * test(streaming): price a priced alias stream at its deployment model and keep the unpriceable case unmapped Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(cost): price an explicit base_model under its own provider after candidate reordering Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(cost): retain priced generalized candidates and skip explicit-pricing lookups Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(cost): assert custom pricing makes no remote model-info requests Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(cost): keep the per-candidate pricing provider final Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(router): pin a Bedrock application profile prompt for the one-hour ttl its base model sends Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): assert the Nova Sonic tool-call argument delta carries the full arguments * fix(cost): sort cost candidates from the static cost map and rules, without provider lookups * test(integration): bill route-priced Bedrock chat streams and responses, and pin application profiles for an hour on messages and cache hits --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: Yuneng Jiang <yuneng@berri.ai>	2026-10-17 06:46:52 +00:00
yuneng-jiang	6530b329f4	fix(proxy): give /team/delete transactions a 30s timeout (#46979) ... Both interactive transactions in /team/delete used Prisma's 5s default. A lock wait or a short database stall past 5s expired the transaction after the keys were already deleted, so the call returned 503 and left the team row, memberships and team models behind	2026-10-16 14:36:48 -07:00
Irvallen Saragih	d6b54ae2b0	fix(openai)!: route gpt-transcribe to GPT audio transcription config (#44021) ... * fix(openai): route gpt-transcribe to GPT audio transcription config * test(openai): move gpt-transcribe tests under tests/unit * fix(openai): bill gpt-transcribe from provider usage seconds when duration is unknown Fall back to the provider-reported usage.seconds when neither a measured file duration nor a response duration is available, so json responses for undecodable uploads are no longer billed at zero. Move the routing test next to the other provider config tests and replace the structure test with a behavior test. * fix(openai): bill provider usage seconds only for gpt transcription models * fix(cost): decide provider-billed transcription seconds from the deployment model, not its pricing id * test(integration): cover gpt-transcribe default request, provider-billed spend, and health probe --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>	2026-10-16 20:27:47 +00:00
yuneng-jiang	b602209ca6	test(integration): hand every harness server a port from one per-worker block (#46761) ... * test(integration): hand every harness server a port from one per-worker block About 45 places in the integration harness picked a port by binding port 0, reading the number back, closing the socket and starting a server on that number later. Another process could take the port in between, which surfaced as "Address already in use" flakes. Earlier fixes retried around the race in single call sites without removing it. tests/integration/_support/ports.py now owns every port the harness picks ahead of a server. Each pytest process claims a 200-port block between 10000 and 32767 by taking an exclusive flock on a per-block lock file in the system temp directory, so parallel runs on one machine never share a block and the lock goes away with the process. The range is above the fixed service ports and below the Linux and macOS ephemeral ranges. Ports go out in order, are never reused in a run, skip anything an outside process already holds, and a process that exhausts its block claims the next free one. The three _free_port copies, worker_endpoints' port-0 binds, free_loopback_port, the owned proxy's lost-port-race retry and the owned Redis retry loop are gone. Tests that restart a server on the same port now allocate that port up front instead of reading it back from the first server. TQ012 in scripts/check_test_quality.py rejects a port-0 bind read back with getsockname anywhere under tests/integration. * ci(integration): record listening sockets and the ephemeral port range * Revert "ci(integration): record listening sockets and the ephemeral port range" This reverts commit d3fafc3da2.	2026-10-15 16:56:40 -07:00
devin-ai-integration[bot]	e344dbd8db	feat(proxy): split Azure PTU deployments across teams with per-team PTU ceilings and PTU-hour reporting (#43043) ... * feat(proxy): split Azure PTU deployments across teams by share A PTU deployment's model_info takes ptu_shares (team_id -> whole PTUs adding up to ptu_count) in place of a single team_id. With LITELLM_ENABLE_PTU_COST_ATTRIBUTION on, routing hides the deployment from teams holding no share, the v3 limiter enforces each share as a per-team TPM ceiling from Azure's per-model TPM-per-PTU sizing, the daily flat-cost rollup splits the hourly cost by share, and /team/daily/activity plus the Team Usage page report PTU-hours next to tokens. * fix(proxy): weight cached input in PTU ceilings and size team-scoped deployments by public name The PTU flag is read from the process environment on every check instead of through the secret-manager path, so a proxy with a hosted secret manager does not pay a round trip per request. A team's ceiling now carries the model's cached-input ratio and settlement charges cache reads at that ratio, matching how Azure sizes a PTU. A deployment registered for one team is found by its team_public_model_name for PTU-hours and the ceiling, since its model_name is rewritten to a unique internal name. ptu_shares are StrictInt on ModelInfo and parsed_ptu_shares refuses booleans, strings, and non-string team ids, a fractional ptu_count reserves nothing, and models_processed counts distinct deployments rather than holdings. * fix(proxy): reserve PTU ceilings in normalized tokens, keep share routing flag-independent, and warn only shared or Azure reservations * fix(proxy): size the uncapped output floor against the PTU share in raw output tokens and escape the exported team id * fix(ptu): refuse ptu_shares naming a team that does not exist * refactor(ptu): keep Azure detection under llms/azure and check share teams with one query * fix(proxy): drop the reverted TeamIdSearchFilter import after merging main Main reverted the team key search change that added TeamIdSearchFilter, so the PTU share team check types its where clause locally * fix(proxy): chunk the ptu_shares team lookup under the IN-list bound * fix(proxy): count a request naming a shared PTU deployment directly against its group's ceiling * fix(proxy): resolve every name a shared PTU deployment is served under to its group's ceiling * fix(ptu): refuse ptu_shares declared without ptu_count and cost_per_ptu_per_hour * fix(ui): seed total_ptu_hours in the empty daily activity metadata * fix(ptu): size usage rows by the name the request used * fix(ptu): size a team's usage rows by the deployment it is served from * fix(ptu): never size a team's usage row by another team's reservation * fix(ptu): reserve an uncapped request against the cap the proxy writes, not its prompt length * fix(ptu): read a share map only through the terms registration requires A ptu_shares map stored without ptu_count, cost_per_ptu_per_hour and ptu_effective_from, or one that does not add up to ptu_count, is refused at registration but was enforced once loaded: every other team got the 400 and the share holder a ceiling while no flat cost accrued. _deployment_shares now reads the shares through ptu_terms, so such a row reserves nothing, the same as before the map was read. * fix(ptu): keep a declared split as the access rule and check a config split at boot with the flag off 6fb24d1d read ptu_shares through the pricing terms so a termless map reserved nothing, but with the flag off the router skips ptu_config_error for config.yaml entries, so such an entry loaded and served every team, against the rule that a declared split restricts access whether or not cost attribution is on. _deployment_shares reads the declared map again, and the router now raises ptu_config_error for a config-sourced entry declaring ptu_shares regardless of the flag, so a malformed split is refused at boot instead of being honoured or ignored silently. * fix(ptu): charge wildcard-served names, pass-through usage, and team-less settlements against the team's PTU ceiling A name served by a wildcard route now resolves to the shared deployment behind it, so the limiter's ceiling and the team daily activity sizing both find it. Pass-through responses settle the PTU counter from combined_usage_object in Azure-normalized tokens, and a reservation taken at admission is settled even when the success metadata lost the team. * fix(ptu): resolve a wildcard-served name to the one pattern the router picks, and a deployment id before any wildcard A team's wildcard reservation behind an open proxy-wide * route was charged and sized for traffic the * route served, and naming a shared deployment by id next to a catch-all wildcard escaped its per-team ceiling. Router.wildcard_route_deployments now owns the most-specific-proxy-wide-else-team precedence, and both the limiter and the consumption page resolve through it after listed rows and deployment ids. * test(router): pin the wildcard route precedence that PTU sizing and the router share * fix(proxy): accept whole-number PTU shares written as floats, as ptu_count already does A share of 2.0 now reads as 2 on POST /model/new and in config.yaml, where it was refused with a 422 and dropped at load. Fractional, infinite, boolean and string shares stay refused. Adds unit tests for the malformed-rate, provider-over-prefix, zero-token Responses usage, fractional-count and sizing-warning paths. * fix(proxy): settle team PTU reservations in their own minute and keep billed PTU days when shares move * fix(proxy): write each PTU deployment day's team charges in one transaction The backfill counts a deployment day as priced once any team's row exists for it, so a rollup that wrote team A's share and lost team C's left team C unbilled for good. Each deployment day's team rows now go through one Prisma batch with the retry around the whole batch, so a day is fully written or not written at all and the next backfill prices it for every team. * fix(proxy): read the PTU flag from a secret manager, serve share holders from their share, and refuse unfittable calls and share-holding team deletes * test(router): list the cached PTU ceiling helpers in the router coverage check * fix(proxy): read the PTU flag only for share holders, guard org and SCIM team deletes, keep share holders on their shared deployment through retries and the scheduler, and date PTU-hours by the terms in force * fix(proxy): bill a rerun PTU day to the current share holders alone * fix(proxy): replace a PTU day's rows by day and deployment, not a negated team list * fix(proxy): read the PTU flag from the secret manager first, like get_secret * fix(router): scope retry health lookups to the caller only for PTU share holders * fix(router): move the share-holder retry check into ptu_shares Adds ptu_hours and total_ptu_hours to the daily activity integration goldens and fixtures, since every route now reports them * test(integration): cover PTU shares across routing, delete guards, cost attribution and config regex timing --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>	2026-10-15 18:44:30 +00:00
Mateo Wang	622daf3cdf	fix(scim): write a group's roster in one locked transaction instead of one member at a time (#45197) ... * fix(scim): write a group's roster in one locked transaction instead of one member at a time POST, PUT and PATCH /scim/v2/Groups resolve every member in one query and write the roster, the membership rows and the users' team tags in one transaction under the team's advisory lock, so a 500-member push runs a fixed number of statements instead of thousands. PUT now answers with the roster it just wrote * test(scim): assert the roster the IdP gets back and boot the bulk roster cell with two workers * fix(scim): skip the team lock for a rename-only PATCH and lock member rows in id order before the roster write A PATCH naming no members opened the roster transaction and took the team advisory lock for nothing. A SIGKILLed worker then left its orphaned query engine holding that lock until the 60 s transaction timeout, and /team/delete, which takes the same lock, timed out behind it. An empty delta is now answered from a plain team read, with no transaction and no lock Two groups trading members in parallel locked user rows in opposite orders, and one request died with a Postgres deadlock. Every user row a push touches is now locked with SELECT ... ORDER BY user_id FOR UPDATE before the first write, so the two requests serialize The bulk roster cell gets a 120 s client through a new owned_proxy_process client_timeout, so an unfixed push fails on the statement count instead of the harness clock, requires positive counts on both pushes, and gains a parallel roster-trade test that answered 500 before the lock * fix(scim): evict the cached team on a group rename and read a no-op roster from the writer A rename-only PUT or PATCH wrote the new alias and left the team cached under its id and old alias on every worker, so keys kept logging spend against the old team alias until the TTL expired. The group write now evicts the cached team (and broadcasts the eviction) right after the alias or metadata update. sync_team_roster reads the team from the writer when the plan names nobody, so a replica cannot hand back the pre-rename row; a plan already matching the roster returns inside the lock without writing, evicting, or emitting the members metric; the response is built from the written row, so lastModified reflects the roster write; and detaching a removed member bumps the user's updated_at like every other user write. * fix(scim): bump detached members' updated_at in UTC The detach UPDATE wrote updated_at = CURRENT_TIMESTAMP, which Postgres casts into Prisma's naive timestamp column in the session timezone, so on a database whose session is not UTC the removed member's row landed hours away from the UTC instants Prisma itself writes. Write NOW() AT TIME ZONE 'UTC' instead, and have the bulk roster cell assert the bumped row sits between two reads of the database's own UTC clock taken around the replace. * fix(scim): chunk the group write's member lookups under the bind cap Every Prisma "in" filter the group write issues bound one parameter per member with nothing bounding the list, which check_unbounded_in_lists reports as 11 new findings. The single-field lookups now go through litellm.repositories.chunked_in (caller_transaction inside the roster transaction, per_chunk_ok for the role recompute), and the one cross-field OR read slices its values at IN_LIST_CHUNK_SIZE and records the bound. A push of IN_LIST_CHUNK_SIZE + 1 members now reads in two slices, and a member named only in the last slice still resolves. * fix(proxy): send bulk cache evictions in slices so the broadcaster drops none Veria pointed out that the invalidation publisher caps its pending tasks at 1,024 and drops the excess with a warning, and that a group push now detaches every removed member's keys in one call instead of one member at a time, so a removal of more than roughly 340 members (three cache keys each) silently left stale key objects on the other replicas. The merge base never hit the cap because each member was a separate request. Bulk evictions (team, user, organization, and session key deletes, plus the roster settle) now publish in slices of 256 and wait, bounded at two seconds, for the backlog already handed to the event loop before each slice. A Redis that never answers costs a bulk caller at most that wait per slice instead of wedging the request, and single-key callers are untouched. Regression tests push 1,500 keys and a 500-member removal through a slow publisher and assert every eviction reaches Redis; both fail against the previous tip. * fix(proxy): clear this worker's cache before pacing bulk eviction broadcasts Greptile pointed out that pacing the broadcasts in slices also paced the local eviction: a bulk delete waited for the publish backlog before dropping the next slice from this worker's own memory, so on a Redis that accepts connections but never answers a key deleted from the database kept authenticating on the handling worker for up to two seconds per slice. The merge base cleared every key locally at once. Bulk evictions now drop every affected entry from this worker's memory first, through the cache's own partition lookup since key objects live apart from the other auth objects, and only then publish in paced slices. The roster settle does the same for the deleted tokens, the JWT mapping keys and every changed member's user and membership entries before any step that can wait. Regression tests run both paths against a publisher that never answers and check that nothing is still cached on this worker before the first bound elapses; both fail with the local eviction removed. * fix(proxy): drop the Redis copies of bulk-evicted auth objects before pacing the broadcasts * docs(proxy): drop the module docstring that repeated the PR description * fix(scim): bound bulk cache evictions to one Redis round trip and close lock-order gaps Bulk key evictions now drop entries from memory, delete them from Redis in one chunked DEL, and broadcast every key in one pipelined publish, so a large roster removal no longer overflows the publisher's pending cap or costs a Redis round trip per member. A push waits for in-flight membership loads before evicting. User rows are locked in byte order to match the spend writer, a bulk user delete takes every team lock before its first user write, and a POST whose roster write fails deletes the empty team so the IdP retry creates it again. * test(team): observe delete broadcasts at the Redis layer * fix(scim): lock user rows in id order on every multi-row writer and create unprovisioned members concurrently * fix(team): chunk the email lookup behind the member row lock * fix(scim): create a member deleted mid-push again instead of failing the group write * fix(scim): refuse strict-mode mid-push deletes with 400 and check license seats once per batch A member deleted mid-push is created again only when scim_upsert_user allows it, under the verb's own created_via. New group members are checked against the license seat limit as one batch before any is created. Auth cache publish drops log the key count instead of the key. * fix(scim): lock member rows while recomputing admin roles after a group push * fix(users): re-read teams under the row locks in bulk user delete so a racing group push keeps no deleted member * fix(users): take /user/delete and SCIM DELETE /Users users off their current teams under the row locks so a racing group push keeps no deleted member * fix(scim): keep retrying a push while deletes race it, lock rows in member bulk delete, evict removed keys from Redis first A group push gave up after one retry when SCIM user deletes kept landing between attempts, answering 500 in 9 of 10 race rounds. It now retries while each attempt finds newly deleted members, and gives up only when a member it created again goes missing a second time. The team member bulk delete now locks the removed members' user rows in the shared order and takes the team off their teams arrays in one array_remove statement, so it no longer writes back a teams array read before a concurrent push added another team. A push that removes members now drops their deleted team keys from memory and Redis together with the members' other cache entries before the team eviction, so a removed key cannot refill from Redis mid-cleanup. * fix(user-delete): keep the team alias label on the team members gauge after removals * test(integration): count relay statements over plaintext so an SSL-enabled Postgres does not hide them * fix(user-delete): skip a team deleted while the user delete waited on its lock instead of answering 404 * fix(scim): answer 404 and roll back when the team is deleted during a roster write * test(integration): audit cells for SCIM bulk roster writes, user delete locking and the eviction broadcast	2026-10-15 10:57:00 -07:00
Thijmen Stavenuiter	a76e723b26	feat(proxy): report when each per-model budget window resets (#45797) ... * feat(proxy): report when each per-model budget window resets model_max_budget_usage on /key/info, /user/info and /team/info now carries budget_reset_at per model, the open window's start plus its period, or null when no window is open. The window start is read in the same batched lookup as the spend counter * fix(proxy): count a monthly budget window from its own start when reporting its reset duration_in_seconds measures Nmo from now, so a window opened on January 15 and read on February 10 reported February 12, then null while the counter still refused requests until February 15. duration_in_seconds_from measures the month from the window start, and duration_in_seconds now shares that month arithmetic * fix(proxy): measure months from timezone-aware datetimes and assert whole usage values datetime.fromtimestamp without a tz tripped DTZ006. The month arithmetic now runs on an aware local datetime, which keeps the same wall-clock result. The duration tests move to parametrized functions, since subTest payloads carrying datetimes cannot cross the xdist worker boundary, and the new usage tests compare the whole entry * fix(budgets): end a monthly budget window a calendar month after it opens Each charge recounted the month from the charge date, so a monthly window opened in a month of another length expired days before or after the reset /key/info reports. The window is now measured from its start, by the same helper the usage report uses. A period ending after year 9999 reports no reset instead of failing the read. * fix(budgets): count monthly budget windows in UTC for every router budget Months were counted on the host's local calendar, so replicas with different TZ settings disagreed on when a monthly window ends. Provider, deployment, tag and per-model budgets now all measure a window from its start in UTC, and a test covers the router budgets. A reset past year 9999 is documented as null. * test(budgets): cover monthly tag budgets in the router window test * test(budgets): add integration cells for budget_reset_at and monthly budget windows * test(budgets): find a monthly window in another span on every calendar day --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>	2026-10-15 10:56:45 -07:00
Kolade Fajimi	ea777eb21b	fix(auth): return a generic 401 for an unknown key (#39787) ... * fix(auth): return a generic 401 for an unknown key The wrong-key 401 shipped the internal verification table name and the submitted key's full SHA-256 hash to any unauthenticated caller. Both now go to the proxy's own log and the response carries only "Invalid API key provided." The Admin UI detected a dead session by string-matching that message, so it now reads the structured error type instead. * Update ui/litellm-dashboard/src/lib/http/client.ts Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> * fix(auth): log the missing key hash where every resolve path passes Review catch: JWT key-mapping misses resolve through IdentityStore without touching the main key path, so logging the hash there left those callers with nothing but the generic 401. Log it at the store's raise site instead, which every resolve path shares, and drop the now-duplicate log. * test(auth): inject a key-less client instead of patching the DB fetch The store test tripped TQ008 by patching _fetch_key_object_from_db_with_reconnect. Hand IdentityStore a client whose get_data returns no row instead, so the miss is driven through the real lookup rather than around it. * fix(auth): log the unknown key's hash once on the auth failure line The per-lookup warnings in get_key_object and IdentityStore logged whatever value they were handed, which on the failure-logging path is the raw key a caller sent (a provider key included), let a newline in ?key= forge log lines, and fired on lookups that expect a miss. The hash now rides KeyNotFoundError onto the single existing auth failure log line next to the requester IP, with control characters escaped. Also drops the new comments and updates skipped tests that quoted the old message. * test(auth): add the blank lines ruff format expects before _InFlightCountingPrisma * test(integration): expect the generic wrong-key 401 in the management lifecycle tests * fix(auth): keep the operator wording on the unknown-key log line and drop the redundant 401 rewrite * test(auth): cover the generic unknown-key 401 across routes, proxies, and a worker kill * test(auth): cover the realtime upgrade refusing an unknown key without echoing it --------- Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>	2026-10-15 01:05:25 -07:00
yuneng-jiang	9271edc734	feat(proxy): let proxy admins open team metadata to team admins, keeping proxy settings with proxy admins (#46506) ... * feat(proxy): let proxy admins open team member defaults to team admins * fix(proxy): keep a team admin's never-resets duration on a team's first member budget * test(proxy): pin the existing-row and stored never-resets cases for member defaults * feat(proxy): let proxy admins open soft_budget to team admins * feat(proxy): let proxy admins open team metadata to team admins, keeping proxy settings with proxy admins * test(ui): name expected objects in team admin edit access tests Keeps local/no-large-inline-object-arg within its eslint budget * fix(proxy): compare team metadata without recursion and count key presence A proxy-controlled metadata key that appears or disappears with a null value is now a change, and the JSON comparison no longer recurses * fix(proxy): compare team metadata numbers exactly Numbers are compared as decimals, so integers past float precision and close floats are no longer treated as equal	2026-10-14 23:51:44 +00:00
yuneng-jiang	df3b44dbee	feat(proxy): let proxy admins open team member defaults to team admins (#46353) ... * feat(proxy): let proxy admins open team member defaults to team admins * fix(proxy): keep a team admin's never-resets duration on a team's first member budget * test(proxy): pin the existing-row and stored never-resets cases for member defaults	2026-10-14 14:05:58 -07:00
yuneng-jiang	47c40586e1	fix(proxy): evict the cached user on every proxy for every /user/update and /user/bulk_update (#46500) ... * fix(proxy): evict the cached user on every proxy for every /user/update and /user/bulk_update Previously the cached user row was only evicted and broadcast when the update touched max_budget, model_max_budget, tpm_limit, rpm_limit or metadata, so a models or user_role change kept being served by peer proxies until the cache TTL expired * test(proxy): cover user cache eviction past the first bulk invalidation batch	2026-10-14 13:25:20 -07:00
yuneng-jiang	1d9dd7b067	feat(proxy): let proxy admins open soft_budget to team admins (#46485)	2026-10-14 19:27:37 +00:00
yuneng-jiang	0915691d32	test(integration): remove six setup causes of CircleCI integration reds (#46490) ... Drain the worker's fake-upstream request log when a test's gateway fixture starts, so traffic left by earlier tests on the same xdist worker no longer shows up in a test's first observation read Generate the fine-tuning rig's provider file and job ids per fixture build, so a rebuilt module fixture in the same worker no longer collides on the managed object unique constraint Wait for every wedged team_id and team_alias invalidation publish to fail before taking the recovery marker, now that team updates publish both keys Pin the cooled-down priority scheduler request to the connection that observed the cooldown, since a sibling worker reads cooldowns from Redis on an interval Simulate upstream outages and the held budget peer on ports the test keeps owning, instead of releasing a port another worker can bind Start a peer proxy for the accounting group, which has a test that needs one	2026-10-14 12:00:35 -07:00
devin-ai-integration[bot]	46a389c60b	feat(claude-code): emit installationPreference in marketplace.json entries (#41430) ... * feat(claude-code): emit installationPreference in marketplace.json entries Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(claude-code): satisfy marketplace CI checks Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(claude-code): qualify marketplace entry as read only Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(claude-code): suppress typed marketplace assembly warning Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(claude-code): restrict installation_preference to the documented values Accept only available, auto_install, or required so a typo returns 422 instead of being silently dropped, and cover the PUT full-replace clearing behavior with a test * fix(claude-code): echo installation_preference on the public skill hub * refactor(claude-code): read installation_preference with one membership check and cover hand-edited rows * test(integration): audit installation_preference on the Claude Code marketplace * test(integration): kill a serving worker in the marketplace chaos cell * test(integration): give the marketplace worker-kill cell the two-boot timeout * fix(proxy): keep installationPreference on the MarketplacePluginEntry model * fix(proxy): refuse self-installing preferences off sha256 archives, accept the camelCase key, and echo the preference on POST and PUT * fix(claude-code): serve installation_preference only on archive plugins and refuse both key spellings Claude Code 2.1.75 and older reject a marketplace.json once any plugin carries installationPreference, and every Claude Code release before archive support rejects a catalog with an archive plugin. Requiring an archive source for every value, available included, keeps the field off catalogs those clients can read. A request that sends installation_preference and installationPreference together now gets a 422. * fix(claude-code): read the preference spelling that carries a value when the other is null --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>	2026-10-14 07:59:25 +00:00
devin-ai-integration[bot]	feb92d3255	test(integration): cover orgs_users_customers endpoint contracts (#44725)	2026-10-14 07:38:06 +00:00
devin-ai-integration[bot]	5f7de8d6ac	test(integration): cover keys_auth endpoint contracts (#44674) ... * test(integration): cover keys_auth endpoint contracts Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): cover hashed, path and token spellings for keys_auth contracts Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): strengthen JWT mapping lifecycle coverage Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): cover reset_spend over plaintext and hashed path spellings with reset_to 0 and 0.02 Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): drop Final in reset loop and use drain-then-fresh-refusal in warmed-peer delete proof Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): disable model info refresh in JWT lifecycle rig Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): drop unused discovery constant Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): cover cache invalidation and key paths Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: mateo <277851410+mateo-berri@users.noreply.github.com> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: mateo <mateo@berri.ai>	2026-10-14 07:12:28 +00:00
devin-ai-integration[bot]	8822e086ec	test(integration): cover budgets_spend endpoint contracts (#44736) ... * test(integration): cover budgets_spend endpoint contracts Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): cover budgets_spend endpoint contracts Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): cover budgets_spend endpoint contracts Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): cover budgets_spend endpoint contracts Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): harden budgets_spend contracts per review Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): make spend report window midnight safe and drop redundant comments Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): pin organization spend report scope Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): close budgets and spend review findings Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): drop streamed cost header absence assertion Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(spend): cover streamed spend on linked budget tiers Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(spend): wait for async spend logs Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: mateo <mateo@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-10-14 06:58:22 +00:00
devin-ai-integration[bot]	e226016aee	test(integration): cover health_routing endpoint contracts (#44744) ... * test(integration): cover health_routing endpoint contracts Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): tighten health_routing contracts per review Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): close health_routing review gaps Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * style(integration): format observed routing fallback cells Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): close health_routing final review findings Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): resolve health_routing review findings Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: mateo <mateo@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-10-14 06:34:22 +00:00
devin-ai-integration[bot]	0f0c1bc80b	test(integration): cover teams endpoint contracts (#44705) ... * test(integration): cover teams endpoint contracts Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): address review on teams contract tests Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): drain shared upstream observations at fixture setup Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): prove team block, member and bulk permission contracts at runtime Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): assert email admin adds and whole membership rows for team members Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): drive key update grant and revoke, Z-suffixed temp budget expiry, link LIT-9257 Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): close review gaps in teams endpoint contracts Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): harden team member endpoint contracts Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: mateo <mateo@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-10-14 06:29:20 +00:00
devin-ai-integration[bot]	148ca66ecb	test(integration): cover prompts_tags_caching endpoint contracts (#44686) ... * test(integration): cover prompts_tags_caching endpoint contracts * test(integration): assert prompt lifecycle requests per environment * test(integration): pin non-latest prompt versions, label/path spellings and tag header/key spellings Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): cover production PATCH, streamed and root /responses prompts, and memory create scope Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): ignore proxy model discovery probes in prompt upstream doubles Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): prove tag update, peer prompt resync, memory team rows and Responses prompt spellings at runtime Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): cover memory metadata and key_prefix listing, multi-tag spellings for tag cache invalidation Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: mateo <mateo@berri.ai>	2026-10-14 04:10:31 +00:00
devin-ai-integration[bot]	02f56c6319	test(integration): cover guardrails_policies endpoint contracts (#44722) ... * test(integration): cover guardrails_policies endpoint contracts Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): let the admin submissions list carry other rows Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): close guardrails policies review gaps Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): cover guardrail auth and policy attachments Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): tidy guardrail policy test helpers * test(integration): keep plain non-admin submission refusal live * test: expand policy attachment integration coverage Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): accept the intended 403 for plain submission keys Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): pin multi-tag header and spend row tags for policy tag scope Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): drop the duplicate pydantic import left by the main merge Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: mateo <mateo@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: Devin <noreply@devin.ai>	2026-10-14 04:07:26 +00:00
devin-ai-integration[bot]	c9a1d9b026	test(integration): cover small_inference endpoint contracts (#44654) ... * test(integration): cover small_inference endpoint contracts Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): assert full search tool listing for scoped keys Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): keep moderation and search tool peers stateless Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): cover filename-agnostic OCR uploads, list queries, object rerank documents and search tool authorization Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): build search tool and moderation proxy configs without mutating the loaded yaml Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): cover the /ocr alias, model search spelling and Cohere v2 max_tokens_per_doc, clean up non-admin search tool rows Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): cover octet-stream OCR uploads and BUG-skip the Python requests boolean spelling * test(integration): pin Exa passthrough of native fields and caller contents Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: mateo <mateo@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: Mateo Wang <277851410+mateo-berri@users.noreply.github.com>	2026-10-14 04:06:15 +00:00
devin-ai-integration[bot]	903f04dd46	test(integration): cover vector_stores endpoint contracts (#44739) ... * test(integration): cover vector_stores endpoint contracts Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): strengthen vector store route contracts Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): update vector store route contracts Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): drop owner-update BUG skip for admin-only vector store routes Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(vector_stores): cover the managed-file fallback, owner delete and per-request retries Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): probe vector store worker readiness by deployment id Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): pin vector store retry header and create routings Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * style(integration): ruff format vector store managed file tests Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: mateo <mateo@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-10-14 03:59:08 +00:00
devin-ai-integration[bot]	0a4ab269d7	test(integration): cover mcp_management endpoint contracts (#44687) ... * test(integration): cover mcp_management endpoint contracts Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): require server_not_found for pending and rejected submissions Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): pin per-worker submission reloads, toolset and alias-collision spellings, raw key overrides Assert the approving and rejecting worker immediately and only wait on the peer, add an outsider key and exact upstream calls, list by toolset_name, cover a config alias shadowing a DB server_name, send key_hash as a raw sk- key, and add BUG skips for API disallowed_tools and overrides on keys or teams without object_permission Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): pin XFF trusted-peer, load balancer chains and sibling refusal body Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): enforce output_policy trust chain, JSON-RPC XFF views, MCP tool-policy BUG pins Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(mcp): force a registry reload during the pending window, pin the admin default listing and the whole tools/call body Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: mateo <mateo@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-10-14 03:55:50 +00:00
devin-ai-integration[bot]	11e91f464a	test(integration): cover scim endpoint contracts (#44656) ... * test(integration): cover scim endpoint contracts Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): assert scim admin key access and required active Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): cover scim cold-cache reactivation, entra membership ops, username filter and external id Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): assert full SCIM responses and denied writes Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): pin full SCIM resources and group filters Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): deny non-admin SCIM user deletes Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: mateo <mateo@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-10-14 03:41:54 +00:00
devin-ai-integration[bot]	117684cce5	test(integration): cover agents_a2a endpoint contracts (#44650) ... * test(integration): cover agents_a2a endpoint contracts Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): pin agent daily spend to the logged request day and assert cleanups Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): expect agent daily activity newest day first Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): cover A2A 1.0 task aliases, agent id headers, declared access groups and per-token spend Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): assert full A2A stream, header, alias, task and spend contracts Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): assert forwarded a2a-version header on A2A sends Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): release A2A stream gate only after a full SSE frame Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): pin A2A trace id, stream headers, denied alias and task methods Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): pin A2A card protocol version Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): assert the whole accepted push config response Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: mateo <mateo@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-10-14 03:40:07 +00:00
devin-ai-integration[bot]	013c440597	test(integration): cover model_management endpoint contracts (#44684) ... * test(integration): cover model_management endpoint contracts Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): pin the exact model access refusal after a rename Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): wait for allowlist convergence in the rename test Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): give the rename test lint suppression its reason Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): close reviewer gaps on model management contracts Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): share the blocked serving assertions Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): strengthen authorization and configuration regression tests Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): converge spend readbacks, ignore model discovery, cover slash credential names Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): pin the dashboard credential edit body round-trip Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: mateo <mateo@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-10-14 03:36:47 +00:00
mrinal-berri	93b29c3977	fix(proxy): broadcast team cache invalidation on team writes (#46113) ... * fix(proxy): broadcast team cache invalidation on team writes Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(proxy): cover cross-instance team cache invalidation Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-10-13 16:25:22 -04:00
yuneng-jiang	ec58b889cc	test(integration): isolate shared proxy state behind six order-dependent flakes (#46172) ... Each test below passed or failed on the same commit depending on test order or port timing in the CircleCI integration workflow - readiness probe only polls a port the launched proxy owns, so a stolen port no longer sends GET /health/integration-readiness to another test's wire server - team update tests seed fresh member users, since a proxy admin relocation adds team members to the destination org and leaked org_b membership into the shared INTERNAL_USER - realtime whisper tests wait until every proxy worker serves only the new gpt-realtime-whisper deployment, via a per-worker deployment id route - configured fallbacks test runs on its own proxy and database, because /config/update persists router settings and cannot unset fallbacks, which left [] and changed the no-fallback error text other tests assert - otel excluded services tests each get a private Redis so a sibling proxy cannot warm the budget registry cache the tenant span depends on - Langfuse audit upstream uses unique response ids instead of len(targets), which could repeat under concurrency and drop SpendLogs rows	2026-10-13 09:55:11 -07:00
mrinal-berri	f516668f96	fix(managed_files): enforce input file ownership on batch and fine-tuning create (#45855) ... * test(managed_files): pin ownership checks on batch and fine-tuning create Tenant B can create a batch or fine-tuning job from tenant A's managed file id today. These fail on main for acreate_batch input_file_id and acreate_fine_tuning_job training_file and validation_file * fix(managed_files): check input file ownership on batch and fine-tuning create Batch create and fine-tuning create only mapped the managed input file to provider ids and never checked who owned it, so any key with access to the model could build a batch or job on another tenant's file and then own the outputs. Both now run check_file_ids_access like chat and responses do, which 403s a non-owner and 404s a managed id with no ownership row. Fine-tuning also checks validation_file, which was never looked at * test(batches): pin that a pre-call ownership denial stops every batch create branch LIT-4797 noted an endpoint-level check missed the load-balanced branch because it dispatches first. The ownership check lives in the pre-call hook, which runs before branch selection, so a 403 there must keep both the load-balanced and unified-file branches from reaching the router * test(integration): pin managed-file ownership on batch and fine-tuning create Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(managed_files): check ownership before the batch rate limiter reads the input file and keep key-owned resources across key regenerate Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-10-12 13:58:25 -04:00
devin-ai-integration[bot]	7efce7f8e0	test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) ... * test(integration): give chaos and timing tests a harness API and gate raw signals Chaos tests killed and suspended proxy workers with raw os.kill and psutil calls and bounded elapsed time with fixed literals, so they raced the uvicorn worker healthcheck and failed on slow CI boxes OwnedProxy now offers kill_worker and replacing_worker, which return only once a replacement answers readiness with its own pid, and suspended, which is refused unless the proxy was launched with a lenient worker healthcheck. timing.py bounds durations against a measured control or the injected delay. run.py measures the proxy import time once and derives readiness deadlines and the per-test timeout from it, writing boot-calibration.json check_test_quality.py adds TQ010 (raw process signals) and TQ011 (elapsed time asserted under a numeric literal) for tests/integration outside _support, and every call site is migrated so both report zero with no suppressions * test(integration): steer the assistants wire burst onto both workers The mixed-status burst assumed the OS would spread 48 connections across both workers and failed when one worker took them all. Each half is now sent while the other worker is suspended, and the test asserts an exact 24 and 24 split * test(integration): restore product-enforced timing bounds lost in the migration Some TQ011 migrations dropped or loosened a bound whose time is the behaviour under test. The websocket session cap only kept its floor, so a cap closing at 300 s would have passed, and several timeouts were bounded by the injected stall instead of the configured product value assert_enforced_within bounds an elapsed time by the configured value plus explicit slack, with an optional floor. The websocket cap, the MCP validation timeout, the guardrail litellm_params timeout, the prisma command timeouts, the baseline projection lock timeout and the key metadata statement timeout use it. The device code refusal keeps its 15 s margin, and the langfuse retried prompt fetch is now bounded by a cold fetch of a steady prompt as well as by the ignored retry-after * fix(tests): wait for the supervisor to reap killed workers and fix real-kill migrations The lenient worker healthcheck was the whole case budget, so uvicorn's supervisor sat in a ping on the killed worker for up to that long before respawning it. It now uses the graceful stop budget, and a worker kill returns only after the victim is reaped, so pooled connections no longer race the dead worker's socket. Several tests on main picked the multiprocessing resource tracker as their victim and never killed a worker. Now that they kill a real one, presidio counts streams cut by the death, delete_model waits for the config reload that removes a deleted model from a replacement worker, and the wedged coordination Redis test kills before the wedge and waits for every parked publish to fail before resuming Redis. The redis cluster test imports the harness through the tests package so it collects in the unit job, and the per-model fallback rig exposes the owned proxy it now needs. * test(observability): only require S3 deliveries the killed worker could not have buffered A real SIGKILL drops the victim's unflushed batch, so requests it already answered can't be expected to land. The test now requires every request served after the replacement to land, some of the surviving worker's burst share to land, and nothing unsent to land --------- Co-authored-by: Yuneng Jiang <yuneng@berri.ai>	2026-10-12 05:27:33 -07:00
devin-ai-integration[bot]	daa3d36c4d	test(integration): migrate before timed owned proxy launch and gate credential probe on every worker (#46030) ... * test(integration): migrate before timed owned proxy launch and gate credential probe on every worker * test(integration): retry closed-connection samples and require both workers in the credential gate --------- Co-authored-by: yuneng <yuneng@berri.ai>	2026-10-12 02:14:18 -07:00
devin-ai-integration[bot]	85b1095f1e	test(integration): give the ownership proxy its own DB pool headroom (#46025) ... Co-authored-by: yuneng <yuneng@berri.ai>	2026-10-12 01:39:29 -07:00
devin-ai-integration[bot]	780c504e00	test: migrate tests/proxy_behavior into integration and unit tiers (#45939) ... * test: migrate tests/proxy_behavior into integration and unit tiers (partial) * test(integration): run settings-dependent management tests on an owned proxy * test(integration): assert stored values in the rewritten management rows * test(integration): pin sibling team allocations against the org guaranteed throughput cap * test(integration): keep Prisma clients out of the shared integration conftest import path --------- Co-authored-by: yuneng <yuneng@berri.ai>	2026-10-11 08:28:42 -07:00
devin-ai-integration[bot]	3090f4b23e	fix(ci): isolate integration workers and repair test fixtures (#45863) ... * test(integration): wait for customer spend rows and use owned OpenRouter cost * ci: isolate integration workers and clean up resources after crashes * docs: describe isolated extensions workers and shard census * fix(ci): restore integration service and fixture guarantees * fix(integration): preserve replay IDs and native route fixtures * fix(tests): scope integration harness imports to each unit case * fix(ci): preserve fixtures and wait for owned workers * fix(ci): isolate nested runs and telemetry cases * fix(ci): make worker readiness and fault tests deterministic * chore(ci): drop MCP fixture repair already covered by main * chore(ci): keep the local catalog fix from main --------- Co-authored-by: yuneng <yuneng@berri.ai>	2026-10-11 01:56:02 -07:00
devin-ai-integration[bot]	1dbde0845e	test: migrate SDK and proxy-booting legacy CircleCI tests to their tiers (#45758) ... * test: move guardrails, image, audio, batches, search legacy tests to unit * test: restore the custom guardrail and remaining chunk 1 unit rows * test: move the ocr matrix to e2e * test: drop comments from moved tests and fix vertex ocr credentials * test: move litellm utils and unified google legacy tests to their tiers * test: move logging callback and pass-through unit legacy tests to their tiers Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test: move store_model_in_db legacy tests and start the admin ui migration Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test: finish the proxy job rows and drop their CircleCI jobs Moves the remaining multi-instance, openai endpoint, otel, pass-through and top-level proxy tests into tests/integration with scripted upstreams, rewrites the weak replacements to assert the outbound wire and exact responses, and removes the build_and_test job group this PR owns from CircleCI * test: tighten the key generation and error log assertions, rename migrated unit files to their mapped names * test: shard the pagerduty unit tests and record the rust harness file on the coverage allowlist * test: drive the SSO sign-in tests through a scripted IdP and isolate the cost center stand-in * test: type the e2e beta header, prompt cache and secret manager values * test: assert the missing model error message on the OpenAI routes * test: drop the deleted otel base class import and restore mcp redaction and exact langfuse usage checks * test: make the langfuse fixture directory a package for the layout check * test: restore collected no-match cases, real langfuse and gcs payload checks, and gate the aws secret lane * test: keep the ocr e2e pdf fixture next to the test * test: put meta below covers on the streaming tool search e2e * test: run the OCR matrix against a scripted upstream and fix two runner-only e2e failures * test: count only OCR calls in the OCR matrix recorder * test: pin OCR parse bodies and billed cost, group new e2e cases into classes * ci: restore the python image anchor dropped in the main sync --------- Co-authored-by: yuneng <yuneng@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-10-11 00:17:21 -07:00
devin-ai-integration[bot]	ca0ff24726	fix(databricks): call the ai_decide AI Function route for decisions and the auto-router (#45722) ... * fix(databricks): call the ai_decide AI Function route for decisions and the auto-router Databricks serves ai_decide at POST <host>/api/2.0/ai-functions/ai-decide, not on a serving endpoint's /invocations route. The decisions provider and the auto-router classifier now accept only the ai_decide model, post to that route without a model field, unwrap the response envelope, and map a noul answer's probability to the System One noul field. * test(databricks): bound the megabyte model name test by scaling, not wall time * test(databricks): assert the whole OpenAI-format decision and floor the liveliness bound * test(decisions): move the Databricks OpenAI-format wire row to the ai_decide route * fix(databricks): keep serving endpoint decisions beside the ai_decide route * test(databricks): assert the whole classifier request body in the test routing case * test(databricks): correct System One spend call type Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> Co-authored-by: mateo <mateo@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-10-10 23:58:16 -07:00
devin-ai-integration[bot]	4c532a5713	feat(decisions): accept decisions as a model mode alongside evaluation (#45474) ... * refactor(decisions): rename the evaluation model mode to decisions Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(decisions): canonicalize the health check mode without rebinding Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(decisions): replace canonical_model_mode with is_decisions_model_mode Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(decisions): preserve evaluation mode compatibility Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * chore: drop the rebuilt dashboard bundle from this PR Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * style(dashboard): format decision mode helper Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: kerry <kerry@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-10-10 19:18:53 -07:00
yuneng-jiang	9b53ee878f	perf(ci): isolate and shard integration suites (#44146) ... * perf(ci): distribute unit cases and reuse security sweep connections * revert(ci): reject first speed trial after new failures * perf(ci): avoid repeated test discovery and parsing * perf(ci): isolate parallel security workers * revert(ci): restore measured security execution isolation * perf(tests): reuse source scans and deduplicate cache cleanup * perf(ci): restore parallel security with safe log batching * test(ci): isolate fixture state and schema comparisons * test(ci): make schema and scheduler fixtures deterministic * test(ci): drain queued logging before fixture teardown * test(ci): preserve logging delivery without idle worker restarts * test(ci): keep guardrail sinks available during parallel bursts * test(ci): isolate full proxy lifecycle from gateway imports * test(ci): preserve first failure responses and sweep evidence * test(ci): restore documentation URL settings between cases * test(ci): isolate open MCP tools in private gateways * test(ci): group private MCP cases and handle exited workers * test(ci): retain proxy exits and MCP failure details * test(ci): repair merge interactions with main Key the schema shard's full clone on artifact-name, since main removed the unit-flag input. Give each owned Redis its own log file again. Hand the realtime transcription tests main added a logging object whose dispatch handlers are awaitable, so the per-test logging drain no longer waits on a MagicMock task forever * perf(ci): shard owned integration cases and close lifecycle leaks * fix(ci): qualify shard execution and harden interrupted cleanup * fix(ci): bound worker supervision and own fault traffic * perf(ci): reuse unparametrized fixtures across bounded case groups * fix(ci): preserve boot fault allowances and complete exit records * test(ci): tolerate startup retries and retain readiness details * perf(ci): balance integration shards by recorded duration * fix(ci): load migration database helpers only for workers	2026-10-10 11:15:19 -07:00
devin-ai-integration[bot]	5282aed09b	fix(proxy): keep keys and projects on the same team (#44165) ... * fix(key mgmt): a project may only be attached to keys of its own team A project is created under exactly one team, and its budget and models are validated against that team's. Nothing checked that a key's team matched, so `POST /key/generate` with `team_id: team-a` and a `project_id` owned by `team-b` answered 200 and stored exactly that — a key belonging to team-a, recorded under team-b's project and validated against its models and budget. The same held with no team at all. It needs no proxy-admin rights: an admin of team-a who is a member of no other team can issue keys under another tenant's project with nothing but the project id, and that tenant sees the spend without having granted anything. `_check_project_key_limits` now takes the key's team and refuses a project owned by a different one, before the model and budget checks so the refusal reads as what it is. A project with no owning team is left alone — nobody owns it, so there is no boundary to cross. `/key/update` passes the key's stored team when the request does not carry one, and now runs the check whenever the project itself is set or changed, which a request that moves only the project previously skipped. Two existing cells needed the project's team passed explicitly: they measure the model allowlist, not tenancy, and would otherwise have been asserting model behaviour on a request the new gate refuses. A third builds an unowned project for the same reason. Fixes #41089 * fix(key mgmt): check project ownership on every key mutation path /key/update ran the project check only when project_id, models or max_budget were supplied, so a request that changed team_id alone left a foreign project attached. /key/regenerate never ran it at all. Both now go through one helper that validates the key as the mutation leaves it. On /key/generate the check moves after default_key_generate_params is applied, because that can supply team_id; it was rejecting a valid key whose team came from the defaults. Tests move into the mapped test file per CLAUDE.md. * test(key mgmt): read the project from the cache instead of patching the lookup The test-quality gate rejected the new cells: 12 TQ008 for patching `get_project_object`, an SDK internal, and 4 TQ001 for accept controls that could only fail by raising. The cells now seed `UserApiKeyCache` with the project, which is the idiom the neighbouring project cells already use and removes the patching. The accept controls are parametrised together with the rejecting ones, so each test function carries a real assertion. The three regenerate seams that stay stubbed each carry a reason. * fix(proxy): prevent cross-team project keys Co-authored-by: L4XB <lukas.buck@e-mail.de> Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): validate bulk key team changes Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(proxy): use behavioral assertions in project ownership tests Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): read project owner from the database for key ownership checks Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): read project ownership from the primary database under the lookup deadline Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): check project moves against the primary database team Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): run ownership checks after existing validation without the lookup deadline Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): type ownership writer reads Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(proxy): make ownership tests independent of runner salt and clock Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): check key project ownership after existing validation Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): run key project ownership right before the key row write Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): reject key project ownership before permission and budget writes Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): keep project object permission validation on the stored payload Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * style(proxy): format key regeneration update payload Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): invalidate the written permission id on bulk update and regenerate Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(proxy): audit project team ownership across endpoints, bulk paths and chaos Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): drop budget and permission rows written for a rejected cross-team key Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(keys): require project-team membership when regenerate binds a key to a team-owned project /key/{key}/regenerate and /key/regenerate accepted a team_id + project_id pair without checking that the caller belongs to the destination team, so a user with no team could regenerate their own key into another team and its project. Regenerate now requires a proxy admin or a member/admin of the project's team whenever the change lands the key on a team-owned project. Also adapts tests to main's premium_user_check rename, drops a duplicate import left by the merge, and replaces the created object-permission id cast with an isinstance check. * fix(keys): 404 a missing project before any key write, and match the project move refusal to its siblings A missing project_id on /key/service-account/generate and on both regenerate routes fell through to a foreign-key 500; regenerate had already written a deleted-token history row, and service-account generate left its budget and object-permission rows behind. The ownership check now answers 404 with the same message /key/generate uses, before any write, and the generate rollback covers it. The /project/update move refusal now uses ProxyException (bad_request, param team_id) like the delete-with-keys refusal. Tests: integration coverage for a team admin regenerating a team key into another team's project (403), the missing-project 404s with no history rows, and the unit test that never reached the ownership check is removed. * test(keys): keep the existing-permission case in the regeneration cache eviction test Parametrize the test over a key that already has an object_permission_id and one that mints a new row, so eviction under the key's existing permission id stays covered. --------- Co-authored-by: L4XB <lukas.buck@e-mail.de> Co-authored-by: yucheng <yucheng@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-10-10 00:53:10 -07:00
Mateo Wang	883e3210ee	feat(decisions): add Databricks ai_decide as a /v1/decisions provider and auto-router decider (#45200) ... * feat(decisions): add Databricks ai_decide as a /v1/decisions provider and auto-router decider * fix(decisions): reject Databricks endpoint names carrying URL separators and keep the provider wiring under llms/ * fix(decisions): reject dot-segment Databricks endpoint names and add Databricks to the dashboard's OSS classifier editor * fix(ui): name every OSS classifier provider in the classifier radio copy * feat(decisions): register the Databricks OpenJev decider as an evaluation-mode cost-map entry * fix(cost-map): declare zero cache rates on the databricks-openjev-qwen35-4b entry * fix(proxy): reconcile every DB deployment when a registry miss lands on an auto router A /model/new for an auto router lands on one replica. A request routed to that auto router on a sibling replica missed it, the model read-through loaded only the auto router's own row, and the pre-routing hook then picked a tier the sibling had not loaded, so the caller got a 400 "no healthy deployments for <tier>" until the next periodic reload. An auto router miss now runs the full add_deployment reconcile, tiers included. Found by the audit's two-worker rig on this PR's routing cells; CircleCI's one-worker proxy never opens the window. * test(integration): audit cells for Databricks decisions and the Databricks auto-router classifier Wire cells for /v1/decisions with databricks/<endpoint> deployments, routing cells for the complexity router's databricks classifier on all three endpoints with the Postgres rows read back, management cells for the classifier's spend writes and for the evaluation-mode health check of a Databricks serving endpoint.	2026-10-09 16:19:23 -07:00