Commit graph

5922 commits

Author SHA1 Message Date
devin-ai-integration[bot]
cad49ee171
fix(proxy): gate disable_global_guardrails on keys and teams to proxy admins (#42699)
* fix(proxy): gate disable_global_guardrails on keys and teams to proxy admins

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test: cover metadata smuggle with explicit false and UI toggle gating

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(proxy): satisfy PT017 in resend-stored guardrail flag test

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(proxy): keep regenerate_key_fn under the C901 ceiling via a guardrail opt-out helper

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* chore(ui): regenerate schema.d.ts for guardrail opt-out docstrings

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): gate disable_global_guardrails on caller-sent metadata, not server defaults

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): audit cells for disable_global_guardrails admin gate

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): restore contracts.json formatting

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): share guardrail opt-out helpers

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(ui): hide the team disable_global_guardrails switch from non proxy admins

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): drop covers markers and bound the slow sink check to the sink delay

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-23 18:03:02 -07:00
devin-ai-integration[bot]
1e5403288c
feat(proxy): honor model_info.discoverable on the model listing endpoints (#42825)
Some checks failed
Unit Tests: Proxy DB Operations / budgets (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / custom-logging (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / db-and-spend (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / endpoints-and-responses (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / guardrails-hooks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / jwt-and-keys (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / key-generation (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / logging-misc (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-runtime (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-server-core (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-utils (push) Blocked by required conditions
Unit Tests / caching-local (push) Waiting to run
Unit Tests / core-utils (push) Waiting to run
Unit Tests / enterprise-package (push) Waiting to run
Unit Tests / enterprise-routing (push) Waiting to run
Unit Tests / integrations (push) Waiting to run
Unit Tests / All Other Providers (push) Waiting to run
Unit Tests / Vertex AI (push) Waiting to run
Unit Tests / mcp-integration (push) Waiting to run
Unit Tests / misc (push) Waiting to run
Unit Tests / proxy-auth (push) Waiting to run
Unit Tests / proxy-endpoints (push) Waiting to run
Unit Tests / proxy-extras (push) Waiting to run
Unit Tests / proxy-infra (push) Waiting to run
Unit Tests / proxy-server (push) Waiting to run
Unit Tests / responses-caching-types (push) Waiting to run
GitHub Actions Security Analysis / zizmor (push) Waiting to run
Terraform Modules / fmt, validate, test (gcp) (push) Has been cancelled
Terraform Modules / fmt, validate, test (aws) (push) Has been cancelled
VS Code Extension / vscode-extension (push) Has been cancelled
* feat(proxy): honor model_info.discoverable on the model listing endpoints

A model_list entry marked model_info: {discoverable: false} is left out of GET /v1/models (OpenAI and Anthropic shapes, scope=expand and wildcard routes included), the list path of GET /v1/model/info and GET /model_group/info for every caller without the admin view, while direct requests naming the model keep routing to it. The field defaults to None so an absent flag reads as discoverable and nothing is persisted or echoed for configs that never set it.

* fix(proxy): hide flagged team models under their public name and cover the scope=expand filter

The discoverability lookup now resolves a listed name with the caller's team context, so a team-scoped deployment marked discoverable: false drops out for that team's keys under its public name instead of failing open. The scope=expand branch is now exercised by a team admin caller, and the OCI secrets test builds a real UserAPIKeyAuth instead of a spec mock that has no pydantic fields.

* perf(proxy): resolve only candidate names in the discoverable filter

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-09-23 17:51:16 -07:00
devin-ai-integration[bot]
ce582affaa
fix(mcp): reject duplicate MCP server names and aliases (#42791)
* fix(mcp): reject duplicate MCP server names and aliases

MCP server_name and alias were unchecked at write time, so two servers
could share one tool prefix and tool routing resolved to an arbitrary
winner. Writes now run inside an advisory-locked transaction that
rejects a collision on either column case-insensitively with a 400
naming the colliding identifier, covering create, edit, connector
import and restricted-admin submission. Server reload logs one warning
per identifier already shared in the database.

Co-Authored-By: bot_apk <apk@cognition.ai>

* fix(ui): block duplicate MCP server names and aliases before submit

The create and edit forms now check the normalized name/alias against
the loaded server list (case-insensitive, spaces to underscores, own
row excluded on edit) and show a field error instead of submitting.
Structured proxy error bodies are unwrapped so a 400 no longer renders
as 'Error: [object Object]'.

Co-Authored-By: bot_apk <apk@cognition.ai>

* fix(mcp): check identifier conflicts when an alias is cleared

Clearing an alias drops the tool prefix to the stored server_name, so
that name must go through the conflict check too; an explicit alias:null
is now treated as an identifier write. Also narrows the new db tests to
behavioral assertions instead of pinning prisma where shapes.

Co-Authored-By: bot_apk <apk@cognition.ai>

* fix(mcp): treat an empty alias as a clear in conflict checks

An empty-string alias was written unchecked even though the prefix falls
back to server_name; the update path now treats any falsy alias like a
clear. The edit form likewise compares a cleared alias as empty instead
of re-checking the alias being removed.

Co-Authored-By: bot_apk <apk@cognition.ai>

* test(mcp): cover clearing an alias to an empty string

Co-Authored-By: bot_apk <apk@cognition.ai>

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: bot_apk <apk@cognition.ai>
2026-09-23 17:50:27 -07:00
tin-berri
b1194aa373
fix(ui): show ten prompt caching requests per page (#42638) 2026-09-23 17:49:11 -07:00
tin-berri
0c1c3e18d5
fix(ui): prefer native providers in auto-router presets (#42639) 2026-09-23 17:33:22 -07:00
tin-berri
bab86555e7
fix(ui): hide LiteAdmin while Playground is open (#42755) 2026-09-23 17:32:22 -07:00
devin-ai-integration[bot]
9fd25b2228
fix(ui): keep per-user MCP credentials updatable and clearable after setup (#42652)
* fix(ui): keep per-user MCP credentials updatable and clearable after setup

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(ui): keep card keyboard activation off nested credential buttons

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(ui): confirm before clearing saved per-user MCP credentials

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(ui): reset the clear confirmation when the credentials modal closes

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(mcp): cover per-user env var edge paths in browser and integration contracts

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): wait for the peer process to grant the key before listing its MCP tools

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(e2e): drop the mutable removed flag from the deleted-server browser contract

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: ryan <ryan@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-23 11:51:05 -07:00
devin-ai-integration[bot]
04b1b077de
fix(ui): keep untouched stored auto-router booleans and reminder marker casing on save (#42703)
Co-authored-by: yuneng <yuneng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-23 07:54:54 -07:00
Oliver Jensen
bc3b5b1d5b
fix(proxy): revoke UI session tokens on logout and password change (#42463)
* fix(proxy): revoke UI session tokens on logout and password change

Adds POST /session/logout to revoke the presented UI session key server
side (previously logout was client-side only and the key stayed valid
until expiry). Password changes now revoke the user's other UI sessions:
self-change keeps the caller's session, admin reset and onboarding claim
revoke all. The BYOK OAuth cookie auth now re-resolves the embedded key
against the DB so revoked sessions get a 401.

* fix(proxy): satisfy B008 budget and backend allowlist for /session/logout

* refactor(proxy): satisfy type-discipline budget in session_endpoints
2026-09-23 10:31:38 +02:00
devin-ai-integration[bot]
f44052d87b
fix(vector_stores): keep config-defined vector stores listed and read-only (#42574)
* fix(vector_stores): keep config-defined vector stores listed and read-only

Vector stores declared in config.yaml were purged from the in-memory registry by /vector_store/list because the database was treated as the only source of truth. Config-defined stores now carry is_config=True, stay in the list beside database rows, are never overwritten or evicted by database state, and reject /vector_store/new, /vector_store/update and /vector_store/delete with 400. The Admin UI renders them read-only

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(ui): show vector store source and read-only state for config-defined stores

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): audit config-owned vector stores across list, writes, search, authz, peers and redis outage

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(ui): show a visible read-only hint in the config vector store actions menu

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: ryan <ryan@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-23 04:02:16 +00:00
tin-berri
5b287f66d7
feat(ui): add native LiteAdmin assistant (#42443) 2026-09-22 18:06:40 -07:00
moe-berri
2157351004
feat(ui): simplify auto-router setup and clarify feature limits (#42625)
* feat(ui): simplify auto-router setup and clarify feature limits

* fix(ui): validate auto-router drafts before saving

* fix: keep auto-router allowances consistent after deletes and refreshes
2026-09-22 18:03:32 -07:00
devin-ai-integration[bot]
da82ea8e94
fix(ui): let the Create Key user picker find users by user_id, not just email (#41687)
* feat(ui): search users by id or email when assigning a key owner

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(ui): label users without an email by user id in key owner picker

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): freeze merged user-filter where, format create key test

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(proxy): suppress module-global patch findings in ui_view_users search test

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): keep merged user-filter where as a plain dict for prisma serialization

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(ui): forward search param from userFilterUICall to /user/filter/ui

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(ui): mention user ID in the Create Key user picker helper text

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: jesus <jesus@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-22 16:30:29 -07:00
togear
fc0055497c
feat: add configurable provider affinity header mapping (#41033)
* feat: add configurable provider affinity header mapping

* fix: sync provider affinity API types

* fix: harden provider affinity header mapping

* fix: avoid provider affinity import cycle

* fix: preserve input callback header mutations

* fix: address provider affinity code scanning findings

* fix: satisfy provider affinity type discipline gate

* test: cover omitted pre-call argument isolation

* fix: resolve remaining provider affinity codeql alerts

* fix: redact provider affinity headers after calls

* refactor: drop provider affinity header log redaction

* fix: reject control characters in affinity session ids as a bad request

* chore: regenerate the openapi snapshot on python 3.12 and reuse the session marker constant

* fix(responses): read the affinity session from the named metadata argument

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-09-22 13:31:33 -07:00
tin-berri
e841f32e50
fix(ui): rename reminder markers to Ignore Custom Tags (#42370) 2026-09-22 13:16:18 -07:00
tin-berri
3b9c9e0523
feat(router): add group-scoped priority routing strategy (#42378)
* feat(router): add group-scoped priority routing strategy

* fix(router): satisfy priority routing type-discipline checks
2026-09-22 13:09:38 -07:00
devin-ai-integration[bot]
1373322e0c
feat(logs): add span type filter to request logs (#42491)
* feat(logs): add span type filter to request logs

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(logs): look up span type sql conditions from a mapping

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Mubashir Osmani <mubashir@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-22 12:44:03 -07:00
devin-ai-integration[bot]
5035c458fb
feat(proxy): admin-only /debug/report sharing the bug report environment (#42440)
* feat(proxy): add admin-only /debug/report sharing the bug report environment fields

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* feat(proxy): add verbose=true to /debug/report listing every config key with typed values

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): inject auth into /debug/report through Annotated to keep the B008 budget flat

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): bound the verbose config walk, drop nested-list recursion from the safe renderer, regenerate schema.d.ts

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): count pass-through and mcp header maps plus operator-named budget maps in verbose /debug/report, single-exit scalar renderers

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* chore(ui): regenerate schema.d.ts after dropping the verbose query from /debug/report

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: ryan <ryan@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-22 12:05:23 -07:00
devin-ai-integration[bot]
21c442759e
feat(router): time-windowed team reservation of deployments via model_info.access_windows (#42398)
* feat(router): time-windowed team reservation of deployments via model_info.access_windows

Deployments can declare model_info.access_windows, a list of local wall-clock windows (IANA timezone, cross-midnight allowed) that reserve the deployment for the listed team_ids. While a window is active the router drops the deployment for every other request, including no-team and admin requests, on every candidate path (model name, model id, specific_deployment, early-resolve, wildcard, litellm_params.model lookup, fallbacks). If every candidate is reserved the request fails with a 400 naming the window end instead of falling back. Outside a window routing is unchanged and reserved deployments stay visible in /model/info and /v1/models. Malformed windows (bad time, unknown timezone, empty team_ids, start equal to end, offset-aware times) fail proxy startup with a clear error since the proxy router runs with ignore_invalid_deployments=True

Resolves LIT-8308

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(router): cover _filter_reserved_deployments directly for coverage gate

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(router): keep reservation filtering immutable

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(router): drop strategy markers before reservation filtering

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-22 13:22:15 -05:00
devin-ai-integration[bot]
ded69f8d03
fix(ui): surface the owner's user budget on keys without their own budget (#38220)
* fix(ui): surface the owner's user budget on keys without their own budget

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(ui): apply the owner's budget hint to team keys when apply_user_budget_to_team_keys is on

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(ui): read only the apply_user_budget_to_team_keys flag from general_settings

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): mark the general_settings cast as cast-ok

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(ui): load the owner's budget for keys opened outside the current page

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: jesus <jesus@berri.ai>
2026-09-22 10:53:10 -07:00
Mateo Wang
deba473821
fix(cost): bill batch prompts above 272K at OpenAI's long-context batch tier (#39861)
* fix(cost): bill batch prompts above 272K at OpenAI's long-context batch tier

* fix(cost): mirror batch long-context keys on custom pricing params

Register the two *_above_272k_tokens_batches keys on CustomPricingLiteLLMParams so a per-deployment override stays out of the shared backend key, add them to the inline model-info schema and alias-count tests, and build LiteLLM_Params and GenericLiteLLMParams through model_validate at the two dict-splat call sites so basedpyright's reportArgumentType budget ratchets down instead of blocking the new fields.

* fix(cost): add the gpt-5.5-pro batch long-context tier and ignore malformed batch tier keys

* fix(cost): bill cached batch tokens at OpenAI's cached batch rate

Adds cache_read_input_token_cost_batches and
cache_read_input_token_cost_above_272k_tokens_batches for the tiered
OpenAI entries at half the standard cached rate, bills cached batch
tokens at that rate per output line, and parses string-valued batch
rates in deployment-level model_info.

* fix(cost): bill batch cache writes at the batch cache-write rate and carry published batch rates for one-sided deployments

OpenAI's Batch table prices cache writes for gpt-6-astra, gpt-5.6, gpt-5.6-sol, gpt-5.6-terra and gpt-5.6-luna at half the standard cache-write rate, so the cost map gains cache_creation_input_token_cost_batches and its above_272k tier for those entries and batch cost pulls written tokens out of the input bucket at that rate; models without the key keep billing writes at the batch input rate.

A deployment declaring only one side of its batch pricing now carries every published batch rate of the other side (tier, cached, cache write), its own keys win, and a lone tier, cached or cache-write batch key counts as declared pricing instead of being ignored.

* fix(cost): select the batch long-context tier from any batch tier key

A deployment that declares its own flat standard input rate keeps every
published batch rate of the output direction, including the 272K output
tier, but the tier was only ever selected when an input tier key was also
present. Detect the crossed tier from any of the four batch tier keys so
the carried output, cache-read, and cache-write tiers bill at their tier
rate above 272K tokens.

* chore(proxy): keep the OpenAPI snapshot as CI generates it

* fix(cost): pick each batch price component's tier from its own keys

The batch rate picker crossed one threshold for every component, so a
deployment declaring only an output tier also moved its input, cached, and
cache-write rates to that cutoff. Each component now crosses its own
*_above_<N>k_tokens_batches keys and falls back to its flat key.

The JSON schema is regenerated with the generator as it is on main:
cost-map-guard renders the PR's cost map with the base branch's generator,
so the descriptions for the new batch cache keys move to a follow-up.

* chore(proxy): restore the lazy OpenAPI snapshot to what CI's Python 3.12 generates

The merge commit carried a snapshot regenerated on a Python 3.14 venv, which dedents
docstrings at compile time, so one description line differed from the file CI regenerates
on 3.12 and the schema.d.ts sync check went red. The snapshot is byte-identical to main again
2026-09-22 10:22:41 -07:00
devin-ai-integration[bot]
b673ee61e7
feat(arize): per-team success and error sampling rates for the Arize AX callback (#42383)
* feat(arize): per-team success and error sampling rates for the Arize AX callback

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(arize): fail open on invalid sampling rates and type the new sampling code

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(arize): type the sampling test helpers and parametrized fixture

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-22 11:21:40 -05:00
joshua-berri
25af172b85
fix(mcp): keep config-defined servers read-only (#42299)
* fix(mcp): persist config server edits in the database

* test(mcp): cover config loading and failed promotion responses

* test(mcp): preserve auth policy during config server promotion

* fix(mcp): preserve existing YAML metadata during config loading

* fix(mcp): reuse bounded traversal for config secret checks

* fix(mcp): honor database access groups after config promotion

* fix(mcp): keep config-defined servers read-only

* fix(ui): clear frontend warnings and require warning-free green checks

* fix(ui): preserve legacy MCP access group labels

* ci: remove remaining frontend action runtime warnings

* fix(mcp): limit read-only fix to ticket scope

* fix(mcp): preserve API stability and remove unrelated guidance

---------

Co-authored-by: Joshua Valluru <326636767+joshua-berri@users.noreply.github.com>
2026-09-21 22:56:21 -07:00
tin-berri
5a8c4f48e4
feat(router): native compact-to-fit across conversation APIs (#42074)
* feat(router): native compact-to-fit across conversation APIs

* fix(router): preserve compaction admission and shared client boundaries

* fix(router): honor compaction fit fallbacks and router-scoped access

* fix(router): charge compaction usage to caller token limits

* test(http): keep FastAPI inside proxy tests

* fix(router): check compactor capacity before skipping escalation
2026-09-21 22:52:29 -07:00
Zachary Lyon
19c13ac971
feat(proxy): add TinyFish Agent API passthrough with per-step billing (#41099)
* feat(proxy): add TinyFish Agent API passthrough with per-step billing

* chore(ui): regenerate dashboard API types for /tinyfish passthrough

* fix(proxy): satisfy strict lint budget for tinyfish passthrough

* style: ruff format tinyfish passthrough handler

* test(proxy): exercise tinyfish route through the app with a faked upstream

* refactor(proxy): make cross-module tinyfish billing hooks public

* fix(proxy): tolerate transient tinyfish poll failures instead of dropping the charge

* fix(proxy): defer billing for disconnected tinyfish SSE runs to the background poller

* Revert "fix(proxy): defer billing for disconnected tinyfish SSE runs to the background poller"

This reverts commit ef0bcfb4a0.

* fix(proxy): bill tinyfish SSE runs via detached poller and only COMPLETED runs

Disconnected run-sse clients previously left completed runs unbilled: the stream-end
handler saw a still-RUNNING run and logged $0. The poller now spawns from the streaming
path on the first run_id frame, outlives the disconnect, and writes the one spend row when
the run turns terminal; the stream-end path only logs the $0 fallback for run_id-less
streams. Costs now apply only to COMPLETED runs ($0 for FAILED/CANCELLED, matching the
upstream invoice), spend rows carry the request's litellm_call_id (previously NULL
request_ids collided and were silently dropped), and the GET /v1/runs listing is blocked
so callers behind the shared key cannot discover each other's runs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pxz11TQKYvCXvKawNUa2g4

* fix(proxy): drop GET /v1/runs from the tinyfish allowlist error message

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pxz11TQKYvCXvKawNUa2g4

* chore(proxy): sync openapi artifacts for tinyfish docstring, suppress LIT011 on flag write

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pxz11TQKYvCXvKawNUa2g4

* style(proxy): ruff-format the sse poller flag write

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pxz11TQKYvCXvKawNUa2g4

* style(proxy): keep the rebind-ok suppression on the flag write's own line

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pxz11TQKYvCXvKawNUa2g4

* fix(proxy): harden tinyfish billing paths from review findings

Skip failure dispatch when the SSE poller owns billing (a failure row collided with the
poller's billed row on request_id and dropped the charge), late-spawn the poller for
run_ids that arrive in unterminated frames instead of mispricing RUNNING runs at $0,
thread litellm_params into poller-billed standard logging objects so SLO consumers see
attribution, untype the run error field so upstream error-shape drift cannot void a
billable run, normalize a schemeless TINYFISH_AGENT_API_BASE, extend the poll budget to
cover queue wait (3600s) with ~60s outage tolerance, and log poller cancellation.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pxz11TQKYvCXvKawNUa2g4

* chore(proxy): satisfy ratcheted BLE001/LIT002 budgets from main in tinyfish handler

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pxz11TQKYvCXvKawNUa2g4

* style(proxy): drop stray blank line from merge resolution

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pxz11TQKYvCXvKawNUa2g4

* fix(proxy): reject passthrough envelope controls on tinyfish route, raise blocking-run timeout

The generic passthrough unwraps a caller-supplied custom_body as the forwarded request and
honors a caller stream flag, so custom_body.use_vault bypassed the credentialed-run 403 and
stream: true flipped a blocking run into the streaming pipeline. The route now 400s the
envelope fields (custom_body, stream, query_params); streaming comes from the endpoint.
Blocking runs also get a 1500s default timeout covering the upstream 1200s run cap, unless
the operator configured pass_through_request_timeout.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pxz11TQKYvCXvKawNUa2g4

* style(proxy): resolve operator timeout without a dict-literal default

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pxz11TQKYvCXvKawNUa2g4

* test(passthrough): list the TinyFish route among protocol-constrained pass-through routes

* chore(proxy): regenerate the lazy OpenAPI snapshot after merging main

* chore(proxy): keep the lazy OpenAPI snapshot as CI's Python 3.12 renders it

* fix(proxy): reject TinyFish POST bodies that are not a JSON object

A form-encoded or text body carried stream and use_vault past both field
gates, because the gates only saw fields the body parsed to as JSON. The
route now checks the content type before reading the body and answers 400
for anything that is not a JSON object.

* fix(tinyfish): reject submit paths with extra slashes so run-async always bills

The allowlist dropped empty path segments, so POST /v1/automation/run-async/
was forwarded upstream while the billing dispatch only recognises the exact
path and would have logged the submit at $0 without starting the poller.
Any path with a trailing or doubled slash now returns 403 before forwarding.

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-09-21 21:21:43 -07:00
devin-ai-integration[bot]
0fd1c191ca
feat(fal_ai): add queue-only /fal_ai pass-through route with spend tracking (#42360) 2026-09-22 02:59:58 +00:00
devin-ai-integration[bot]
1106b16745
feat(openrouter): price typesafe/jev-1.13 and add an openrouter decisions pass-through (#42301) 2026-09-22 02:44:15 +00:00
devin-ai-integration[bot]
8b33da7bb3
feat(proxy): opt-in litellm_call_id in JSON error bodies (#42391)
* feat(proxy): opt-in litellm_call_id in JSON error bodies

Add general_settings.include_call_id_in_error_body. When true, the value
already on the x-litellm-call-id response header is copied into JSON error
bodies: as error.litellm_call_id on the OpenAI-shaped routes, /v1/messages,
and streaming first-chunk errors, and as a top-level litellm_call_id on
pass-through routes. Off by default, so error bodies stay byte-identical
unless an admin opts in

* chore(proxy): drop helper docstring and restore lazy OpenAPI snapshot

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-09-21 19:17:18 -07:00
jesus-berri
f6d5b28443
Merge pull request #39578 from Louis-Vauterin/jwt-key-mapping-token-id
feat(jwt-key-mapping): accept token_id as an alternative to the plaintext key
2026-09-21 18:22:54 -07:00
yuneng-jiang
a59ecfb7c5
Merge pull request #42367 from BerriAI/litellm_team_usage_export_per_user
feat(ui): add per-user breakdown to team usage export
2026-09-21 18:07:54 -07:00
tin-berri
275c0c4d96
Merge pull request #42057 from BerriAI/litellm_classifier_forecast_cards
feat(ui): show Capability and FUSE v2 routing forecasts
2026-09-21 17:55:25 -07:00
yuneng
49126d4434 test(ui): pin the clock in the per-user export filename test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-22 00:34:59 +00:00
yuneng
86a019128e fix(ui): key per-user export buckets on a collision-free tuple
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-22 00:25:13 +00:00
yucheng-berri
12f79308cf
Merge pull request #42354 from BerriAI/litellm_fix_video_key_guardrails_lit_6685
* fix(guardrails): scan video prompts for key-attached guardrails on /v1/videos

/v1/videos dispatches call_type avideo_generation, which CallTypes did not
know and no guardrail translation handler covered, so the unified guardrail
hook returned the request unscanned. Add the video call types and an OpenAI
video guardrail translation package that scans the prompt for create, remix,
edit and extension requests

Resolves LIT-6685

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* chore(ui): regenerate api types for video call types

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test: skip avideo_generation in azure sdk client exhaustive check

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(e2e): retry a leaked video job until the guardrail sync deadline

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(guardrails): satisfy the type-discipline gate in the video handler

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(guardrails): gate the video e2e on a chat probe so a miss starts at most one paid job

Addresses Greptile review: typed RewritingGuardrail override, dropped routine docstrings, and the e2e waits for the key guardrail to sync via /chat/completions before its single /v1/videos call

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 17:23:56 -07:00
Yassin Kortam
9cc5b78c33
Merge pull request #41634 from BerriAI/litellm_agent_access_groups
feat(agents): attach access groups to agents and enforce them for models, MCP servers and agent calls
2026-09-21 19:13:46 -05:00
tin-berri
094a60bb9c
Merge pull request #41872 from BerriAI/litellm_context_escalation_opt_in
fix(router): make context-window escalation opt-in
2026-09-21 17:03:33 -07:00
yuneng-jiang
1f4b595072
Merge pull request #41788 from BerriAI/litellm_config_read_source
feat(proxy): report the source of alerting, UI and router settings on read
2026-09-21 17:02:18 -07:00
yuneng-jiang
a158aa878f
Merge pull request #41773 from BerriAI/litellm_dashboard-form-happy-paths
test(ui): cover dashboard form journeys
2026-09-21 16:56:25 -07:00
Tin Chi Lo
9221109d18 chore(router): resolve merge conflict with main 2026-09-21 16:52:47 -07:00
Tin Chi Lo
21fb652586 chore: merge main to restore lint checker 2026-09-21 16:44:08 -07:00
yuneng-jiang
c21ab96741
Merge pull request #42293 from BerriAI/litellm_complexity_router_advanced_fields
feat(ui): expose remaining complexity router advanced settings
2026-09-21 16:43:50 -07:00
yuneng-jiang
6b254b6be2
Merge pull request #41769 from BerriAI/litellm_model-actor-edges
test(model_management): cover actor edges and wildcard models
2026-09-21 16:43:24 -07:00
yuneng
a2163a2831 style(ui): format entity usage export files with prettier
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 23:41:27 +00:00
tin-berri
c1c1ec4f84
Merge pull request #42026 from BerriAI/litellm_user_jwt_savings
feat(ui): add internal-user savings and auto-router usage
2026-09-21 16:41:16 -07:00
yuneng
b7d1423ad3 feat(ui): add per-user breakdown to team usage export
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 23:26:12 +00:00
yuneng
beceb1bedb test(ui): cover per-user team usage export
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 23:23:15 +00:00
joshua-berri
1baa26d36b
Merge pull request #41074 from BerriAI/litellm_mcp_server_list_stable_order
fix(mcp): keep server lists stable across refreshes
2026-09-21 23:06:09 +00:00
Yuneng Jiang
143725fc06
Merge remote-tracking branch 'origin/main' into litellm_config_read_source 2026-09-21 15:54:52 -07:00
yucheng
8ffca3bd19 chore(ui): regenerate api types for video call types
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 22:47:14 +00:00
ryan
8a1c6e49fb feat(auth): only allow password-login dashboard sessions to call /user/password/change
Password login now stamps login_method=username_password into the UI session key
metadata, and change_password rejects any caller that is not a litellm-dashboard
key carrying that marker with 403 before the user row is read. SSO sessions and
user-associated virtual keys can no longer use the endpoint as a current_password
guessing oracle. The forced-reset session is still minted by the password login
path, so it keeps access to the endpoint

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 22:22:12 +00:00