Commit graph

14308 commits

Author SHA1 Message Date
Yucheng Zhu
b21bf9204f fix(team): pin the family's configured variables as well as its values
Asking only whether a value is one the family holds let a held variable be
given another of the family's values, so the exporter would address or
authenticate with it.

Keep the value membership rule for a variable the family does not configure
yet, which is what lets one credential go in under its other spelling, and
require a variable it does configure to keep the value it has. Between them no
value the caller chose can enter the family.
2026-09-01 14:45:45 -07:00
Yucheng Zhu
c0282621c7 fix(team): compare the family's values, not its variable names
Comparing per variable rejected a credential written under its other spelling:
langfuse_secret and langfuse_secret_key are one key, so repeating the stored
secret under the other name read as a new value.

Ask instead whether the value is one the owning entries already carry. A
destination the caller controls is by definition not, so the redirect stays
closed, and no alias table has to stay complete for that to hold.
2026-09-01 14:28:02 -07:00
Yucheng Zhu
bf29ecf115 fix(team): let one integration cover both callback events
The family rule compared variable names only, so a team admin who registered
an integration for the success event could not register the same integration,
with the same values, for the failure event.

Compare the values as well: repeating what the owning entry already stores
flattens to the same dict, so there is nothing to redirect. The stored side is
decrypted first, because the credentials are encrypted at rest and ciphertext
never equals the plaintext coming in.
2026-09-01 14:22:32 -07:00
yucheng-berri
75b717ba73 fix(team): make one entry own a credential family end to end
Every stored entry's callback_vars are flattened into one dict before a
request reads them, and that dict is what the exporter authenticates and
addresses with. So an entry naming only a destination is enough to
redirect a credential written somewhere else: a host on a second entry
pairs with the key pair from the first, and the request carries that key
pair to the new host. A team admin cannot read the team's masked Langfuse
secret, but could add such an entry and receive it.

Reject, for writers who are not proxy admins, an entry using a credential
family another entry already holds. Family rather than callback name,
because langfuse and langfuse_otel configure one Langfuse project and
would otherwise redirect each other, and because a destination like
dd_agent_host that no integration registry lists still pairs with the
Datadog credentials beside it.

A proxy admin already holds every credential the proxy has, so the rule
would buy nothing there and would break configs that predate it. A team
admin who does want to move a family deletes the entry holding it first,
which reveals nothing.
2026-09-01 13:25:44 -07:00
yucheng-berri
58a3ffd2a6 test(auth): pin that the callback grant reaches no neighbouring team route
The grant is two templates ending in the callback suffix, and the
placeholder now takes slashes and colons. Every other route under
/team/{team_id} registers an ordinary single-segment placeholder, so no
URL the router sends to one of them can end in the callback suffix.

Pin that, so adding a path-converter route beside these fails here
rather than by handing a caller a handler the grant never covered.
2026-09-01 13:06:40 -07:00
yucheng-berri
db80501232 fix(auth): match a :path placeholder the way the router's converter does
A team id may carry a slash, a colon, or both. The gate expanded {x:path}
to "[^:]+", so an id with a colon in it matched no self_managed_routes
entry and its team admin got the proxy-admin-only denial on a route the
router had already resolved for them. Listing a second {x} spelling covered
a colon or a slash but never both.

Expand {x:path} to ".+" instead, except when the template puts a ":"
literal of its own after the placeholder, which is where the narrower form
was earning its keep: the Google routes end in ":generateContent" and
friends, and there the value has to stop before that suffix rather than
swallow it and match a different verb.

That lets self_managed_routes drop back to the two :path spellings the
router itself mounts.
2026-09-01 12:25:35 -07:00
Yucheng Zhu
2e0d81a9da fix(team): reach the callback routes for a team id containing a colon
The route gate expands {team_id:path} to "[^:]+" so a colon-suffixed provider
route is not swallowed, which means the two entries added here matched a team
id with a slash but not one with a colon, while the router accepts both.
team_id is a free-form string, so a team whose id contains a colon kept the
old proxy-admin-only denial and its admin could not manage its own callbacks.
List both spellings rather than relaxing the shared matcher, which every
":path" route depends on. The comment claimed the two matchers agree; they do
not, so it now says what each placeholder actually accepts.
2026-09-01 12:25:35 -07:00
Yucheng Zhu
e64b6f6a68 feat(team): let a team admin manage their own team's logging callbacks
The team callback endpoints already authorize correctly: POST, GET and DELETE
each call _verify_team_access, which admits a proxy admin, an org admin for the
team, or an admin of that team, and 403s everyone else. The route-permission
layer never let a team admin reach them, so it answered 401 naming proxy admin
and the handler's own check was dead code for the caller it was written for.

Adding the two paths to self_managed_routes is how every other team-admin route
works: /team/member_add, /team/member_delete, /team/member_update and
/team/permissions_update all sit in that list and scope per team inside the
handler. The entries use the :path converter the routes are registered with, so
a team id containing a slash resolves the same way at the gate as at the router.

Because any authenticated caller now reaches these handlers, an unknown team had
to stop being distinguishable from one the caller may not manage. All three
handlers looked the team up and raised a distinct 'does not exist' before the
access check, which would have let any valid key probe for team ids. That branch
now returns the same 403 body _verify_team_access raises, and keeps the
diagnosable error for a proxy admin.

disable_logging stays out of the grant. That is a scope decision rather than a
security boundary, since a team admin holding DELETE can clear callbacks one at
a time; it differs only in also clearing the deprecated callback_settings shape.
2026-09-01 12:25:34 -07:00
Mateo Wang
c50d83ece2
Merge pull request #39070 from BerriAI/litellm_bedrock_invoke_native_structured_output
fix(bedrock): forward native structured outputs on Invoke instead of silently inlining the schema
2026-09-01 12:19:15 -07:00
Mateo Wang
435433fa07
Merge pull request #39149 from BerriAI/litellm_qwencloud_provider_aliases
feat(dashscope): add QwenCloud and Qwen AI Platform provider aliases
2026-09-01 12:18:05 -07:00
yuneng-jiang
33004d2f0c
test(e2e/ui): cover the Budgets page create, edit and delete flows (#39052)
* test(e2e/ui): cover the Budgets page create, edit and delete flows

The Budgets page had no browser coverage at all, so an admin creating or
editing a spend cap through the UI was only exercised by hand at RC time.

Each test reads the budget back from /budget/list, a different route from
the one the table renders, so a row that only exists in the table's cache
does not pass. The edit test pins the rate limits an unrelated spend-cap
edit has no business touching.

* test(e2e/ui): trim comments that restate the test steps

Review flagged the explanatory comments as restating ordinary setup rather
than explaining anything. Keeps the two that carry the regression rationale
for an assertion and drops the rest.

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-09-01 12:17:02 -07:00
Mateo Wang
f3dbd253be
Merge pull request #38106 from Timik232/bugfix/streaming-stable-response-id
fix(streaming): keep response id stable across streamed chunks
2026-09-01 12:11:32 -07:00
mateo-berri
0042493bca Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_bedrock_invoke_native_structured_output 2026-09-01 11:50:05 -07:00
yuneng-jiang
2b1bd20834
Merge pull request #31125 from BerriAI/litellm_/stoic-jones-7de871
feat(proxy): default to the v2 migration resolver, keep v1 as an opt-out
2026-09-01 11:46:13 -07:00
Mateo Wang
caeb5d181e
Merge pull request #39074 from BerriAI/litellm_fix_websearch_tool_selection_test
test(websearch): register configured search tool in pre-request hook test
2026-09-01 11:45:29 -07:00
mateo-berri
24393be4a6 Merge branch 'litellm_internal_staging' into bugfix/streaming-stable-response-id 2026-09-01 11:45:27 -07:00
mateo-berri
f3792fb700 feat(dashscope): add qwencloud and qwen_ai_platform provider aliases 2026-09-01 11:20:36 -07:00
Yuneng Jiang
201f60d19c
revert: restore search tool fallback when no router is configured
This reverts commit 65a46a5f32 (#38113)

That change made two edits that combine into a hard failure for SDK
users. It dropped the null-router guard in
_select_search_tool_from_router, so a missing router now yields an empty
search_tools list instead of returning early, and it turned the no-match
case in _select_search_tool_from_list from a debug-logged fallback into
a raised ValueError. It also added a call site in
async_pre_call_deployment_hook that invokes the selection purely for the
side effect of raising, discarding the return value

Used together, any SDK caller that sets search_tool_name and sends a web
search tool now raises "Configured search tool '<name>' was not found"
on every request. There is no way to satisfy the check off the proxy,
because search_tools is only ever populated from the proxy router, so
the SDK path cannot register one

tests/pass_through_unit_tests/test_websearch_interception_e2e.py caught
this, but #38113 only updated the handler unit tests, so the break
landed on staging

Reverting restores the previous behavior while we work out a fix that
keeps the stricter validation on the proxy path, where a silently
substituted search provider is the real problem worth rejecting, without
turning the SDK path into an unconditional error
2026-09-01 11:16:23 -07:00
Mateo Wang
ef7a984fbb
Merge pull request #38990 from BerriAI/devin_ai_1788201394-veo31-pricing-tiers
fix(registry): veo 3.1 pricing tiers + roll up open registry PRs (glm-5.2, Qwen3.8-Flash, gemma-4-31b, scribe_v2, fireworks/databricks deepseek v4) + deprecation dates
2026-09-01 10:32:59 -07:00
Mateo Wang
25b11301b2
Merge pull request #38891 from BerriAI/litellm_deflake_20260830
test: deflake MCP registry state, savings cost map, and MCP identity env reload tests
2026-09-01 10:15:29 -07:00
mateo-berri
06d4521fc0 fix(registry): add vertex veo 3.1 resolution tier pricing per vertex pricing page 2026-09-01 10:15:18 -07:00
Mateo Wang
6661e915a5
Merge pull request #38774 from BerriAI/litellm_fix_openai_embedding_encoding_format_omit
fix(embeddings): omit encoding_format when the client omits it on OpenAI-compatible calls
2026-09-01 10:13:38 -07:00
Mateo Wang
cfd9d27be2
Merge pull request #39038 from BerriAI/litellm_fix_pipeline_guardrail_list_clobber
fix(policy_engine): restore request guardrails list after pipeline allow
2026-09-01 10:13:30 -07:00
Mateo Wang
b86791b4b9
Merge pull request #38985 from BerriAI/litellm_fix_ocr_annotation_page_cost
fix(cost): bill OCR annotation pages via annotation_cost_per_page
2026-09-01 10:13:01 -07:00
Mateo Wang
30bf592aaf
Merge pull request #35417 from BerriAI/litellm_fix_responses_bridge_tool_call_arguments_json
fix(responses): json-encode object tool call arguments in the chat completions bridge
2026-09-01 10:08:12 -07:00
ryan-crabbe-berri
190b8c7d8e test(e2e): drop coverage registry cell for the alias-grant test 2026-09-01 09:03:47 -07:00
ryan-crabbe-berri
4118db5a50 test: drop litellm-internal patches from the cross-region alias test (TQ008)
The pure helpers express the same regression: the save-side drop must
leave the alias in place, and two registries must expand it to their
own ids. The full validate path is already covered by the
persists-verbatim test and the live e2e test.
2026-09-01 08:23:26 -07:00
ryan-crabbe-berri
7a761ccf5a test(e2e): cover alias MCP grant persisting verbatim on key generate
A single-instance run cannot reproduce the two-region setup, but the
regression is fully visible in one: the alias must survive to /key/info
unrewritten, and the alias-granted key must still list the server's
tools. The broken write path stored the resolved server id instead.
2026-09-01 08:10:41 -07:00
ryan-crabbe-berri
0d7035989c fix(mcp): persist alias MCP grants verbatim instead of rewriting to local server ids
Since PR #29128, key create/update/regenerate resolved every
object_permission.mcp_servers entry against the saving instance's
DB + config registry and persisted the resolved server ids. For
config-loaded servers the id is derived from a hash of the regional
URL, so in a shared-database multi-region deployment the rewrite baked
one region's ids into the row and every other region denied the key.
Grants written before v1.88.0 kept the raw alias and kept working,
which is why only newly provisioned keys broke.

Keep the validation and the stale-entry drop (the LIT-3278 fix), but
persist the caller's original identifiers for everything that resolves.
Read-time expand_permission_list already maps a name to each region's
local server id.
2026-09-01 08:10:36 -07:00
Devin AI
9a1aebc146 fix(registry): declare databricks deepseek cache-write rate at the input rate per repo convention
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-01 13:50:21 +00:00
Devin AI
7bfa0d7fb4 Registry audit: Fireworks DeepSeek V4 Flash 0731 pricing, Databricks DeepSeek V4 entries, provider deprecation dates
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-01 13:15:22 +00:00
Devin AI
5f44bdd1c1 test: trim mcp fixture docstring and reload comment
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-01 11:06:08 +00:00
Devin AI
95c7ca8801 fix(tests): restore module attributes after reload in mcp identity env tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-01 10:59:08 +00:00
Devin AI
2019c309ea merge: absorb litellm_internal_staging; drop python duplicate-issues closer fix superseded by ts rewrite
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-01 09:18:31 +00:00
Mateo Wang
ec3f8183c3
Merge pull request #39066 from BerriAI/litellm_aws_external_id_files_batches
fix(bedrock): forward aws_external_id in files and batches credential loading
2026-08-31 23:31:29 -07:00
Mateo Wang
9f6024bf97
Merge pull request #39044 from BerriAI/litellm_fix_bedrock_guardrail_token_log
fix(bedrock): mask signed request headers in guardrail debug log
2026-08-31 23:30:14 -07:00
Yassin Kortam
b11f0bcb92
fix(proxy): include litellm_model_table in GET /v2/team/list (#39045)
* fix(proxy): include litellm_model_table in GET /v2/team/list

GET /v2/team/list built its find_many queries without joining the
LiteLLM_ModelTable relation, so litellm_model_table (and the
model_aliases it carries) always read back as null there, same bug
class as GH #26312 which PR #33047 fixed on /team/info and /team/list
but never touched this endpoint.

* fix(test): assert observable output, not mock calls, in v2 team list test

The test-quality gate flagged the regression test for asserting on
find_many's call args instead of what the caller gets back. Rewritten
so the fake find_many only attaches litellm_model_table when its own
include kwarg asks for it, so the assertions are on the response.

* fix(proxy): drop invalid litellm_model_table include on deleted-team query

Greptile caught that LiteLLM_DeletedTeamTable has no litellm_model_table
relation in the Prisma schema, so passing that include on the deleted-team
find_many raised UnknownRelationalFieldError against a real database on
every GET /v2/team/list?status=deleted call. Confirmed live against
Postgres. Scope the fix to the active-team branch only, where the relation
exists; update the test to reflect that and assert the deleted branch no
longer requests it.
2026-08-31 23:19:45 -07:00
Tin Chi Lo
a27e12367e fix(bedrock): forward native structured outputs on Invoke instead of silently inlining the schema 2026-08-31 23:19:22 -07:00
yuneng-jiang
635a8ad308
Merge pull request #39073 from BerriAI/litellm_e2e_stored_credential_poll
test(e2e-ui): poll credential availability before Test Connect to deflake multi-instance runs
2026-08-31 23:07:34 -07:00
Mateo Wang
d83d9645fe
Merge pull request #39065 from BerriAI/litellm_fix_openai_alias_reasoning_effort
fix(openai): forward reasoning_effort for unknown model aliases instead of failing closed
2026-08-31 22:56:38 -07:00
Mateo Wang
b35aa7d2d9
Merge pull request #38796 from BerriAI/litellm_decrease_anys_opus5_r2
chore(typing): clear 1.1k basedpyright Any errors across 53 backend files
2026-08-31 22:56:35 -07:00
Yuneng Jiang
a48953a0a8
test(e2e-ui): require consecutive credential probe successes to cover multi-replica routing 2026-08-31 22:37:35 -07:00
mateo-berri
f65bee6d74 test(websearch): carry a reasoned test-quality suppression on the router patch 2026-08-31 22:32:05 -07:00
yuneng-jiang
c78f405473
Merge pull request #39051 from BerriAI/litellm_/non-admin-key-type-change-e76bec
fix(key_management): allow non-admin key_type preset transitions on /key/update
2026-08-31 22:24:29 -07:00
mateo-berri
191313e756 test(websearch): register configured search tool in pre-request hook test
PR #38113 made a configured search_tool_name fail fast when the router
does not carry a matching search tool, which broke
test_pre_request_hook_modifies_request_body: it names test-search-tool
but never registers it. Stub the proxy router with that tool so the test
exercises the conversion path again.
2026-08-31 22:22:05 -07:00
Yuneng Jiang
fae5aabc5c
Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_/jovial-archimedes-1d743b 2026-08-31 22:21:35 -07:00
Yuneng Jiang
88501a074d
test(e2e-ui): poll credential availability before Test Connect to deflake multi-instance runs 2026-08-31 22:20:36 -07:00
yuneng-jiang
174acf90c1
Merge pull request #39025 from BerriAI/litellm_/jovial-archimedes-1d743b
test(e2e/ui): automate 8 manual QA checklist flows
2026-08-31 22:13:07 -07:00
Yuneng Jiang
81c48f810e
fix(key_management): keep read-only keys read-only in non-admin preset transitions
A non-admin could widen a read-only (info_routes) key to llm_api or full
access through the preset carve-out. Read-only keys now stay read-only
unless a proxy admin widens them; the other preset transitions, including
the LIT-4891 llm_api to full access switch, still work. Also converts the
transition tests to assert on a returned outcome so the no-403 cases
carry real assertions.
2026-08-31 22:09:21 -07:00
yuneng-jiang
aabfbd6e37
Merge pull request #39055 from BerriAI/litellm_e2e_drop_unsafe_mgmt_registry_cells
test(e2e): drop the two mgmt registry cells no shared-proxy test can cover
2026-08-31 21:48:27 -07:00