Commit graph

47404 commits

Author SHA1 Message Date
ryan-crabbe-berri
c14cf9d173 fix(access_groups): reconcile update deltas against the derived team set
Team membership deltas on PUT now start from the teams that really carry
the group, so a team the mirror column missed can be detached. Read
endpoints go through a typed TeamRepository instead of the untyped db
handle, and the where clause always carries both OR arms.

Claude-Session: https://claude.ai/code/session_01QvQzYztinxj8ZuD5YxbVdL
2026-09-01 16:03:22 -07:00
ryan-crabbe-berri
0c2d4c5773 Refuse a source path carrying a colon
`path:line` cannot represent a path that itself contains a colon, and the
one way pytest produces one is a Windows absolute location: separator
normalization turns `C:\app\e2e\a2a\test_x.py` into `C:/app/...`, which
slipped past the leading-slash check and composed the nonsense repo path
`tests/e2e/C:/app/e2e/a2a/test_x.py`.

Reject the colon itself rather than special-casing a drive letter: it is
the character the format reserves, so no path containing one was ever
linkable.

Claude-Session: https://claude.ai/code/session_017dTKXwJkzhtVLzDhePHsKG
2026-09-01 16:02:54 -07:00
ryan-crabbe-berri
00e40c0afe Record each e2e test's source location in the JUnit report
The JUnit report is the only thing that leaves the e2e run, and it says
where a test's results came from but never where its code lives. A reader
looking at `test_cell_claimed_only_by_a_skipped_test_is_uncovered` on the
status page has a name and nothing else -- no file, no line, no way to
reach the source short of grepping the repo by hand.

Pytest knows the location; the report format loses it. The `xunit1` family
wrote `file=` and `line=` onto every `<testcase>`, and the `xunit2` default
this suite runs on drops both. Switching families back would change the
document for every consumer of the same XML -- the Buildkite Test Engine
upload and the Loki pipeline included -- so add the location the way this
suite already adds `package` and `covers`: as a `<property>`, which is
purely additive.

`source` is repo-relative and one-based (`tests/e2e/a2a/test_x.py:41`), so
a consumer can build a link without knowing how pytest was started. That
takes normalizing the two launch shapes -- the runner image runs from its
own copy at /app/e2e, a developer runs from the repo root -- which is the
same normalization `package_from_nodeid` was already doing in reverse, now
factored into `suite_parts` so the two cannot drift apart. Paths that
escape the suite, and tests pytest reports no line for, emit an empty
string: a test with no link beats a link that 404s.

Claude-Session: https://claude.ai/code/session_017dTKXwJkzhtVLzDhePHsKG
2026-09-01 16:02:54 -07:00
Yujong Lee
0cc0c47f8b style(vector-store): satisfy import lint 2026-09-01 16:01:22 -07:00
ryan-crabbe-berri
346efa0c33
Merge pull request #39197 from BerriAI/litellm_e2e_reliability_retry_context_window
test(e2e): cover retry-on-timeout and the context-window fallback
2026-09-01 15:51:01 -07:00
ryan-crabbe-berri
4acc1d15fb fix(ui): map a cleared Team dropdown back to an empty string in the auto-router form 2026-09-01 15:50:19 -07:00
ryan-crabbe-berri
c493fc855c fix(ui): show MCP servers and agents inherited from access groups on the team overview
The team Overview and Settings tabs fed only object_permission into the Object
Permissions card, so a team whose access group grants MCP servers or agents read
"MCP Servers 0" and "Agents 0" while the Models card next to it already listed the
inherited models. /team/info has returned access_group_mcp_server_ids and
access_group_agent_ids for a while, nothing in the dashboard read them.

ObjectPermissionsView now accepts the inherited ids and MCPServerPermissions /
AgentPermissions merge them into their lists with an Inherited tag, deduped
against direct grants, so an admin can tell a group grant from a direct one.

Claude-Session: https://claude.ai/code/session_01QvQzYztinxj8ZuD5YxbVdL
2026-09-01 15:46:14 -07:00
ryan-crabbe-berri
6a469c2159 fix(access_groups): derive attached teams from the team table and reject unknown team ids
GET /v1/access_group and GET /v1/access_group/{id} (and the /v1/unified_access_group aliases) used to
return the assigned_team_ids column verbatim. That column is a denormalized mirror of
LiteLLM_TeamTable.access_group_ids and can be stale or hold ids of teams that no longer exist, so the
Attached Teams view drifted from reality.

The read path now runs one team find_many per request, unioning teams whose access_group_ids carry any
group in the response with teams listed in the stored columns. Only real team rows come back, so ghost
ids drop out and teams the mirror missed are added. The stored order is kept for ids that survive and
newly discovered teams are appended.

Create and update now resolve the requested assigned_team_ids inside the transaction and answer 400
with the missing ids before anything is written, instead of silently storing ids that point nowhere.

Refs LIT-6593

Claude-Session: https://claude.ai/code/session_01QvQzYztinxj8ZuD5YxbVdL
2026-09-01 15:46:09 -07:00
Yujong Lee
5799a32cdd fix(vector-store): route pre-call searches through router 2026-09-01 15:37:50 -07:00
Yujong Lee
6805d01709 fix(vector-store): preserve aliases with embedding config 2026-09-01 15:33:18 -07:00
Yujong Lee
5635811726 fix(vector-store): route embeddings through router 2026-09-01 15:33:18 -07:00
Yujong Lee
bfa5eac76b fix(vector-store): resolve embedding aliases for search 2026-09-01 15:33:18 -07:00
ryan-crabbe-berri
9c417ba08b
Merge pull request #38969 from emerzon/litellm_strict_order_fallback
fix(router): keep order fallback on the requested order level
2026-09-01 15:21:59 -07:00
ryan-crabbe-berri
55d638412b fix: stop a cleared Team field from blocking personal key creation
Clearing the Team combobox in the Create Key modal left team_id set to an
empty string, so /key/generate treated the request as team key generation
and failed with a team-not-found error for non-admin members.

TeamDropdown now emits null on clear, and GenerateKeyRequest normalizes an
empty team_id to None so the request runs the personal key path.
2026-09-01 15:17:00 -07:00
ryan-crabbe-berri
45fa78470d test(router): inject the upstream client instead of mutating litellm.aclient_session
The text-completion wire test set litellm.aclient_session, which the
test-quality gate (TQ005) flags as a process-wide global write. Pass an
AsyncOpenAI client through the router's client kwarg instead, so the test
owns its transport and needs no cache flush or global restore.

Claude-Session: https://claude.ai/code/session_01XKkTFa6g7Rmd6vtHL91GMn
2026-09-01 15:14:56 -07:00
devin-ai-integration[bot]
97dbd8efcb
fix(docker): add public Wolfi apk repo to runtime image (#39033)
* fix(docker): add public Wolfi apk repo to runtime image

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(docker): accept quote variants in Wolfi repo assertion

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-01 15:11:15 -07:00
devin-ai-integration[bot]
846900320e
feat(alerting): slack alerts for per-user daily/monthly spend thresholds and spend anomaly detection (#38438)
* feat(alerting): slack alerts for per-user daily/monthly spend thresholds and spend anomaly detection

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(alerting): use specific ValidationError matches in config rejection test

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): tolerate mocked slack alerting args when scheduling user spend scan

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(alerting): reject non-finite values in user spend alert settings

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-01 15:09:03 -07:00
devin-ai-integration[bot]
5988d93fed
fix(logging): guarantee max_parallel_requests slot release when streaming logging fails (#39093)
* fix(logging): guarantee max_parallel_requests slot release when stream logging fails

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(logging): cover guardrail branch of streaming logging hook failure isolation

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-01 15:08:11 -07:00
devin-ai-integration[bot]
f9c6eda909
fix(cli): quote the Claude Code apiKeyHelper for cmd.exe on Windows (#39174)
* fix(cli): quote the Claude Code apiKeyHelper for cmd.exe on Windows

lite up and lite login --config-claude wrote the helper command with
POSIX shlex quoting, so a backslashed Windows install path came out
wrapped in single quotes that cmd.exe and PowerShell take literally.
Quote every token with the cmd.exe rules already used for agent shims
when running on Windows, and keep the POSIX output unchanged elsewhere.

Resolves LIT-6627

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(cli): split the Windows apiKeyHelper with cmd.exe and C runtime rules

The invocation test pulled tokens back out with a regex, which cannot see
the doubled quotes or the percent guard quote_for_cmd emits. Model the two
parsers that read the helper on Windows instead and check argv round
trips for backslashed, spaced, metacharacter, percent and quoted tokens

---------

Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-01 15:08:00 -07:00
ryan-crabbe-berri
0b89c59be2 fix(router): consume _target_order at deployment selection so it never reaches a provider
Reading _target_order with .get left it in the request kwargs after selection, and only
nine provider boundaries stripped it. _atext_completion and _aadapter_completion spread
the raw kwargs, so an order-2 hop on /completions sent _target_order upstream, which
real providers reject as an unknown argument. Popping at selection strips it for every
path in one place; the PR's retry-keeping test already passed with pop because each retry
hands the callee its own kwargs copy.

Claude-Session: https://claude.ai/code/session_01XKkTFa6g7Rmd6vtHL91GMn
2026-09-01 15:02:01 -07:00
Mateo Wang
bad55da9bf
Merge pull request #38872 from BerriAI/litellm_fix_viewer_add_model_tab
fix(ui): hide model write affordances from view-only admin sessions
2026-09-01 14:52:54 -07:00
ryan-crabbe-berri
af11db9fe5 test(e2e): cover retry-on-timeout and the context-window fallback
Two P0 rows in the reliability coverage registry had no test.

reliability.retry.timeout.succeeds_within_retries gets a new file. The model
group is a pair: an always-timing-out deployment holding all of the group's
shuffle weight, and a healthy backup at weight 0. The weighted pick always opens
on the timing-out one, its first Timeout benches it via an allowed_fails_policy
of TimeoutErrorAllowedFails 0, and the retry falls through to the only
deployment left, so the outcome is a completion plus a reported retry with no
random first pick in the middle.

reliability.fallback.context_window.routes_to_fallback joins the existing
fallbacks spec. It registers a genuinely small-context OpenAI deployment, sends
a prompt past its limit so the provider refuses it on length, and reroutes with
context_window_fallbacks, which is the setting that handles that refusal rather
than plain fallbacks.

Both drive real provider calls through router_settings_override, so no config
change and no second proxy is needed. Reliability & Performance goes 16/36 to
18/36.

Claude-Session: https://claude.ai/code/session_01QvQzYztinxj8ZuD5YxbVdL
2026-09-01 14:51:15 -07:00
mateo
4da12795fc fix: filter deployment default API key limits
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-01 21:50:45 +00:00
mateo-berri
a38dfecd96 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_fix_stream_modify_response_chunks 2026-09-01 14:49:06 -07:00
mateo-berri
d59fcda8af fix(rerank): adopt declared authenticating providers in arerank instead of resolving them
get_llm_provider runs the OAuth device flow for github_copilot and chatgpt,
so calling it on the event loop before the executor dispatch let an
authenticated caller block the loop for the length of the polling window.
Adopt the declared provider via declared_authenticating_provider, matching
the metadata callers in utils.py, and only resolve for everything else.
2026-09-01 14:47:59 -07:00
ryan-crabbe-berri
4914914801 feat(guardrails): roll up Bedrock guardrail cost per usage counter
The daily guardrail usage rollup stored billable units per counter but no
cost, so the usage endpoints could only report units. The Bedrock hook now
stamps guardrail_cost_by_unit next to guardrail_usage, the spend-log
aggregator sums it into a new nullable cost column on
LiteLLM_DailyGuardrailUsageUnits, and /guardrails/usage/overview and
/guardrails/usage/detail/{id} return cost, totalCost and cost_by_unit /
cost_by_team / cost_by_key alongside the existing unit breakdowns.

Cost is nullable on purpose. Rows written before this migration, and rows
whose hook had no pricing entry, read as null rather than $0, and a single
unpriced increment keeps that row's cost unknown instead of partial.
guardrail_cost and the spend/budget path are untouched.

Claude-Session: https://claude.ai/code/session_01EX13mWex6RaBo9PYnkAtFW
2026-09-01 14:45:12 -07:00
Mateo Wang
b52b5d9421
Merge pull request #38734 from BerriAI/litellm_fix_bedrock_buffered_responses_stream
fix(bedrock): route streamed responses-API output through the unified guardrail
2026-09-01 14:44:41 -07:00
Mateo Wang
d3a399f6a7
Merge pull request #38870 from BerriAI/litellm_fix_azure_chat_anyof_tool_schema
fix(azure): flatten top-level tool schema combinators on Azure chat completions
2026-09-01 14:43:04 -07:00
Mateo Wang
981c5924a3
Merge pull request #39000 from BerriAI/litellm_fix_supported_openai_params_router_alias
fix(proxy): resolve router model aliases in /utils/supported_openai_params
2026-09-01 14:40:04 -07:00
Yujong Lee
386946353a fix(vertex): avoid duplicate DeepSeek OCR model namespace 2026-09-01 14:39:15 -07:00
ryan-crabbe-berri
ac964918c5 fix(router): strip _target_order at every provider boundary via a shared helper 2026-09-01 14:34:48 -07:00
yucheng-berri
5767a2da0f
fix(mcp): follow tools/list pagination from upstream servers (#39172)
* fix(mcp): follow tools/list pagination from upstream servers

Adopts BerriAI/litellm#32244 by Jupiter363 onto litellm_internal_staging
with merge conflicts resolved

* fix(mcp): degrade buggy pagination to partial results and bound the preview walk

A repeated nextCursor now returns the tools collected so far instead of
discarding every page with a RuntimeError, an empty-string cursor is treated
as terminal, load_mcp_tools shares the same pagination walk instead of
returning only the first page, and the tools/list preview is bounded by the
listing timeout instead of only the per-request timeout times the page cap

* fix(mcp): annotate deliberate rebind for the preview timeout scope

* fix(mcp): bound the shared pagination walk with an overall listing deadline

The per-request session read timeout restarts on every page, so direct SDK
callers of list_tools and load_mcp_tools could run up to the page cap with
no overall bound. The walk now returns the tools collected so far when
max(MCP_CLIENT_TIMEOUT, MCP_TOOL_LISTING_TIMEOUT) expires

* fix(mcp): let a per-server timeout extend the pagination deadline

MCPClient carries a per-server timeout that can exceed the global default;
list_tools now passes max(self.timeout, MCP_TOOL_LISTING_TIMEOUT) into the
shared walk so a deliberately slow server is not silently truncated at the
global deadline

* fix(mcp): honor per-server timeouts in the preview deadline and test the walk sessionless

The preview deadline now extends with the created client's own timeout, and
the pagination walk's cap, repeated-cursor, and empty-cursor cases are tested
directly against the helper instead of through patched SDK internals

* fix(mcp): forward the preview request's per-server timeout to the temporary server model

The tools preview built its temporary MCPServer without the request's
timeout field, so the client factory always fell back to the global
default and a per-server timeout could never extend the preview's
listing deadline (or its per-request timeout).
2026-09-01 14:34:14 -07:00
ryan-crabbe-berri
c7212e7fe2 refactor(router): drop _target_order via pop to satisfy the mutable-collection budget 2026-09-01 14:16:57 -07:00
devin-ai-integration[bot]
558f42e304
fix(proxy): default max_idle_connection_lifetime to 60s on DB URLs (#39134)
* fix(proxy): default max_idle_connection_lifetime to 60s on DB URLs

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): regenerate schema.d.ts for database_max_idle_connection_lifetime

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): keep URL-pinned max_idle_connection_lifetime over config value

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-01 14:16:51 -07:00
Mateo Wang
deb67ce6e2 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_fix_bedrock_buffered_responses_stream
# Conflicts:
#	tests/test_litellm/proxy/guardrails/guardrail_hooks/test_bedrock_guardrails.py
2026-09-01 14:11:29 -07:00
mateo-berri
b0aa1506fc Merge branch 'litellm_internal_staging' into litellm_fix_supported_openai_params_router_alias 2026-09-01 14:11:17 -07:00
ryan-crabbe-berri
2de33555ea style: run ruff format on fallback_event_handlers 2026-09-01 14:08:40 -07:00
mateo-berri
b311c24253 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_fix_azure_chat_anyof_tool_schema 2026-09-01 14:08:34 -07:00
mateo-berri
28d0ac5339 fix(router): guard the declared-provider check for requests without a model 2026-09-01 14:06:14 -07:00
mateo-berri
5192b2162c fix(proxy-extras): schema-qualify the _prisma_migrations logs lookup for non-public Prisma schemas 2026-09-01 14:04:12 -07:00
mateo-berri
0f35328a21 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_fix_azure_chat_anyof_tool_schema
# Conflicts:
#	tests/test_litellm/litellm_core_utils/prompt_templates/test_litellm_core_utils_prompt_templates_common_utils.py
2026-09-01 13:59:59 -07:00
mateo-berri
647f32c407 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_bump_tornado_658 2026-09-01 13:59:41 -07:00
yuneng-jiang
8bc862f52c
Merge pull request #39129 from BerriAI/litellm_/litellm-issue-39078-c025be
fix(ui): render the logs Tools panel with theme tokens
2026-09-01 13:58:03 -07:00
mateo-berri
848a3edbe8 merge: sync litellm_internal_staging into litellm_rerank_provider_error_body 2026-09-01 13:55:04 -07:00
mateo-berri
1eea8e2831 fix(deps): raise the tornado floor to 6.5.8 for GHSA-8423-8fgw-73vq and GHSA-wwv5-g3v4-889x 2026-09-01 13:54:34 -07:00
Mateo Wang
2c5f429ad4
Merge pull request #39185 from BerriAI/litellm_fix_embedding_encoding_format_suite_break
test: exempt MockTransport request-shape embedding tests from VCR replay
2026-09-01 13:52:51 -07:00
mateo-berri
018d8ae9eb Merge remote-tracking branch 'origin/litellm_internal_staging' into HEAD 2026-09-01 13:49:51 -07:00
Yassin Kortam
aab9abdd1d
fix: keep litellm_credential_name from LiteLLM Params JSON and gate stored credential attach to proxy admins (#39047)
* fix(ui): keep litellm_credential_name from LiteLLM Params JSON when no credential is selected

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(ui): drop null litellm_credential_name from AddModelPanel payload fixture

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(ui): validate JSON litellm_credential_name against accessible credentials

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): enforce proxy-admin-only credential attachment on model create/update

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): raise ProxyException for unauthorized credential attach and gate /model/update

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(proxy): fold credential-change detection into can_user_attach_credential to satisfy complexity budget

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): decrypt stored credential name before unchanged-credential comparison

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(proxy): cover credential attach rejection on add_new_model and patch_model

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(proxy): annotate proxy-global patches with test-quality suppressions

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-01 13:46:18 -07:00
mateo-berri
692f3b513c fix(proxy-extras): recover the v2 migration resolver from concurrent migrate deploy deadlocks
Two instances racing prisma migrate deploy on one database deadlock on
CREATE INDEX CONCURRENTLY: the victim gets P3018 with 40P01 and the
survivor then sees the failed ledger row as P3009. Both were treated as
unrecoverable, so neither instance came up.

Roll the deadlocked migration's ledger row back and retry the deploy on
P3018, consult the failed row's logs in _prisma_migrations to do the
same on P3009, and retry a deadlock reported without a Prisma error
code. Genuinely broken migrations still fail fast.
2026-09-01 13:43:01 -07:00
Mateo Wang
8ce02c0666
Merge pull request #39184 from BerriAI/litellm_fable_5_1_structured_output 2026-09-01 13:41:22 -07:00