Commit graph

6566 commits

Author SHA1 Message Date
yuneng-jiang
da76ba83ad
Merge pull request #42105 from BerriAI/litellm_flip_v2_migration_resolver_default
feat(proxy): default to the v2 migration resolver
2026-09-21 10:05:59 -07:00
Yassin Kortam
def37c6532
Merge pull request #42097 from BerriAI/litellm_budget_exceeded_422
fix(proxy): return 422 instead of 429 for BudgetExceededError
2026-09-21 08:35:30 -05:00
joshua-berri
c25601fb5f
Merge pull request #42064 from BerriAI/litellm_fix_jwt_deactivated_users_8217
fix(auth): reject deactivated JWT users and refresh cached status
2026-09-20 19:35:53 +00:00
moe-berri
1e161f516c
Merge pull request #41879 from BerriAI/litellm_jev_test_budget_1789764884
fix(proxy): enforce virtual key budgets for JEV test routing
2026-09-20 10:10:21 -07:00
Yuneng Jiang
129c4a703b
fix(proxy): only warn about the deprecated flag when it came from the CLI
USE_V2_MIGRATION_RESOLVER=true is a supported way to select v2, but click
sets the same parameter from that env var, so the deprecation notice fired
for environment-based config that is not deprecated. The notice now keys off
click's parameter source.

Also drops an em dash from the notice, and moves the resolver decision under
mock-free tests by making it take the env value as an argument.
2026-09-20 01:22:31 -07:00
Yuneng Jiang
9aec964bac
Merge remote-tracking branch 'origin/main' into litellm_flip_v2_migration_resolver_default
Drops the TQ008 suppressions the new test carried; main removed that rule.
2026-09-20 01:02:19 -07:00
Yuneng Jiang
fac518dbf7
feat(proxy): default to the v2 migration resolver
The migrations Job entrypoint (migrations/run.py) has defaulted to v2 with
USE_V2_MIGRATION_RESOLVER=false as the opt-out, and the Helm chart documents
that knob. Proxy startup still defaulted to v1, so the two paths disagreed
about which resolver a deployment runs.

Proxy startup now resolves the same way: v2 unless USE_V2_MIGRATION_RESOLVER
is false or --use_legacy_migration_resolver is passed.

- --use_v2_migration_resolver stays accepted as a no-op that warns, so
  existing commands and Helm values do not fail on an unknown option.
- The dedicated Postgres smoke-test job is repointed at the legacy resolver
  so v1 keeps real-DB proxy-boot coverage, and the two jobs that deselected
  it by name are updated to match the rename.

#39178 reverted an earlier flip because two replicas sharing a database
deadlocked (40P01 / P3018) with neither answering /health/liveliness. That
contention is what #40932 coordinates, which is why this builds on it.
2026-09-20 00:12:30 -07:00
yassin
92d3a1d87d test(proxy): expect 422 for per-model budget rejections on cursor route
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-20 06:10:16 +00:00
yassin
f9244749e0 fix(proxy): return 422 instead of 429 for BudgetExceededError
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-20 06:00:06 +00:00
Mateo Wang
58065d46fd
Merge pull request #42071 from BerriAI/litellm_remove_dead_telemetry_flag 2026-09-19 21:48:02 -07:00
Mateo Wang
4011367b39
Merge pull request #40986 from BerriAI/litellm_lit_7346_multi_choice_stream_guardrails
fix(guardrails): scan each choice's tool-call arguments apart on n>1 streams and log why a rewrite was discarded
2026-09-19 20:34:05 -07:00
Mateo Wang
9d7f77988a
Merge pull request #41974 from BerriAI/litellm_fix_startup_view_creation_race
fix(proxy): wait for the spend-log table before creating startup views
2026-09-19 20:33:49 -07:00
mateo
b2e123da43 fix(proxy): drop legacy telemetry key from persisted WORKER_CONFIG before initialize
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-20 03:14:11 +00:00
Mateo Wang
ef7da9b49f
Merge pull request #42072 from BerriAI/litellm_mcp_cold_worker_tools_call
fix(mcp): tools/call no longer 404s on a worker that has not served tools/list
2026-09-19 20:04:33 -07:00
mateo-berri
327447bc10 Merge remote-tracking branch 'origin/main' into litellm_fix_startup_view_creation_race
# Conflicts:
#	tests/test_litellm/proxy/test_proxy_server.py
2026-09-19 20:02:33 -07:00
Joshua Valluru
124196cbaa fix(auth): preserve scope-admin email policy during status lookup 2026-09-19 19:48:53 -07:00
mateo
fdd91a347a test: drop narration comment from telemetry flag test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-20 02:44:52 +00:00
mateo
8e530cf819 fix: keep --telemetry as a hidden no-op so existing start commands still parse
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-20 02:36:59 +00:00
Joshua Valluru
6ea74d70f1 fix(auth): enforce SCIM status for admin JWTs and refresh SCIM caches 2026-09-19 19:33:47 -07:00
mateo-berri
a3e9ed34fe fix(mcp): gate the pre-call listing per tool, not per server
A tools/call on a cold worker listed the target server once and then never
again, so a later caller whose credentials expose a wider upstream catalog
got 404 for tools the first caller never had. Gate the pre-call listing on
whether this worker already exposes the requested tool, so callers with
different catalogs no longer mask each other. Removing the per-server guard
also drops the empty-listing case that re-listed on every call.
2026-09-19 19:29:33 -07:00
Mateo Wang
5417abd586
Merge pull request #42011 from BerriAI/litellm_scrub_default_master_key
docs: stop advertising sk-1234 as the master key in shipped configs and examples
2026-09-19 19:05:31 -07:00
Mateo Wang
b6dd3d932c
Merge pull request #42019 from BerriAI/litellm_master_key_boot_enforcement
feat(proxy)!: refuse to start with an unset, empty, or publicly known master key
2026-09-19 19:04:02 -07:00
ryan-crabbe-berri
ecf17513fb refactor(proxy): rename the local development override to dangerously_permit_weak_or_unset_master_key so the name says exactly what it permits 2026-09-19 18:53:14 -07:00
mateo-berri
92ff54f134 fix(mcp): list a never-listed server before its first tools/call
The startup tool-name fill skips servers whose upstream wants the caller's
own token (true_passthrough, OAuth discovery), and mcp 2 no longer runs the
list handler before an uncached tools/call, so every uvicorn worker that had
not served tools/list answered 404 "Tool not found" for prefixed tools/call
and the REST server_id route on those servers.

On a resolution miss, execute_mcp_tool now lists the prefix-matched (or
server_id-requested) server once, with the caller's credentials, through the
existing tools/list path, then resolves as before. Listing failures fall
through to the existing 404, a worker that already listed the server never
re-lists it, and a server outside the caller's allowed set is never listed.
2026-09-19 18:49:26 -07:00
mateo
f820472488 chore: remove the dead telemetry flag from the SDK, proxy CLI and configs
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-20 01:44:10 +00:00
Mateo Wang
93e39d5042
Merge pull request #42062 from BerriAI/litellm_pr38499_batch_retrieve_model_group
fix(router): stamp model_group when retrieving a batch, so batch tokens are attributable (internal copy of #38499)
2026-09-19 18:23:06 -07:00
Yassin Kortam
755f5c535d
Merge pull request #41994 from BerriAI/litellm_redis_spend_requeue_safety 2026-09-19 18:19:57 -07:00
ryan-crabbe-berri
99f99cfb46 fix(proxy): stop the boot when the requested master key migration fails, unless allow_requests_on_db_unavailable tolerates the outage 2026-09-19 18:10:52 -07:00
Joshua Valluru
a82f0a0bd2 fix(auth): reject deactivated JWT users and invalidate cached status 2026-09-19 18:10:49 -07:00
mateo-berri
2a35dc5217 fix(guardrails): keep the undeliverable rewrite reason through copies and name the responses mismatch 2026-09-19 18:00:03 -07:00
mateo-berri
a114af2a26 fix(proxy): resolve the view setup gate through the search_path and set the row count before the views 2026-09-19 17:54:20 -07:00
ryan-crabbe-berri
a6c51ba3de fix(proxy): never treat plaintext that base64-decodes to nothing as a ciphertext during the master key migration
A string such as "*" or "..." has no base64 characters, so it decoded to no bytes and read as an empty plaintext under any key. The migration would have counted it and overwritten it with a ciphertext of the empty string. Also read from the writer database instead of a read replica, report a database error during the migration instead of crashing the boot, skip columns the connected schema lacks across every schema on the search path, cap the JSON walk depth for the recursion detector, and move the boot wiring into one tested function.
2026-09-19 17:53:17 -07:00
mateo-berri
867a4df347 fix(proxy): claim a finished batch's per-model budget charge atomically
A finished batch reports its whole cost on every poll. The charge-once
marker is now taken with one atomic increment on the shared cache, so two
workers polling the same batch at once cannot both charge it, and the
marker's TTL is refreshed on every poll so a batch polled within every
budget window is never charged again after the marker's first expiry.
2026-09-19 17:41:19 -07:00
yucheng-berri
82ddab2405
Merge pull request #41840 from BerriAI/litellm_team_audit_lifecycle
fix(team): emit audit events for member_delete and role changes and carry the final roster on team create
2026-09-19 17:37:16 -07:00
mateo-berri
114fc16554 merge: origin/main into litellm_lit_7346_multi_choice_stream_guardrails 2026-09-19 17:34:43 -07:00
ryan-crabbe-berri
38d776bd2b feat(proxy): re-encrypt stored secrets at boot from LITELLM_MIGRATE_FROM_MASTER_KEY so an unsafe key can be replaced while the proxy refuses to start
Rotating through POST /key/regenerate needs a running proxy, which a refused boot does not have. The refusal now counts the stored values that decrypt under the unsafe key. When there are none it only asks for a new key. When there are some it also asks for LITELLM_MIGRATE_FROM_MASTER_KEY, and the next boot with a safe key re-encrypts them and logs that the variable can be deleted. Leaving the variable set afterwards is a no-op with one notice.
2026-09-19 17:32:58 -07:00
yucheng-berri
2ec5c2c7cd
Merge pull request #41740 from BerriAI/litellm_otel_v2_langfuse_llm_spans_only
feat(otel v2): opt-in llm_only span scope for Langfuse destinations and the operator Langfuse exporter
2026-09-19 17:24:50 -07:00
mateo-berri
1b8f704035 fix(proxy): await the cancelled view setup task quietly and assert it starts at boot
Use contextlib.suppress for the cancelled task in stop_view_setup_task, make the legacy prisma setup test inject a plain mock for the synchronous start_view_setup_task and assert it is called, and drop the docstrings the branch added to tests
2026-09-19 17:17:18 -07:00
yucheng
de70cf842a fix(team): run the role update and budget upsert in one transaction under the team lock
Some checks failed
LiteLLM Rust / rust-lint (push) Has been cancelled
LiteLLM Rust / rust-test (push) Has been cancelled
LiteLLM Rust / rust-wheel (push) Has been cancelled
Terraform Provider / gofmt, vet, build, test (push) Has been cancelled
Terraform Provider / Provider endpoints vs proxy OpenAPI schema (push) Has been cancelled
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-20 00:04:32 +00:00
yucheng
b7db48c7c1 fix(team): 404 a role update whose target left the roster before the locked read
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-19 23:57:38 +00:00
yuneng-jiang
3df159308d
Merge pull request #42042 from BerriAI/litellm_websearch_activation_status
feat(ui): report whether the serving proxy has applied web search interception
2026-09-19 16:42:05 -07:00
mateo-berri
741b10cdc9 chore: merge main into fix/batch-retrieve-model-group 2026-09-19 16:25:39 -07:00
tin-berri
252a0f1eac
Merge pull request #41617 from BerriAI/litellm_fuse_model_profile_presets
feat(router): add maintained Fuse model and harness presets
2026-09-19 16:17:19 -07:00
yucheng
431ddbdd22 test(team): exercise the member-add audit helper directly and drop its dead user_id None guard
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-19 23:17:18 +00:00
yucheng
f63782f678 fix(otel v2): reject a langfuse_span_scope that conflicts with another callback entry on the same team or key
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-19 23:16:54 +00:00
tin-berri
1944d40097
Merge pull request #41177 from BerriAI/litellm_autorouter_baseline_cache
fix(proxy): estimate auto-router baseline costs from durable cache history
2026-09-19 16:16:29 -07:00
Yuneng Jiang
dd6a3558d5
Merge remote-tracking branch 'origin/main' into litellm_websearch_activation_status 2026-09-19 16:16:25 -07:00
Yuneng Jiang
c0bc45224c
feat(ui): report whether the serving proxy has applied the interception setting
The stored flag is the cluster's desired state and is what the form saves, so it
cannot also stand as proof that this process activated the callback: a pod that
lagged or failed to apply it would still read as on. Report the process's own
registration as a separate read-only field and warn on the page when the two
disagree, so a failed activation is visible instead of only logged.
2026-09-19 16:16:17 -07:00
yuneng-jiang
5fc5afd595
Merge pull request #42007 from BerriAI/litellm_/web-search-autoship-scope-01a237
feat(ui): configure web search interception from the Admin UI
2026-09-19 15:54:03 -07:00
yucheng-berri
6afc807ae3
Merge pull request #42027 from BerriAI/litellm_stream_boundary_attr_forwarding
Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-19 15:47:54 -07:00