Commit graph

6616 commits

Author SHA1 Message Date
Yuneng Jiang
a45be4f276
fix(proxy): let the config file win when reporting nested alerting sources
_nested_setting_source returned "db" whenever the stored row held a value,
without first asking whether the config file declares the same key. For a
config-owned alerting_args field that disagrees with the database, the
endpoint reported source "db" while the proxy actually serves the file's
value and rejects any write to it.

Config ownership is now checked first, matching SettingsStore.source and
the precedence the rest of the resolver applies. The source test set grows
a field that only the database sets, a field only the file sets, and a
stored empty list, so each reported source is discriminating.
2026-09-21 13:48:37 -07:00
Yuneng Jiang
ddf6565970
Merge remote-tracking branch 'origin/main' into litellm_config_read_source 2026-09-21 13:25:45 -07:00
Yuneng Jiang
be2f0d081b
fix(proxy): report sources only on the read endpoints main does not cover
/config/field/info and /config/list already report per-key source on main,
so this drops the branch's versions of those and keeps /alerting/settings,
/get/ui_settings and /router/settings.

Read endpoints no longer write the freshly read database row back into the
shared settings store; the reload path already keeps it current, and a GET
that mutates global state leaks across callers.

Regenerates the lazy OpenAPI snapshot on Python 3.12, matching CI, and the
dashboard API types for the two new response fields.
2026-09-21 13:25:39 -07:00
Mateo Wang
662e5b6e32
Merge pull request #42284 from BerriAI/litellm_qianwen_ai_platform_rename
fix: rename the mainland China brand to Qianwen AI Platform
2026-09-21 13:18:20 -07:00
joshua-berri
3549143bcd
Merge pull request #34919 from BerriAI/litellm_fix_mcp_peek_utf8_boundary
fix(mcp): handle split UTF-8 routing previews
2026-09-21 19:56:50 +00:00
yucheng-berri
884407dad4
Merge pull request #42119 from BerriAI/litellm_default_policy_attachments
* feat(policy_engine): add default fallback policy attachments

A policy attachment with default: true applies only when no non-default
attachment matches the request, so an opt-in guardrail policy replaces the
fallback one instead of running alongside it. Supported in config.yaml,
/policies/attachments, the Admin UI Attachments tab and the resolver
(matched_via is prefixed with default:).

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(policy_engine): ignore inapplicable non-default attachments when selecting defaults

A non-default attachment whose policy is missing or whose condition does not match the request
no longer suppresses default attachments. The impact preview marks default counts as an upper bound

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(policy_engine): accept any sequence of policy names in condition matching

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(policy_engine): resolve policies once and apply fallback semantics in get_matching_policies

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 12:55:42 -07:00
mateo-berri
bd8df0ed8f test: keep the Qwen brand missing-key tests off the network and split the ChatGPT fields test back out 2026-09-21 12:54:23 -07:00
Mateo Wang
2e35ae1065
Merge pull request #42049 from BerriAI/litellm_mantle_native_anthropic_messages
feat(bedrock_mantle): serve /v1/messages for Claude models on Mantle's native Anthropic Messages API
2026-09-21 12:52:16 -07:00
tin-berri
457b01e96d
Merge pull request #42055 from BerriAI/litellm_prompt_caching_request_table
feat(ui): show prompt caching requests and net savings
2026-09-21 12:45:11 -07:00
Yuneng Jiang
86c4021e83
Merge branch 'main' into litellm_config_read_source
The shared settings-store work this branch was stacked on landed on main,
along with source reporting for /config/field/info and /config/list.
Resolved by taking main for the store, the config routes and their tests,
and re-applying only the read surfaces main does not cover:
/alerting/settings, /get/ui_settings and /router/settings.
2026-09-21 12:36:15 -07:00
joshua-berri
211ff96943
Merge pull request #39189 from BerriAI/litellm_mcp_list_pagination_lit5594
fix(mcp): paginate prompt and resource discovery
2026-09-21 19:35:17 +00:00
Joshua Valluru
a835e75620 fix(mcp): handle split UTF-8 routing previews in place 2026-09-21 12:24:19 -07:00
Joshua Valluru
bf5dff8986 chore: sync MCP UTF-8 fix with main 2026-09-21 12:15:57 -07:00
mateo-berri
cf00ab1bf8 fix: rename the mainland China brand to Qianwen AI Platform 2026-09-21 12:13:43 -07:00
Yassin Kortam
5673f67727
Merge pull request #42022 from BerriAI/litellm_redis_durable_spend_log_buffer
fix(proxy): park requeued spend logs in Redis so they survive a pod restart during a DB outage
2026-09-21 14:01:34 -05:00
mateo-berri
4eed951e6f Merge commit '36b8be7d81b' into litellm_mantle_native_anthropic_messages_b4dc
# Conflicts:
#	tests/test_litellm/llms/anthropic/experimental_pass_through/messages/test_anthropic_experimental_pass_through_messages_handler.py
2026-09-21 12:00:37 -07:00
moe-berri
a83773cfa5
Merge pull request #41886 from BerriAI/litellm_jev_autorouter_launch_1789767495
feat(auto-router): add JEV classifier alongside LLM classifier
2026-09-21 11:55:44 -07:00
Joshua Valluru
1499d84f5a fix(mcp): paginate optional discovery lists 2026-09-21 11:37:55 -07:00
ryan-crabbe-berri
cc1a3157d3
Merge pull request #42121 from BerriAI/litellm_utils_model_info_lookup
feat(proxy): add GET /utils/model_info to look up cost map info for unregistered models
2026-09-21 10:42:28 -07:00
yucheng
e647255909 fix(policy_engine): resolve policies once and apply fallback semantics in get_matching_policies
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 17:36:03 +00:00
ryan-crabbe-berri
5573265013
Merge pull request #41906 from BerriAI/litellm_team_member_budget_source_reset
feat(team): show whether a member follows the team default budget and allow resetting to it
2026-09-21 10:34:08 -07:00
Moe Khalil
bb46e8b774 chore(auto-router): merge main and preserve JEV configuration
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 17:16:18 +00:00
yucheng-berri
51ae7ab1fd
Merge pull request #41588 from BerriAI/litellm_ui_session_personal_key_budget_ceiling
* fix(key_generate): use user's budget for UI session personal keys

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* Update litellm/proxy/management_endpoints/key_management_endpoints.py

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* fix(user_update): evict cached user on max_budget change so the personal key ceiling refreshes on every worker

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: jesus-berri <jesus@berri.ai>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: yucheng <yucheng@berri.ai>
2026-09-21 10:11:29 -07:00
yuneng-jiang
da76ba83ad
Merge pull request #42105 from BerriAI/litellm_flip_v2_migration_resolver_default
feat(proxy): default to the v2 migration resolver
2026-09-21 10:05:59 -07:00
Yassin Kortam
def37c6532
Merge pull request #42097 from BerriAI/litellm_budget_exceeded_422
fix(proxy): return 422 instead of 429 for BudgetExceededError
2026-09-21 08:35:30 -05:00
Devin AI
53fe97975f Merge remote-tracking branch 'origin/main' into litellm_ui_session_personal_key_budget_ceiling
Some checks failed
LiteLLM Rust / rust-lint (push) Has been cancelled
LiteLLM Rust / rust-test (push) Has been cancelled
LiteLLM Rust / rust-wheel (push) Has been cancelled
Terraform Modules / fmt, validate, test (aws) (push) Has been cancelled
Terraform Modules / fmt, validate, test (gcp) (push) Has been cancelled
Terraform Provider / gofmt, vet, build, test (push) Has been cancelled
Terraform Provider / Provider endpoints vs proxy OpenAPI schema (push) Has been cancelled
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

# Conflicts:
#	litellm/proxy/management_endpoints/internal_user_endpoints.py
2026-09-20 19:40:05 +00:00
joshua-berri
c25601fb5f
Merge pull request #42064 from BerriAI/litellm_fix_jwt_deactivated_users_8217
fix(auth): reject deactivated JWT users and refresh cached status
2026-09-20 19:35:53 +00:00
Moe Khalil
93dddbf047 chore(auto-router): merge main budget fix into JEV classifier
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-20 17:16:48 +00:00
moe-berri
1e161f516c
Merge pull request #41879 from BerriAI/litellm_jev_test_budget_1789764884
fix(proxy): enforce virtual key budgets for JEV test routing
2026-09-20 10:10:21 -07:00
Moe Khalil
8505d10017 chore(auto-router): merge main into JEV launch branch
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-20 14:53:05 +00:00
ryan
59f98d5363 feat(proxy): add GET /utils/model_info to look up cost map info for unregistered models 2026-09-20 10:30:26 +00:00
yucheng
ef55eb6bdf fix(policy_engine): ignore inapplicable non-default attachments when selecting defaults
A non-default attachment whose policy is missing or whose condition does not match the request
no longer suppresses default attachments. The impact preview marks default counts as an upper bound

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-20 09:07:08 +00:00
yassin
9e53ee5f04 Merge remote-tracking branch 'origin/main' into litellm_redis_durable_spend_log_buffer 2026-09-20 08:52:32 +00:00
yucheng
7fc114c24f feat(policy_engine): add default fallback policy attachments
A policy attachment with default: true applies only when no non-default
attachment matches the request, so an opt-in guardrail policy replaces the
fallback one instead of running alongside it. Supported in config.yaml,
/policies/attachments, the Admin UI Attachments tab and the resolver
(matched_via is prefixed with default:).

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-20 08:39:03 +00:00
Yuneng Jiang
129c4a703b
fix(proxy): only warn about the deprecated flag when it came from the CLI
USE_V2_MIGRATION_RESOLVER=true is a supported way to select v2, but click
sets the same parameter from that env var, so the deprecation notice fired
for environment-based config that is not deprecated. The notice now keys off
click's parameter source.

Also drops an em dash from the notice, and moves the resolver decision under
mock-free tests by making it take the env value as an argument.
2026-09-20 01:22:31 -07:00
Yuneng Jiang
9aec964bac
Merge remote-tracking branch 'origin/main' into litellm_flip_v2_migration_resolver_default
Drops the TQ008 suppressions the new test carried; main removed that rule.
2026-09-20 01:02:19 -07:00
Yuneng Jiang
fac518dbf7
feat(proxy): default to the v2 migration resolver
The migrations Job entrypoint (migrations/run.py) has defaulted to v2 with
USE_V2_MIGRATION_RESOLVER=false as the opt-out, and the Helm chart documents
that knob. Proxy startup still defaulted to v1, so the two paths disagreed
about which resolver a deployment runs.

Proxy startup now resolves the same way: v2 unless USE_V2_MIGRATION_RESOLVER
is false or --use_legacy_migration_resolver is passed.

- --use_v2_migration_resolver stays accepted as a no-op that warns, so
  existing commands and Helm values do not fail on an unknown option.
- The dedicated Postgres smoke-test job is repointed at the legacy resolver
  so v1 keeps real-DB proxy-boot coverage, and the two jobs that deselected
  it by name are updated to match the rename.

#39178 reverted an earlier flip because two replicas sharing a database
deadlocked (40P01 / P3018) with neither answering /health/liveliness. That
contention is what #40932 coordinates, which is why this builds on it.
2026-09-20 00:12:30 -07:00
yassin
92d3a1d87d test(proxy): expect 422 for per-model budget rejections on cursor route
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-20 06:10:16 +00:00
yassin
f9244749e0 fix(proxy): return 422 instead of 429 for BudgetExceededError
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-20 06:00:06 +00:00
Mateo Wang
58065d46fd
Merge pull request #42071 from BerriAI/litellm_remove_dead_telemetry_flag 2026-09-19 21:48:02 -07:00
Mateo Wang
4011367b39
Merge pull request #40986 from BerriAI/litellm_lit_7346_multi_choice_stream_guardrails
fix(guardrails): scan each choice's tool-call arguments apart on n>1 streams and log why a rewrite was discarded
2026-09-19 20:34:05 -07:00
Mateo Wang
9d7f77988a
Merge pull request #41974 from BerriAI/litellm_fix_startup_view_creation_race
fix(proxy): wait for the spend-log table before creating startup views
2026-09-19 20:33:49 -07:00
mateo
b2e123da43 fix(proxy): drop legacy telemetry key from persisted WORKER_CONFIG before initialize
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-20 03:14:11 +00:00
mateo-berri
c13dcb0abf fix(proxy): forward a client's anthropic-beta and anthropic-version headers to bedrock_mantle 2026-09-19 20:11:14 -07:00
Mateo Wang
ef7da9b49f
Merge pull request #42072 from BerriAI/litellm_mcp_cold_worker_tools_call
fix(mcp): tools/call no longer 404s on a worker that has not served tools/list
2026-09-19 20:04:33 -07:00
mateo-berri
327447bc10 Merge remote-tracking branch 'origin/main' into litellm_fix_startup_view_creation_race
# Conflicts:
#	tests/test_litellm/proxy/test_proxy_server.py
2026-09-19 20:02:33 -07:00
Joshua Valluru
124196cbaa fix(auth): preserve scope-admin email policy during status lookup 2026-09-19 19:48:53 -07:00
mateo
fdd91a347a test: drop narration comment from telemetry flag test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-20 02:44:52 +00:00
mateo
8e530cf819 fix: keep --telemetry as a hidden no-op so existing start commands still parse
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-20 02:36:59 +00:00
Joshua Valluru
6ea74d70f1 fix(auth): enforce SCIM status for admin JWTs and refresh SCIM caches 2026-09-19 19:33:47 -07:00