Commit graph

6051 commits

Author SHA1 Message Date
yuneng-jiang
c4ab1d98e9
Merge pull request #41779 from BerriAI/litellm_settings_store_precedence
refactor(proxy): make the config file win over the database
2026-09-18 09:52:09 -07:00
ryan-crabbe-berri
4f70b88a1f
Merge pull request #41707 from BerriAI/litellm_jwt_mapping_cache_evict_on_bulk_key_delete
fix(proxy): evict jwt key mapping cache on user, team, org, and bulk key deletion
2026-09-18 08:41:31 -07:00
Yuneng Jiang
2f3e4fa724
test(proxy): pass the new pass-through argument to the side-effect dispatcher
_apply_general_settings_side_effects grew a fourth argument when the reload
started comparing the resolved pass-through list, and this dispatch test calls
it positionally, so it failed with a TypeError.
2026-09-18 02:17:52 -07:00
Yuneng Jiang
460f336d8a
fix(proxy): rebuild pass-through routes whenever the resolved list changes
The reload only re-registered pass-through endpoints when the stored row
still carried the key, so deleting the row left the deleted routes serving
traffic until the process restarted.

It now compares the resolved list before and after the row is applied and
rebuilds on any difference, including a deletion that resolves back to the
config file's list or to nothing.

This matches what _apply_retention_settings already does with the retention
values, so the two reload effects no longer disagree about what counts as a
change.

The tests assert the proxy's registry of live pass-through routes, which is
what decides whether a request is routed upstream or falls through to the
auth error, rather than that the registration helper was called.
2026-09-18 02:05:09 -07:00
Yuneng Jiang
23ee8ad1c3
fix(proxy): close the last three surfaces that bypassed config ownership
The two dashboard toggles under litellm_settings wrote through save_config,
so the refusal applied, but they mutated the litellm module global first: a
refused write still took effect in the running process until the next reload.
Both now check before they mutate.

/config/field/delete drops the stored key without touching the store, so a
deleted key kept reading back from the process. It now refreshes the store
like the other write paths.

/config/list reported source and editable for the general_settings rows but
not for the litellm_settings ones, so the dashboard would have shown a
config-declared toggle as editable.
2026-09-18 01:53:54 -07:00
Yuneng Jiang
8e67a33fc3
fix(proxy): refuse config-owned writes at one choke point and refresh the store
Both write paths now go through the same refusal, so /config/field/update and
/config/update answer identically instead of each phrasing its own rule.

A successful write now applies to the SettingsStore, so the next read sees it.
Without this, /config/field/info reported a key the dashboard had just stored
as "not set" until the process reloaded from the database.

resolve() no longer takes a KeyRule it never reads; the store picks the row.
The matrix tests resolve through SettingsStore instead of calling resolve
directly, so the section and key in each case actually route a lookup.

ConfigFieldInfo and ConfigList type `source` as the FieldSource literal, and
the dashboard API types are regenerated for the two new fields.
2026-09-18 01:49:30 -07:00
Yuneng Jiang
afa4a6fe78
refactor(proxy): make the config file win over the database
The precedence used to vary per key: some keys let a stored row win, some
let the file win, some merged the two. That meant an operator could not
answer "which value is live?" without knowing the key.

Now file presence decides ownership. A key the config file declares is
config-owned, whatever the database holds, and a key the file omits falls
back to the stored row. KeyRule no longer carries a RuleKind, only which
row the stored value lives in.

Writes to a config-owned key are refused at the two surfaces that reach
the database instead of being stored and silently ignored: save_config
and /config/field/update both 400 naming the key and the config file path.

Both read endpoints now report source and editable off the same
SettingsStore, so /config/field/info and /config/list can no longer
disagree inside one process.

Replaces the 786-case checked-in JSON fixture with cases generated from
the rule table, so the matrix tests no longer assert that resolve() agrees
with a snapshot of resolve().

BREAKING CHANGE: a dashboard or /config/field/update write to a setting
the config file declares now returns 400 instead of being stored. Remove
the key from the config file to let the database own it.
2026-09-18 01:28:21 -07:00
Yuneng Jiang
a7d4f7c521
Merge remote-tracking branch 'origin/main' into litellm_settings_store_precedence 2026-09-18 00:51:38 -07:00
yucheng-berri
8fc9c46d1a
Merge pull request #41541 from BerriAI/litellm_prompt_injection_async_llm_check
fix(proxy): run prompt injection heuristics off the event loop
2026-09-18 00:40:25 -07:00
Yuneng Jiang
0524745510
fix(proxy): preserve config pass-through and retention reloads 2026-09-18 00:23:24 -07:00
Yuneng Jiang
d1cd869012
refactor(proxy): resolve config and DB settings precedence in one SettingsStore 2026-09-17 23:36:27 -07:00
Yuneng Jiang
1be40e304d
refactor(proxy): add settings precedence primitives 2026-09-17 23:35:38 -07:00
yuneng-jiang
fc49ad7042
Merge pull request #41748 from BerriAI/litellm_wt_0917_7e12
fix(proxy): persist only the keys a caller changed in save_config
2026-09-17 22:57:06 -07:00
kerry
7975987107 test: keep behavior tests that read the cost map for a later fixture rewrite
Fifty six of the deleted tests turn out to assert the output of litellm code rather than the catalog lookup itself, things like map_openai_params, get_supported_openai_params, should_fake_stream, transform_request bodies, cost_per_token arithmetic, get_llm_provider routing, and provider config dispatch. They only happen to read shipped entries as inputs, so they belong in the later rewrite that injects a local model_cost, not in this deletion

Each one is restored verbatim from origin/main along with the fixtures, helpers, constants and imports it needs, and tests/test_litellm/test_sambanova_model_metadata.py is restored wholesale

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 04:52:06 +00:00
kerry
d2ac51893b test: keep the pinning-test removal free of unrelated reformatting
Regenerated every touched file from origin/main applying only the B1 test deletions and the unused import and helper cleanup they leave behind, without running the formatter across untouched code. CI only checks ruff format under litellm/, so the earlier reflows of test files were pure diff noise for reviewers

Also drops the tests/local_testing/test_prompt_caching.py entry from the caching-local shard in test-unit.yml since that file is deleted

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 04:27:28 +00:00
Yuneng Jiang
77a5e2cb64
test(proxy): isolate environment variable encryption 2026-09-17 21:07:02 -07:00
kerry
8ecbf3dbc1 test: drop tests that pin provider-owned cost map values
The repo rule is that a test must only fail when litellm code changes, never when a vendor updates a price, renames a field, or drops a model. These tests asserted shipped catalog entries directly, comparing lookup results to literals copied from model_prices_and_context_window.json or requiring named entries to exist or be absent, so every cost map sync could break them without any litellm code changing

Tests that exercise real litellm behavior with an injected local model_cost, invariants like backup parity, and assertions on non-lookup code paths are untouched

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 03:55:51 +00:00
Yuneng Jiang
463ece762a
fix(proxy): preserve opted-in environment variable saves 2026-09-17 20:53:21 -07:00
Yuneng Jiang
af17691389
fix(proxy): persist only the keys a caller changed in save_config 2026-09-17 20:32:04 -07:00
Yuneng Jiang
349e8b9358
test(proxy): forward each method through the typesafe route to a mocked upstream
The route test only resolved route names. It now sends every method through
the proxy with a virtual key and asserts the upstream receives that method,
the proxy's TypeSafe key and the caller's body
2026-09-17 18:19:42 -07:00
Yuneng Jiang
e8c1fe884e
Merge remote-tracking branch 'origin/main' into litellm_/circleci-specific-sha-0cf414 2026-09-17 18:05:39 -07:00
Yuneng Jiang
acc375a2a9
fix(proxy): forward every method on the typesafe pass-through route
#41607 registered the typesafe pass-through with a route that only accepted
GET and POST, so a PUT, DELETE or PATCH to /typesafe/... came back 405
before reaching the upstream. CircleCI's pass-through method test caught it,
but that lane does not run on the PR gate, so the mapped unit test now
covers the same invariant for typesafe

The same CircleCI run also failed test_models_by_provider because typesafe
is not a key of models_by_provider. Registering it there would satisfy the
assertion without changing behaviour: typesafe has no LlmProviders member,
so a typesafe/* deployment never loads and get_valid_models returns nothing,
and its spend is priced straight from model_cost. The test already skips
search-mode providers for that reason, so it now skips evaluation mode too
2026-09-17 18:04:58 -07:00
Mateo Wang
3424390101
Merge pull request #41448 from BerriAI/litellm_fix_passthrough_empty_query_params_drop_url_query
fix(passthrough): keep target URL query when client sends no query params
2026-09-17 18:02:48 -07:00
kerry-berri
f51f01fb54
Merge pull request #41443 from BerriAI/litellm_remove_brittle_price_pinning_tests
test: delete unit-test assertions that pin cost-map prices, limits and deprecation dates
2026-09-17 17:52:42 -07:00
ryan-crabbe-berri
85fe646776
Merge pull request #41694 from BerriAI/litellm_rename_model_sync_allowlists
fix(proxy): propagate db model renames to key, team, org, project and user model allowlists
2026-09-17 17:42:06 -07:00
kerry
8504c51f6c Revert "test: keep prompt cache prediction logic tests and drop only their price pins"
This reverts commit 810df257d8.
2026-09-18 00:40:38 +00:00
kerry
810df257d8 test: keep prompt cache prediction logic tests and drop only their price pins
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 00:39:34 +00:00
ryan-crabbe-berri
03ba5634a2
Merge pull request #40632 from BerriAI/litellm_fix_ui_disable_cache_control_injection_points
fix(ui): persist disabling cache control injection points on model update
2026-09-17 17:35:07 -07:00
Mateo Wang
cf42b607c3
Merge pull request #41615 from BerriAI/litellm_jev_complexity_classifier
feat(router): add TypeSafe Jev as a complexity router classifier
2026-09-17 17:31:08 -07:00
kerry
84d4d17bf8 Merge remote-tracking branch 'origin/main' into litellm_remove_brittle_price_pinning_tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

# Conflicts:
#	tests/test_litellm/llms/bedrock/chat/test_converse_transformation.py
2026-09-18 00:29:52 +00:00
kerry
c4620170ca test: delete assertions that pin vendor cost map facts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 00:28:49 +00:00
yucheng-berri
a9bea4f64d
Merge pull request #41571 from BerriAI/litellm_policy_attachment_priority
feat(policy_engine): explicit priority for policy attachment execution order
2026-09-17 17:27:55 -07:00
ryan
696587c4ab fix(ui): persist disabling cache control injection points on model update
Turning Cache Control off on the model edit screen omitted the field from the
PATCH body, which the backend reads as leave unchanged, so the stored
cache_control_injection_points list survived the save. The dashboard now sends
an explicit null when a stored list is being disabled, and update_db_model
clears that field on null the same way it already clears the mirrored pricing
fields. Omitted keys keep the stored value

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 00:20:22 +00:00
ryan
761e0775b9 test(proxy): share the jwt key mapping test doubles across the deletion endpoint tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 00:17:45 +00:00
yucheng-berri
e99902c4dd
Merge pull request #41569 from BerriAI/litellm_azure_ptu_spillover_cost
fix(cost): price Azure PTU spillover requests at standard token rates
2026-09-17 17:09:40 -07:00
ryan
e79d03e604 fix(proxy): evict jwt key mapping cache on bulk user and team member deletion
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 00:00:17 +00:00
ryan
1d50d1ad3b fix(proxy): rewrite every model allowlist in one statement on rename
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-17 23:40:59 +00:00
mateo-berri
42f2978061 Merge remote-tracking branch 'origin/main' into HEAD 2026-09-17 16:38:54 -07:00
mateo-berri
1e7e5b695f fix(router): reject a blank Jev api_key so it cannot pair with a caller-chosen api_base 2026-09-17 16:38:54 -07:00
Mateo Wang
deb9d8aedd
Merge pull request #41607 from BerriAI/litellm_typesafe_passthrough
Some checks are pending
LiteLLM Rust / rust-test (push) Waiting to run
Unit Tests: Documentation Validation / documentation (push) Waiting to run
Unit Tests: Proxy DB Operations / assert-shard-coverage (push) Waiting to run
Unit Tests: Proxy DB Operations / endpoints-and-responses (push) Blocked by required conditions
Unit Tests / caching-local (push) Waiting to run
Unit Tests / core-utils (push) Waiting to run
Unit Tests / enterprise-package (push) Waiting to run
Unit Tests / enterprise-routing (push) Waiting to run
Unit Tests / integrations (push) Waiting to run
Unit Tests: Proxy DB Operations / auth-checks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / budgets (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / custom-logging (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / db-and-spend (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / guardrails-hooks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / jwt-and-keys (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / key-generation (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / logging-misc (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-runtime (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-server-core (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-utils (push) Blocked by required conditions
Unit Tests / Vertex AI (push) Waiting to run
Unit Tests / All Other Providers (push) Waiting to run
Unit Tests / misc (push) Waiting to run
Unit Tests / proxy-auth (push) Waiting to run
Unit Tests / proxy-endpoints (push) Waiting to run
Unit Tests / proxy-extras (push) Waiting to run
Unit Tests / proxy-infra (push) Waiting to run
Unit Tests / proxy-server (push) Waiting to run
Unit Tests / responses-caching-types (push) Waiting to run
GitHub Actions Security Analysis / zizmor (push) Waiting to run
feat(proxy): add TypeSafe AI Jev evaluate passthrough with registry-priced spend tracking
2026-09-17 16:37:33 -07:00
ryan
884a467a7f fix(proxy): evict jwt_key_mapping cache when user, team, or org deletion removes mapped keys
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-17 23:33:59 +00:00
ryan-crabbe-berri
e5aa10a1ea
Merge pull request #41686 from BerriAI/litellm_member_budget_clone_reset_and_audit
fix(team): keep a forked member budget's reset window and audit bulk member budget writes
2026-09-17 16:32:31 -07:00
ryan
de0047c802 fix(proxy): skip allowlist rewrite when the model name is unchanged
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-17 23:24:26 +00:00
ryan
d3f5cde530 fix(proxy): propagate db model renames to key, team, org, project and user model allowlists
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-17 23:11:52 +00:00
Mateo Wang
57d41bda29
Merge pull request #41684 from BerriAI/litellm_wildcard_license_auto_router
fix(license): let a wildcard allowed_features license grant the auto_router feature
2026-09-17 16:11:40 -07:00
mateo-berri
7f581f6bc7 fix(router): keep the TypeSafe key off caller-chosen Jev endpoints 2026-09-17 16:09:31 -07:00
Mateo Wang
0da001901b
Merge pull request #41672 from BerriAI/litellm_autoroute_start_stop
feat(cli): rename lite autoroute up/down to start/stop, keeping the old names as deprecated aliases
2026-09-17 16:07:00 -07:00
ryan-crabbe-berri
29a959b3e8
Merge pull request #41632 from BerriAI/litellm_bulk_team_member_budget_update
feat(management_v1): bulk update team member budgets
2026-09-17 15:58:32 -07:00
mateo-berri
d3963e5d63 test(cli): pin the up alias port forwarding and the removed-settings stop message 2026-09-17 15:55:50 -07:00
Mateo Wang
289c52bcd6
Merge pull request #39512 from BerriAI/litellm_fix_image_edits_bracketed_alias
fix(images): stop forwarding the raw image[] and mask[] form keys
2026-09-17 15:48:32 -07:00