Commit graph

6123 commits

Author SHA1 Message Date
ryan-crabbe-berri
6b54238083
Merge pull request #40807 from BerriAI/litellm_service_account_team_key_mgmt
feat(keys): let team service account keys use key management endpoints for their own team
2026-09-18 12:58:41 -07:00
Yassin Kortam
2edda5aec3
Merge pull request #41757 from BerriAI/litellm_typesafe_compaction_guardrail
feat(guardrails): add TypeSafe Jev relevance-based compaction guardrail
2026-09-18 12:56:52 -07:00
ryan-crabbe-berri
ed9e666b54
Merge pull request #41351 from BerriAI/litellm_routing_groups_atomic_validation
fix(router): validate routing_groups at save time and keep invalid DB groups from blocking SSO load
2026-09-18 11:59:21 -07:00
Mateo Wang
88c9dd1294
Merge pull request #41511 from BerriAI/litellm_foundry_a2a_entra_agents
feat(a2a): reach Microsoft Foundry agents with Entra auth and versioned card discovery
2026-09-18 11:44:14 -07:00
Yassin Kortam
1653d132c5
Merge pull request #41506 from BerriAI/litellm_vertex_gcs_file_content_streaming
feat(vertex_ai): stream GCS batch output files from /v1/files/{id}/content
2026-09-18 11:42:14 -07:00
Yassin Kortam
0594dd7caf
Merge pull request #41539 from BerriAI/litellm_vault_login_secret_namespace
feat(vault): add separate login and secret namespaces for HashiCorp Vault
2026-09-18 11:38:49 -07:00
Yassin Kortam
84ae0805ba
Merge pull request #41620 from BerriAI/litellm_team_member_temp_budget_increase
feat(proxy): temporary budget increase for team members
2026-09-18 11:38:34 -07:00
Yassin Kortam
f6d9b2552f
Merge pull request #41692 from BerriAI/litellm_mcp_gateway_sessions_by_client_user
feat(mcp): show live gateway sessions by AI client and user
2026-09-18 11:37:16 -07:00
Yassin Kortam
ca79c393d5
Merge pull request #41636 from BerriAI/litellm_per_key_end_user_default_budget
feat(proxy): per-key default budget for dynamically created customers
2026-09-18 11:36:59 -07:00
Yassin Kortam
f129e7e2d0
Merge pull request #41515 from BerriAI/litellm_transcribe_passthrough
feat(proxy): add Amazon Transcribe pass-through with completion-time job pricing
2026-09-18 11:35:06 -07:00
kerry-berri
fd58c31cc7
Merge pull request #41832 from BerriAI/litellm_lit_8111_cache_read_missing_rate
fix(cost): bill cache-read tokens at the input rate when the map has no cache-read rate
2026-09-18 10:56:25 -07:00
yassin
fa70e49b81 chore: merge main into litellm_transcribe_passthrough
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 17:54:59 +00:00
ryan
8dfda93123 test(proxy): drop explanatory docstrings from routing_groups regression tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 17:32:31 +00:00
ryan
c3bc55d18f chore: merge main into litellm_routing_groups_atomic_validation
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 16:59:23 +00:00
yuneng-jiang
c4ab1d98e9
Merge pull request #41779 from BerriAI/litellm_settings_store_precedence
refactor(proxy): make the config file win over the database
2026-09-18 09:52:09 -07:00
kerry
af61d9247c test(cost_tracking): expect cache reads without a map rate to estimate at the input rate
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 16:33:17 +00:00
ryan-crabbe-berri
4f70b88a1f
Merge pull request #41707 from BerriAI/litellm_jwt_mapping_cache_evict_on_bulk_key_delete
fix(proxy): evict jwt key mapping cache on user, team, org, and bulk key deletion
2026-09-18 08:41:31 -07:00
Yuneng Jiang
2f3e4fa724
test(proxy): pass the new pass-through argument to the side-effect dispatcher
_apply_general_settings_side_effects grew a fourth argument when the reload
started comparing the resolved pass-through list, and this dispatch test calls
it positionally, so it failed with a TypeError.
2026-09-18 02:17:52 -07:00
ryan
b78793b153 fix(auth): limit team service account route carve-out to /key/generate and /key/update
The key_management_routes group also contains /spend/logs, /team/daily/activity
and other routes whose handlers scope non-admin callers by user_id. A userless
service account key would have reached them unscoped, so the route check now
uses a dedicated two-route allowlist

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 09:16:57 +00:00
shivam
a052da6974 feat(keys): let team service account keys use key management endpoints for their own team
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 09:05:10 +00:00
Yuneng Jiang
460f336d8a
fix(proxy): rebuild pass-through routes whenever the resolved list changes
The reload only re-registered pass-through endpoints when the stored row
still carried the key, so deleting the row left the deleted routes serving
traffic until the process restarted.

It now compares the resolved list before and after the row is applied and
rebuilds on any difference, including a deletion that resolves back to the
config file's list or to nothing.

This matches what _apply_retention_settings already does with the retention
values, so the two reload effects no longer disagree about what counts as a
change.

The tests assert the proxy's registry of live pass-through routes, which is
what decides whether a request is routed upstream or falls through to the
auth error, rather than that the registration helper was called.
2026-09-18 02:05:09 -07:00
ryan
a9ad1bbad8 Merge remote-tracking branch 'origin/main' into litellm_routing_groups_atomic_validation 2026-09-18 09:04:26 +00:00
yassin
bc636bc5ff Merge remote-tracking branch 'origin/main' into litellm_vault_login_secret_namespace 2026-09-18 08:57:50 +00:00
Yuneng Jiang
23ee8ad1c3
fix(proxy): close the last three surfaces that bypassed config ownership
The two dashboard toggles under litellm_settings wrote through save_config,
so the refusal applied, but they mutated the litellm module global first: a
refused write still took effect in the running process until the next reload.
Both now check before they mutate.

/config/field/delete drops the stored key without touching the store, so a
deleted key kept reading back from the process. It now refreshes the store
like the other write paths.

/config/list reported source and editable for the general_settings rows but
not for the litellm_settings ones, so the dashboard would have shown a
config-declared toggle as editable.
2026-09-18 01:53:54 -07:00
Yuneng Jiang
8e67a33fc3
fix(proxy): refuse config-owned writes at one choke point and refresh the store
Both write paths now go through the same refusal, so /config/field/update and
/config/update answer identically instead of each phrasing its own rule.

A successful write now applies to the SettingsStore, so the next read sees it.
Without this, /config/field/info reported a key the dashboard had just stored
as "not set" until the process reloaded from the database.

resolve() no longer takes a KeyRule it never reads; the store picks the row.
The matrix tests resolve through SettingsStore instead of calling resolve
directly, so the section and key in each case actually route a lookup.

ConfigFieldInfo and ConfigList type `source` as the FieldSource literal, and
the dashboard API types are regenerated for the two new fields.
2026-09-18 01:49:30 -07:00
Yuneng Jiang
afa4a6fe78
refactor(proxy): make the config file win over the database
The precedence used to vary per key: some keys let a stored row win, some
let the file win, some merged the two. That meant an operator could not
answer "which value is live?" without knowing the key.

Now file presence decides ownership. A key the config file declares is
config-owned, whatever the database holds, and a key the file omits falls
back to the stored row. KeyRule no longer carries a RuleKind, only which
row the stored value lives in.

Writes to a config-owned key are refused at the two surfaces that reach
the database instead of being stored and silently ignored: save_config
and /config/field/update both 400 naming the key and the config file path.

Both read endpoints now report source and editable off the same
SettingsStore, so /config/field/info and /config/list can no longer
disagree inside one process.

Replaces the 786-case checked-in JSON fixture with cases generated from
the rule table, so the matrix tests no longer assert that resolve() agrees
with a snapshot of resolve().

BREAKING CHANGE: a dashboard or /config/field/update write to a setting
the config file declares now returns 400 instead of being stored. Remove
the key from the config file to let the database own it.
2026-09-18 01:28:21 -07:00
Yuneng Jiang
a7d4f7c521
Merge remote-tracking branch 'origin/main' into litellm_settings_store_precedence 2026-09-18 00:51:38 -07:00
yucheng-berri
8fc9c46d1a
Merge pull request #41541 from BerriAI/litellm_prompt_injection_async_llm_check
fix(proxy): run prompt injection heuristics off the event loop
2026-09-18 00:40:25 -07:00
Yuneng Jiang
0524745510
fix(proxy): preserve config pass-through and retention reloads 2026-09-18 00:23:24 -07:00
Yuneng Jiang
d1cd869012
refactor(proxy): resolve config and DB settings precedence in one SettingsStore 2026-09-17 23:36:27 -07:00
Yuneng Jiang
1be40e304d
refactor(proxy): add settings precedence primitives 2026-09-17 23:35:38 -07:00
yassin
0765f6d571 fix(guardrails): keep typesafe registries as dicts so guardrail discovery finds them
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 06:05:43 +00:00
yuneng-jiang
fc49ad7042
Merge pull request #41748 from BerriAI/litellm_wt_0917_7e12
fix(proxy): persist only the keys a caller changed in save_config
2026-09-17 22:57:06 -07:00
yassin
351afc8519 test(guardrails): cover typesafe failure paths and edge shapes
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 05:10:48 +00:00
yassin
91d4c579e4 refactor(guardrails): freeze or suppress mutable constructions in typesafe guardrail
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 05:06:48 +00:00
kerry
7975987107 test: keep behavior tests that read the cost map for a later fixture rewrite
Fifty six of the deleted tests turn out to assert the output of litellm code rather than the catalog lookup itself, things like map_openai_params, get_supported_openai_params, should_fake_stream, transform_request bodies, cost_per_token arithmetic, get_llm_provider routing, and provider config dispatch. They only happen to read shipped entries as inputs, so they belong in the later rewrite that injects a local model_cost, not in this deletion

Each one is restored verbatim from origin/main along with the fixtures, helpers, constants and imports it needs, and tests/test_litellm/test_sambanova_model_metadata.py is restored wholesale

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 04:52:06 +00:00
yassin
fd4476b130 fix(guardrails): bound typesafe tuning params, preserve result tail, log fail-open status
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 04:49:50 +00:00
yassin
dffb6a38d9 refactor(guardrails): tighten typesafe guardrail typing and error handling
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 04:35:58 +00:00
yassin
4863e1775b feat(guardrails): add TypeSafe Jev relevance-based compaction guardrail
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 04:33:16 +00:00
kerry
d2ac51893b test: keep the pinning-test removal free of unrelated reformatting
Regenerated every touched file from origin/main applying only the B1 test deletions and the unused import and helper cleanup they leave behind, without running the formatter across untouched code. CI only checks ruff format under litellm/, so the earlier reflows of test files were pure diff noise for reviewers

Also drops the tests/local_testing/test_prompt_caching.py entry from the caching-local shard in test-unit.yml since that file is deleted

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 04:27:28 +00:00
Yuneng Jiang
77a5e2cb64
test(proxy): isolate environment variable encryption 2026-09-17 21:07:02 -07:00
kerry
8ecbf3dbc1 test: drop tests that pin provider-owned cost map values
The repo rule is that a test must only fail when litellm code changes, never when a vendor updates a price, renames a field, or drops a model. These tests asserted shipped catalog entries directly, comparing lookup results to literals copied from model_prices_and_context_window.json or requiring named entries to exist or be absent, so every cost map sync could break them without any litellm code changing

Tests that exercise real litellm behavior with an injected local model_cost, invariants like backup parity, and assertions on non-lookup code paths are untouched

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 03:55:51 +00:00
Yuneng Jiang
463ece762a
fix(proxy): preserve opted-in environment variable saves 2026-09-17 20:53:21 -07:00
Yuneng Jiang
af17691389
fix(proxy): persist only the keys a caller changed in save_config 2026-09-17 20:32:04 -07:00
yassin
3f824793e5 fix(proxy): price a Transcribe job from the parsed start response instead of the relayed httpx body
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 02:48:09 +00:00
yassin
504b464417 test(proxy): assert the persisted member budget row as a whole instead of field absence
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 02:12:12 +00:00
yassin
507219e26f fix(proxy): resolve temporary budget grants against the live team default instead of a snapshot
A temporary-only member update no longer clones the team default budget into the private row. The row stores just the temp pair and auth, spend admission and reservation add the active increase to the current shared default, so a later lowering of the default reaches members with an active grant

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 02:09:09 +00:00
yassin
2decf3761a fix(proxy): price a deleted Transcribe job from the start response the proxy relayed
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 02:08:21 +00:00
yassin
3926e74e9d test(proxy): lock plain member updates to the merge-patch contract without a team default snapshot
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 01:28:49 +00:00
Yuneng Jiang
349e8b9358
test(proxy): forward each method through the typesafe route to a mocked upstream
The route test only resolved route names. It now sends every method through
the proxy with a virtual key and asserts the upstream receives that method,
the proxy's TypeSafe key and the caller's body
2026-09-17 18:19:42 -07:00