Commit graph

18927 commits

Author SHA1 Message Date
yassin
048aaad627 Merge remote-tracking branch 'origin/main' into litellm_mcp_admin_terminate_sessions_revoke_credentials 2026-09-18 18:39:16 +00:00
Yassin Kortam
0594dd7caf
Merge pull request #41539 from BerriAI/litellm_vault_login_secret_namespace
feat(vault): add separate login and secret namespaces for HashiCorp Vault
2026-09-18 11:38:49 -07:00
Yassin Kortam
84ae0805ba
Merge pull request #41620 from BerriAI/litellm_team_member_temp_budget_increase
feat(proxy): temporary budget increase for team members
2026-09-18 11:38:34 -07:00
ryan-crabbe-berri
294a9e15de ci: classify new issues into domain, provider, kind, priority and lift labels
Every issue opened from now on is gated on the template headings, sent once
through the LiteLLM proxy with a strict JSON schema, and labelled from the
manifest in .github/labels.json. Old-template issues are not touched. The bug
template shrinks to Description, Config, LiteLLM Version and Steps to Repro,
both templates gain a domain dropdown, and the labelers that keyed off the old
component dropdown go away.
2026-09-18 11:37:55 -07:00
Yassin Kortam
f6d9b2552f
Merge pull request #41692 from BerriAI/litellm_mcp_gateway_sessions_by_client_user
feat(mcp): show live gateway sessions by AI client and user
2026-09-18 11:37:16 -07:00
Yassin Kortam
ca79c393d5
Merge pull request #41636 from BerriAI/litellm_per_key_end_user_default_budget
feat(proxy): per-key default budget for dynamically created customers
2026-09-18 11:36:59 -07:00
Yassin Kortam
f129e7e2d0
Merge pull request #41515 from BerriAI/litellm_transcribe_passthrough
feat(proxy): add Amazon Transcribe pass-through with completion-time job pricing
2026-09-18 11:35:06 -07:00
yassin
3219a875d0 fix(proxy): serialize in-memory window rollover and reset siblings once per expired window
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 18:20:18 +00:00
Yassin Kortam
d42f448e41
Merge pull request #41585 from BerriAI/litellm_lazy_fastapi_bpe_imports
perf: defer fastapi and tiktoken BPE imports out of import litellm
2026-09-18 11:19:13 -07:00
Yassin Kortam
660f3dfdd3
Merge pull request #41555 from BerriAI/litellm_max_parallel_requests_queue_size
feat(router): reject with 429 when a deployment's max_parallel_requests slots are all in use
2026-09-18 11:18:08 -07:00
kerry
4b215e2a60 fix(schema): require a schedule and well-formed windows in off_peak_pricing
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 18:17:36 +00:00
mateo-berri
860b0203c0 test(bedrock): expect canonical session tags in the dynamic auth params propagation test 2026-09-18 11:17:34 -07:00
kerry
cbcb55af89 fix(schema): classify off_peak_pricing as a structured object in the model prices schema generator
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 18:11:17 +00:00
kerry
d68e20e586 Merge remote-tracking branch 'origin/litellm_e2e_cost_calculation_scripted_provider' into litellm_e2e_cost_calculation_scripted_provider
Some checks failed
LiteLLM Rust / rust-lint (push) Has been cancelled
LiteLLM Rust / rust-test (push) Has been cancelled
LiteLLM Rust / rust-wheel (push) Has been cancelled
2026-09-18 18:05:20 +00:00
Mateo Wang
db756b9393
Merge pull request #40730 from BerriAI/litellm_responses_nested_additional_drop_params
fix(responses): honor nested additional_drop_params paths
2026-09-18 11:04:38 -07:00
kerry
e87830feba Merge remote-tracking branch 'origin/main' into litellm_e2e_cost_calculation_scripted_provider 2026-09-18 17:56:49 +00:00
kerry-berri
fd58c31cc7
Merge pull request #41832 from BerriAI/litellm_lit_8111_cache_read_missing_rate
fix(cost): bill cache-read tokens at the input rate when the map has no cache-read rate
2026-09-18 10:56:25 -07:00
yassin
fa70e49b81 chore: merge main into litellm_transcribe_passthrough
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 17:54:59 +00:00
mateo-berri
b03957ba9c fix(proxy): unpin cost-map pricing copied into model_info and report pricing overrides
A model_info blob that carries key next to pricing fields is a copy of a /model/info response (only litellm.get_model_info emits key), so those pricing fields are dropped when the row is loaded from the DB and on every Reload Price Data, and the deployment follows the current cost map again. Prices typed into litellm_params, or into model_info without key, stay as they are.

/model/info, /v1/model/info and /v2/model/info now report model_info.pricing_overrides, the pricing fields the deployment sets itself, and the Admin UI model page says whether a price follows the cost map or overrides it.
2026-09-18 10:49:36 -07:00
mateo-berri
2a76f54a5a fix(bedrock): merge main and thread aws_session_tags through the auth struct
Merge origin/main (a9ee15372f) into the typed AwsAuthParams refactor so the
session tags PR #40446 added land in the struct: resolve_credentials
canonicalizes aws_session_tags before STS, the realtime path forwards them,
and Files upload/download plus bodiless S3 signing now assume the role with
the tags instead of dropping them.
2026-09-18 10:41:52 -07:00
mateo-berri
b523ed9a2d merge: origin/main into litellm_jwt_token_exchange_grant
Some checks failed
LiteLLM Rust / rust-lint (push) Has been cancelled
LiteLLM Rust / rust-test (push) Has been cancelled
LiteLLM Rust / rust-wheel (push) Has been cancelled
2026-09-18 10:35:53 -07:00
ryan
8dfda93123 test(proxy): drop explanatory docstrings from routing_groups regression tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 17:32:31 +00:00
mateo-berri
f6ee046199 fix(proxy): read the user row past the recent-miss memo on a database-only lookup
get_user_object skipped the database for db_cache_expiry seconds after a miss on the same worker even when the caller asked for check_db_only, so the token exchange mint could answer no_active_key for a user JWT auth had just created. A database-only read now always reaches the database.
2026-09-18 10:25:45 -07:00
Devin AI
dba9ff801f fix(proxy): reset sibling tpm/rpm counters when shared window rolls over
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 17:18:49 +00:00
ryan
c3bc55d18f chore: merge main into litellm_routing_groups_atomic_validation
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 16:59:23 +00:00
yuneng-jiang
c4ab1d98e9
Merge pull request #41779 from BerriAI/litellm_settings_store_precedence
refactor(proxy): make the config file win over the database
2026-09-18 09:52:09 -07:00
kerry
af61d9247c test(cost_tracking): expect cache reads without a map rate to estimate at the input rate
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 16:33:17 +00:00
kerry
88d1eb3b35 fix(cost): bill cache-read tokens at the input rate when the map has no cache-read rate
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 16:22:14 +00:00
ryan
817cdcef64 test(scim): drop redundant docstring from the valueless entitlements PUT test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 16:00:54 +00:00
Mateo Wang
861f79797f
Merge pull request #41663 from BerriAI/litellm_remove_legacy_interactions_schema_flag
refactor(interactions): remove expired use_legacy_interactions_schema shim
2026-09-18 08:48:30 -07:00
ryan
b6410d563b fix(scim): accept entitlements and roles entries without a value on SCIM user PUT
SCIMMultiValuedAttribute required value, so a PUT /scim/v2/Users/{id} that
carried an IdP-specific entitlements entry such as {"groups": [...]} failed
body validation with 422 and the suspend (active: false) never reached
update_user. value is now optional and unknown members are kept, so the
suspend is applied, keys are blocked, and the entries are stored as sent

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 15:46:33 +00:00
ryan-crabbe-berri
4f70b88a1f
Merge pull request #41707 from BerriAI/litellm_jwt_mapping_cache_evict_on_bulk_key_delete
fix(proxy): evict jwt key mapping cache on user, team, org, and bulk key deletion
2026-09-18 08:41:31 -07:00
kerry
5f3a86aee5 test(e2e): use TypeAlias over 3.12 type statements in e2e models
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 13:27:14 +00:00
kerry
dda7776346 test(e2e): make cost-calculation cases MECE by rate-key ownership with realistic fixtures
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 13:15:29 +00:00
yucheng
8fc74a78bc fix(proxy): reject blank trusted_proxy_ranges entries before they are dropped
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 11:50:30 +00:00
yucheng
cacd12b87d fix(proxy): treat a malformed trusted_proxy_ranges entry as an undeclared topology
A list with an entry that is not an address or CIDR range no longer switches
the per-source Admin UI sign-in limit on against the direct peer address, so a
typo cannot make a shared ingress address the bucket for every user behind it

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 11:28:35 +00:00
yucheng
cfdf4fa5dc fix(proxy): break ties between equivalent login limit overrides deterministically
Two spellings of one network share a prefix length, so the exemption wins the tie, then the higher limit, regardless of mapping order

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 10:20:39 +00:00
yucheng
65765d6550 test(proxy): import LoginThrottle under TYPE_CHECKING for the throttle helper annotation
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 09:57:35 +00:00
yucheng
0da2f5b96c feat(proxy): round the per-username sign-in allowance down and exempt an address with an override of 0
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 09:54:54 +00:00
Yuneng Jiang
0d9c5159df
test(e2e): assert config ownership instead of absence from the stored row
/config/field/info used to answer from the LiteLLM_Config row, so "the field
400s" proved the row did not carry it. It now answers from the resolved
settings, and the CI stack config declares general_settings.max_parallel_requests,
so the endpoint returns that value and the old assertion could never hold.

The check that /add/allowed_ip writes only what the caller changed moves to
/config/list, which still reports stored_in_db off the row, and the field/info
call now asserts the ownership the endpoint reports: the config file owns the
key, so it reads back as source=config and editable=false.

Verified against a live proxy on an isolated Postgres rather than in CI, where
this check has never run: it waits on protected-environment approval.
2026-09-18 02:53:37 -07:00
yucheng
b6bb212248 refactor(proxy): raise the sign-in block explicitly and type the empty settings mapping
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 09:40:05 +00:00
yassin
f631301cfa Merge branch 'litellm_usage_key_free_aggregate_split' into litellm_daily_global_spend_table 2026-09-18 09:29:47 +00:00
yucheng
fade26b969 feat(proxy): derive the per-username sign-in allowance from the address limit
The per-address-and-username allowance is now half the effective address allowance, rounded up, instead of a separate max_failed_login_attempts_per_user setting. A per-address override therefore raises or effectively removes both limits for that address, and no second override table is needed

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 09:25:34 +00:00
Yuneng Jiang
2f3e4fa724
test(proxy): pass the new pass-through argument to the side-effect dispatcher
_apply_general_settings_side_effects grew a fourth argument when the reload
started comparing the resolved pass-through list, and this dispatch test calls
it positionally, so it failed with a TypeError.
2026-09-18 02:17:52 -07:00
ryan
b78793b153 fix(auth): limit team service account route carve-out to /key/generate and /key/update
The key_management_routes group also contains /spend/logs, /team/daily/activity
and other routes whose handlers scope non-admin callers by user_id. A userless
service account key would have reached them unscoped, so the route check now
uses a dedicated two-route allowlist

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 09:16:57 +00:00
shivam
a052da6974 feat(keys): let team service account keys use key management endpoints for their own team
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 09:05:10 +00:00
Yuneng Jiang
460f336d8a
fix(proxy): rebuild pass-through routes whenever the resolved list changes
The reload only re-registered pass-through endpoints when the stored row
still carried the key, so deleting the row left the deleted routes serving
traffic until the process restarted.

It now compares the resolved list before and after the row is applied and
rebuilds on any difference, including a deletion that resolves back to the
config file's list or to nothing.

This matches what _apply_retention_settings already does with the retention
values, so the two reload effects no longer disagree about what counts as a
change.

The tests assert the proxy's registry of live pass-through routes, which is
what decides whether a request is routed upstream or falls through to the
auth error, rather than that the registration helper was called.
2026-09-18 02:05:09 -07:00
ryan
a9ad1bbad8 Merge remote-tracking branch 'origin/main' into litellm_routing_groups_atomic_validation 2026-09-18 09:04:26 +00:00
yassin
3d2913307a Merge remote-tracking branch 'origin/main' into litellm_usage_key_free_aggregate_split 2026-09-18 08:59:24 +00:00
yassin
61ce1b46d9 Merge remote-tracking branch 'origin/main' into litellm_max_parallel_requests_queue_size
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

# Conflicts:
#	tests/test_litellm/test_utils.py
2026-09-18 08:57:53 +00:00