Commit graph

53300 commits

Author SHA1 Message Date
Yuneng Jiang
c508df64fe
test(e2e): accept common cat descriptions 2026-09-18 03:13:28 -07:00
yucheng
65765d6550 test(proxy): import LoginThrottle under TYPE_CHECKING for the throttle helper annotation
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 09:57:35 +00:00
yucheng
0da2f5b96c feat(proxy): round the per-username sign-in allowance down and exempt an address with an override of 0
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 09:54:54 +00:00
Yuneng Jiang
0d9c5159df
test(e2e): assert config ownership instead of absence from the stored row
/config/field/info used to answer from the LiteLLM_Config row, so "the field
400s" proved the row did not carry it. It now answers from the resolved
settings, and the CI stack config declares general_settings.max_parallel_requests,
so the endpoint returns that value and the old assertion could never hold.

The check that /add/allowed_ip writes only what the caller changed moves to
/config/list, which still reports stored_in_db off the row, and the field/info
call now asserts the ownership the endpoint reports: the config file owns the
key, so it reads back as source=config and editable=false.

Verified against a live proxy on an isolated Postgres rather than in CI, where
this check has never run: it waits on protected-environment approval.
2026-09-18 02:53:37 -07:00
Yuneng Jiang
3af44daf6d
test(e2e): cover chat and responses registry gaps 2026-09-18 02:43:22 -07:00
yucheng
b6bb212248 refactor(proxy): raise the sign-in block explicitly and type the empty settings mapping
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 09:40:05 +00:00
Yuneng Jiang
066cc1883a
test(router): cover legacy lowest TPM selection 2026-09-18 02:39:58 -07:00
ryan
b50265d75f refactor(keys): freeze the service account route tuple and stamp metadata without seeding mutable literals
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 09:34:44 +00:00
yassin
f631301cfa Merge branch 'litellm_usage_key_free_aggregate_split' into litellm_daily_global_spend_table 2026-09-18 09:29:47 +00:00
yassin
efd2b8a3cd refactor(vault): type the KV read body walk as Mapping[str, object]
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 09:28:20 +00:00
ryan
e68ea077f1 style(keys): keep rebind-ok reason within the line limit
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 09:26:29 +00:00
yucheng
fade26b969 feat(proxy): derive the per-username sign-in allowance from the address limit
The per-address-and-username allowance is now half the effective address allowance, rounded up, instead of a separate max_failed_login_attempts_per_user setting. A per-address override therefore raises or effectively removes both limits for that address, and no second override table is needed

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 09:25:34 +00:00
Yuneng Jiang
2f3e4fa724
test(proxy): pass the new pass-through argument to the side-effect dispatcher
_apply_general_settings_side_effects grew a fourth argument when the reload
started comparing the resolved pass-through list, and this dispatch test calls
it positionally, so it failed with a TypeError.
2026-09-18 02:17:52 -07:00
ryan
b78793b153 fix(auth): limit team service account route carve-out to /key/generate and /key/update
The key_management_routes group also contains /spend/logs, /team/daily/activity
and other routes whose handlers scope non-admin callers by user_id. A userless
service account key would have reached them unscoped, so the route check now
uses a dedicated two-route allowlist

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 09:16:57 +00:00
shivam
2542b9cfe8 style(keys): fix ruff format and drop comment that restates the call
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 09:05:10 +00:00
shivam
062e17a7ac refactor(keys): keep validate_key_team_change permission check on the key's assigned member
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 09:05:10 +00:00
shivam
a052da6974 feat(keys): let team service account keys use key management endpoints for their own team
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 09:05:10 +00:00
Yuneng Jiang
460f336d8a
fix(proxy): rebuild pass-through routes whenever the resolved list changes
The reload only re-registered pass-through endpoints when the stored row
still carried the key, so deleting the row left the deleted routes serving
traffic until the process restarted.

It now compares the resolved list before and after the row is applied and
rebuilds on any difference, including a deletion that resolves back to the
config file's list or to nothing.

This matches what _apply_retention_settings already does with the retention
values, so the two reload effects no longer disagree about what counts as a
change.

The tests assert the proxy's registry of live pass-through routes, which is
what decides whether a request is routed upstream or falls through to the
auth error, rather than that the registration helper was called.
2026-09-18 02:05:09 -07:00
ryan
a9ad1bbad8 Merge remote-tracking branch 'origin/main' into litellm_routing_groups_atomic_validation 2026-09-18 09:04:26 +00:00
yassin
3d2913307a Merge remote-tracking branch 'origin/main' into litellm_usage_key_free_aggregate_split 2026-09-18 08:59:24 +00:00
yassin
61ce1b46d9 Merge remote-tracking branch 'origin/main' into litellm_max_parallel_requests_queue_size
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

# Conflicts:
#	tests/test_litellm/test_utils.py
2026-09-18 08:57:53 +00:00
yassin
bc636bc5ff Merge remote-tracking branch 'origin/main' into litellm_vault_login_secret_namespace 2026-09-18 08:57:50 +00:00
yassin
b10944ef1d Merge remote-tracking branch 'origin/main' into litellm_lazy_fastapi_bpe_imports 2026-09-18 08:56:21 +00:00
Yuneng Jiang
23ee8ad1c3
fix(proxy): close the last three surfaces that bypassed config ownership
The two dashboard toggles under litellm_settings wrote through save_config,
so the refusal applied, but they mutated the litellm module global first: a
refused write still took effect in the running process until the next reload.
Both now check before they mutate.

/config/field/delete drops the stored key without touching the store, so a
deleted key kept reading back from the process. It now refreshes the store
like the other write paths.

/config/list reported source and editable for the general_settings rows but
not for the litellm_settings ones, so the dashboard would have shown a
config-declared toggle as editable.
2026-09-18 01:53:54 -07:00
yucheng
c5cf32b49d test(otel v2): drop the explanatory docstrings from the span scope tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 08:50:10 +00:00
Yuneng Jiang
8e67a33fc3
fix(proxy): refuse config-owned writes at one choke point and refresh the store
Both write paths now go through the same refusal, so /config/field/update and
/config/update answer identically instead of each phrasing its own rule.

A successful write now applies to the SettingsStore, so the next read sees it.
Without this, /config/field/info reported a key the dashboard had just stored
as "not set" until the process reloaded from the database.

resolve() no longer takes a KeyRule it never reads; the store picks the row.
The matrix tests resolve through SettingsStore instead of calling resolve
directly, so the section and key in each case actually route a lookup.

ConfigFieldInfo and ConfigList type `source` as the FieldSource literal, and
the dashboard API types are regenerated for the two new fields.
2026-09-18 01:49:30 -07:00
yucheng
b346eefd1f fix(proxy): rerun requested model guardrail merge on fallback instead of carrying a raw list
Structured guardrail entries (dicts) are unhashable, so merging a carried list through _merge_guardrails_with_existing raised TypeError on the fallback path. Resolve the requested model alias through _check_and_merge_model_level_guardrails instead and add a regression test for structured request guardrails

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 08:33:46 +00:00
Yucheng He
ffe5d303e5 fix(llmguard): accept proxy async call types 2026-09-18 01:30:46 -07:00
Yuneng Jiang
afa4a6fe78
refactor(proxy): make the config file win over the database
The precedence used to vary per key: some keys let a stored row win, some
let the file win, some merged the two. That meant an operator could not
answer "which value is live?" without knowing the key.

Now file presence decides ownership. A key the config file declares is
config-owned, whatever the database holds, and a key the file omits falls
back to the stored row. KeyRule no longer carries a RuleKind, only which
row the stored value lives in.

Writes to a config-owned key are refused at the two surfaces that reach
the database instead of being stored and silently ignored: save_config
and /config/field/update both 400 naming the key and the config file path.

Both read endpoints now report source and editable off the same
SettingsStore, so /config/field/info and /config/list can no longer
disagree inside one process.

Replaces the 786-case checked-in JSON fixture with cases generated from
the rule table, so the matrix tests no longer assert that resolve() agrees
with a snapshot of resolve().

BREAKING CHANGE: a dashboard or /config/field/update write to a setting
the config file declares now returns 400 instead of being stored. Remove
the key from the config file to let the database own it.
2026-09-18 01:28:21 -07:00
yucheng
318b027782 fix(otel): keep caller traceparent and tracestate on pass-through relays
Pass-through requests inject the proxy span into upstream headers since #40669, which
replaced an explicit x-pass-traceparent with an unrelated trace and dropped its
x-pass-tracestate. Keep the caller's context when the carrier already names a
different trace, and keep the proxy child span for same-trace or missing headers.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 08:26:39 +00:00
Yuneng Jiang
8dcf9e8b78
fix(proxy): correct settings source provenance 2026-09-18 01:26:10 -07:00
yucheng
1ccbc51ed8 test(otel v2): clear the cached LITELLM_OTEL_V2 flag after each destination test so it stops leaking into later modules
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 08:25:54 +00:00
yucheng
75f926da36 fix(otel v2): type the operator sink scope pairs so the widest scope reduction stays Literal
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 08:06:12 +00:00
yucheng
d264cdf231 fix(otel v2): record the wider scope when two operator exporters write one account
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 08:04:49 +00:00
yucheng
a187a5bfc6 fix(proxy): keep requested model guardrails and key disable_fallbacks on rate-limit fallback
The local rate-limit fallback path introduced in #40596 restores the request from a snapshot taken before the pre-call pass, so the guardrails resolved for the requested model were dropped when another deployment was selected, and the raw request-body disable_fallbacks field gated the fallback before the key-level disable_fallbacks override had been applied

Carry the requested model's merged guardrail list onto every fallback pass and read the effective disable_fallbacks value after the pre-call pass

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 07:54:01 +00:00
yucheng
755890b59a fix(otel v2): scope-aware additive dedupe, reject langfuse_span_scope off langfuse_otel, render the scope as a select
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 07:53:32 +00:00
Yuneng Jiang
a7d4f7c521
Merge remote-tracking branch 'origin/main' into litellm_settings_store_precedence 2026-09-18 00:51:38 -07:00
yucheng
b2b64f8185 merge: origin/main into litellm_prompt_injection_llm_api_check_dispatch
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 07:43:34 +00:00
David Steele
054771cac5
test(azure): remove redundant o-series assertion 2026-09-18 08:41:46 +01:00
yucheng-berri
8fc9c46d1a
Merge pull request #41541 from BerriAI/litellm_prompt_injection_async_llm_check
fix(proxy): run prompt injection heuristics off the event loop
2026-09-18 00:40:25 -07:00
Yuneng Jiang
0524745510
fix(proxy): preserve config pass-through and retention reloads 2026-09-18 00:23:24 -07:00
Yuneng Jiang
5a8d1f5eca
feat(proxy): report sources in config read endpoints 2026-09-18 00:06:56 -07:00
Yuneng Jiang
eb831d956c
test(ui): address review feedback 2026-09-17 23:46:53 -07:00
Yuneng Jiang
d1cd869012
refactor(proxy): resolve config and DB settings precedence in one SettingsStore 2026-09-17 23:36:27 -07:00
Yuneng Jiang
1be40e304d
refactor(proxy): add settings precedence primitives 2026-09-17 23:35:38 -07:00
Yuneng Jiang
372a36aa75
Merge remote-tracking branch 'origin/main' into litellm_dashboard-form-happy-paths 2026-09-17 23:28:52 -07:00
Yuneng Jiang
1c08c78ad5
test(ui): retain primary cleanup failures 2026-09-17 23:27:54 -07:00
kerry-berri
c256c3c1a6
Merge pull request #41770 from BerriAI/litellm-providers/price-sync-openrouter
chore(prices): sync OpenRouter prices: 2 models, 1 deprecated
2026-09-17 23:27:49 -07:00
Yuneng Jiang
99ec682a05
Merge remote-tracking branch 'origin/main' into litellm_dashboard-form-happy-paths 2026-09-17 23:24:22 -07:00
Yuneng Jiang
a8ab1187ca
test(ui): clean up synchronous browser failures 2026-09-17 23:23:48 -07:00