Commit graph

5634 commits

Author SHA1 Message Date
mateo
12c71ab407 fix(proxy): block bulk user creation during the master key lockout
Some checks failed
Terraform Provider / gofmt, vet, build, test (push) Has been cancelled
ai-gateway image / ai-gateway release image (push) Has been cancelled
LiteLLM Rust / rust-lint (push) Has been cancelled
LiteLLM Rust / rust-test (push) Has been cancelled
Terraform Provider / Provider endpoints vs proxy OpenAPI schema (push) Has been cancelled
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 18:40:44 +00:00
Devin AI
417d33911c Merge remote-tracking branch 'origin/main' into litellm_refuse_example_master_key
# Conflicts:
#	litellm/proxy/_types.py
2026-09-15 18:15:12 +00:00
ryan-crabbe-berri
8b6c398b92
Merge pull request #41039 from BerriAI/litellm_bulk_user_delete
feat(proxy): add POST /user/bulk_delete and POST /team/bulk_member_delete
2026-09-15 11:12:40 -07:00
mateo
a40804b7dc fix(proxy): block POST /user/new during the master key lockout
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 18:06:57 +00:00
ryan
6ea1085bc3 Merge branch 'main' into litellm_bulk_user_delete
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 18:03:15 +00:00
ryan-crabbe-berri
08b433267e
Merge pull request #40917 from BerriAI/litellm_credential_conflict_409
fix(credentials): answer 409 on a credential name collision, make Terraform adoption opt-in
2026-09-15 10:58:26 -07:00
ryan-crabbe-berri
6dcca8c4ae
Merge pull request #41028 from BerriAI/litellm_bulk_new_user
feat(proxy): add POST /management/v1/users/bulk for batched user and team membership creation
2026-09-15 10:57:46 -07:00
ryan-crabbe-berri
81806f33cf fix(credentials): answer 409 on a name collision, let PATCH resolve values from model_id
POST /credentials let a duplicate name hit the unique index and handed back
Prisma's "Unique constraint failed" as a 500, so callers string-matched that
message to tell a caller mistake from a server fault. The unique violation now
maps to a 409 whose message names the PATCH route, two concurrent creates of
one name agree on it, and the detection lives in a repository helper the five
hand-rolled copies can move onto later

PATCH /credentials/{name} took a CredentialItem body, so the model_id the
Terraform adopt path sent was dropped. It now accepts UpdateCredentialItem and
shares the deployment lookup with create. Both handlers take the router as a
FastAPI dependency instead of reading the proxy global, which is what the
tests override
2026-09-15 10:41:43 -07:00
mateo
7f5278eab3 fix(proxy): cover credentials by_model route param in lockout
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 17:04:08 +00:00
Devin AI
2bc38b0b65 Merge remote-tracking branch 'origin/main' into litellm_refuse_example_master_key
# Conflicts:
#	tests/test_litellm/proxy/auth/test_user_api_key_auth.py
#	tests/test_litellm/proxy/test__types.py
#	tests/test_litellm/test_utils.py
2026-09-15 14:24:56 +00:00
Devin AI
74c0e1a339 feat(proxy): lock out credential storage and virtual key management while the master key is insecure 2026-09-15 13:52:30 +00:00
mateo
b9b0f0e7ea Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_refuse_example_master_key 2026-09-15 13:36:33 +00:00
mateo-berri
1f2d050386 Merge remote-tracking branch 'origin/main' into litellm_unified_key_policy_hook 2026-09-15 05:11:29 -07:00
Mateo Wang
3ed6c19b8d
Merge pull request #40915 from BerriAI/litellm_internal_copy_37075
fix(vertex-live): bill Gemini Live sessions end to end (internal copy of #37075)
2026-09-15 02:06:49 -07:00
Mateo Wang
80b9ed4f2c
Merge pull request #41191 from BerriAI/litellm_router_test_cap_resets_per_fallback_hop
fix(router): count num_retries_per_request across fallback hops
2026-09-15 01:13:41 -07:00
Mateo Wang
e5cb8b7534
Merge pull request #40984 from BerriAI/litellm_anthropic_guardrail_system_and_tool_use
fix(guardrails): scan the Anthropic top-level system prompt and tool_use arguments
2026-09-15 00:55:42 -07:00
mateo-berri
1b040af414 test(router): type the retry-cap tests this PR adds or touches 2026-09-15 00:34:38 -07:00
mateo-berri
4f27573424 merge: origin/main into litellm_internal_copy_37075 2026-09-15 00:34:18 -07:00
tin-berri
feab83aae1
Merge pull request #41186 from BerriAI/litellm_statusline_router_cost_label
fix(cli): label savings cost bars with the auto-router name
2026-09-15 00:32:08 -07:00
mateo-berri
f80cb5cb46 fix(router): ignore planted request_retry_count seeds and cover the rust OCR cap path
The router clamps a negative request_retry_count found in request metadata before counting a failure, and the proxy strips a client-supplied request_retry_count with the other router-reserved metadata fields. The rust OCR lifecycle test that trips the per-request cap now plants request_retry_count instead of attempted_retries, which the cap no longer reads since the previous commit
2026-09-15 00:04:02 -07:00
Tin Chi Lo
d0fdf1c237 fix(cli): show only the routed model in the footer header 2026-09-14 23:54:50 -07:00
Tin Chi Lo
4fca818f34 fix(cli): align wide and combining Unicode cost labels 2026-09-14 23:45:48 -07:00
tin-berri
4805c6d51f
Merge pull request #41072 from BerriAI/litellm_lit5201_provider_split
fix(router): honor team and key provider weights
2026-09-14 23:41:45 -07:00
Tin Chi Lo
398300c4e7 fix(router): honor team and key provider weights 2026-09-14 23:31:52 -07:00
Mateo Wang
b52de1675a
Merge pull request #40994 from BerriAI/litellm_sdk_exception_body_headers
fix(sdk): keep body and proxy headers on BadRequestError mapped from a litellm_proxy 400
2026-09-14 23:27:29 -07:00
mateo-berri
62b2b36ce9 test(proxy): drop the reformat-only diff of the request processing tests
The proxy edge test file no longer carries any test of this change, and
the remaining diff was the scoped format gate reflowing the whole file to
the 120 limit, so it goes back to the merge base bytes
2026-09-14 22:39:19 -07:00
yucheng-berri
91588221cd
Merge pull request #39050 from BerriAI/litellm_lit6314_guardrail_metadata_transfer
fix(guardrails): record not_run evaluation when scoping leaves nothing to scan
2026-09-14 22:31:16 -07:00
Mateo Wang
a78b24c195
Merge pull request #41172 from BerriAI/litellm_azure_spend_log_zero_cost
fix(proxy): log the provider usage on deferred /v1/messages calls and price cache writes without a creation rate
2026-09-14 22:26:22 -07:00
mateo-berri
6a635cbb64 fix(sdk): carry a litellm_proxy error's headers on e.response, not e.headers
A mapped litellm_proxy exception now attaches an httpx.Response that
carries the proxy's response headers whenever the handler attached a
header-less synthetic one, on every status branch and on the relay
path. BadRequestError keeps its base-class contract: .headers stays the
proxy-supplied channel, so the proxy edge keeps forwarding an upstream
proxy's headers under the llm_provider- prefix and the date and server
edge change is no longer needed.
2026-09-14 22:22:26 -07:00
mateo-berri
04410967ff Merge remote-tracking branch 'origin/main' into litellm_anthropic_guardrail_system_and_tool_use
main's test_one_text_per_row_over_a_system_prompt_is_rejected_by_name assumed the
top-level system prompt stays out of the scanned texts. This branch scans it, so one
text per structured row now lines up and the rewrite is applied; the test asserts that,
and a multi-block system prompt keeps the length-guard rejection covered.
2026-09-14 22:19:18 -07:00
yucheng
9acdebf563 chore: merge origin/main into litellm_lit6314_guardrail_metadata_transfer
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 05:03:31 +00:00
Mateo Wang
c16c172a65
Merge pull request #40939 from BerriAI/litellm_responses_per_message_guardrail_rewrite
fix(guardrails): write per-message guardrail rewrites back onto Responses input items
2026-09-14 21:57:29 -07:00
mateo-berri
8573241c49 fix(cost): resolve a missing 1h cache write rate after off-peak pricing
The one-hour cache write fallback now takes the applied cache write rate, so an off-peak write price carries into it instead of the input rate

The cost estimate test for a cost-map model without cache prices now expects writes at the input rate, which is what the proxy bills

The recording logger in the deferred guardrail test types its callback parameters
2026-09-14 21:46:33 -07:00
mateo-berri
f4f1e2eace refactor(sdk): move the None sentinel to constants and freeze the init kwargs filter 2026-09-14 21:46:22 -07:00
Tin Chi Lo
bebc76316c fix(cli): label savings cost bars with the auto-router name 2026-09-14 21:42:57 -07:00
mateo-berri
f8c2539ba7 Merge remote-tracking branch 'origin/main' into litellm_azure_spend_log_zero_cost 2026-09-14 21:37:19 -07:00
mateo-berri
e01d97ea08 fix(guardrails): read Prompt Security modified rows with the slot count's own predicate
A chat row whose content carried an empty text part counted two slots in the
chat completions handler while Prompt Security read one text out of the
modified row, so the structured rewrite was dropped and the request got the
named rejection. One shared helper now lists a row's slot texts and both the
slot count and the modified-row reader use it.
2026-09-14 21:31:22 -07:00
mateo-berri
879fcd847f Merge remote-tracking branch 'origin/main' into litellm_sdk_exception_body_headers_do0914 2026-09-14 21:28:23 -07:00
Mateo Wang
2bdafa1206
Merge branch 'main' into fix/headroom-protect-cache-control-rows 2026-09-14 20:53:07 -07:00
mateo-berri
8b80971586 Merge remote-tracking branch 'origin/main' into litellm_anthropic_guardrail_system_and_tool_use 2026-09-14 19:27:58 -07:00
mateo-berri
22b377fe2a fix(proxy): log the provider usage on deferred /v1/messages calls and price cache writes without a creation rate
With a post-call guardrail the proxy defers async success logging, and every nested wrapper on a
/v1/messages call bridged to the Responses API overwrote the stored closure, so the spend log was
built from the outermost Anthropic-shaped reply under Responses semantics and recorded the prompt
tokens without the cache hit. The first wrapper to exit now keeps the slot, which is the innermost
provider response, the same one the non-deferred path logs.

The flat cost path also billed cache-creation tokens at 0 when the model had no
cache_creation_input_token_cost. It now falls back to the input rate, and the 1h rate to the
creation rate, matching the tiered path and the custom pricing helper.
2026-09-14 19:21:03 -07:00
mateo-berri
16c326537f fix(guardrails): define UnappliableRequestRewrite in the shared guardrail translation utils
The three guardrail translation handlers imported the exception from the proxy policy engine through a function-local import, which CodeQL flagged as a cyclic import. The exception and its helper now live next to the handlers in the shared guardrail translation utils, and the tests import it from there.

The Prompt Security modify-mode helper is also restructured into early-return TypedDict displays so the LIT002 budget stays at its limit
2026-09-14 18:39:54 -07:00
Mateo Wang
1ba97665b2
Merge pull request #41046 from BerriAI/devin/1789368982-cli-strenum-py310
fix(cli): drop enum.StrEnum so the CLI imports on Python 3.10
2026-09-14 18:32:30 -07:00
mateo-berri
6574d83eae Merge remote-tracking branch 'origin/main' into HEAD
# Conflicts:
#	litellm/proxy/guardrails/guardrail_hooks/prompt_security/prompt_security.py
2026-09-14 18:19:38 -07:00
Mateo Wang
dad6e9be4a
Merge branch 'main' into fix/headroom-protect-cache-control-rows 2026-09-14 17:58:17 -07:00
yucheng-berri
8481bc27f9
Merge pull request #41132 from BerriAI/litellm_bedrock_grounding_from_plain_messages
fix(bedrock guardrails): derive contextual grounding source and query from plain messages
2026-09-14 17:56:58 -07:00
yucheng-berri
b67137b67f
Merge pull request #40702 from BerriAI/litellm_model_armor_logging_only_streaming
feat(model_armor): logging_only mode scans completed streams after delivery
2026-09-14 17:23:32 -07:00
ryan
98ea14758f fix(proxy): only widen an id-only bulk member delete to its email when the user lists the team
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 00:09:57 +00:00
ryan
5ace6fa731 fix(proxy): match an id-only bulk member delete against a legacy email-only roster entry
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 23:51:15 +00:00
Yassin Kortam
d1fc231545
Merge pull request #41145 from BerriAI/litellm_health_check_skip_save_on_failed_read_main 2026-09-14 16:48:47 -07:00