Commit graph

16834 commits

Author SHA1 Message Date
mateo-berri
6574d83eae Merge remote-tracking branch 'origin/main' into HEAD
# Conflicts:
#	litellm/proxy/guardrails/guardrail_hooks/prompt_security/prompt_security.py
2026-09-14 18:19:38 -07:00
kerry
4a8ec7b9d8 fix(cost): bill gemini-embedding-2-preview per token like the GA entries
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 01:18:50 +00:00
kerry
6a18105275 fix(vertex): only bill image rate without modality details when every input is an image
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 01:16:00 +00:00
kerry
ac8e1a355c test(vertex): load local pricing in embedding billing tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 01:00:32 +00:00
Mateo Wang
dad6e9be4a
Merge branch 'main' into fix/headroom-protect-cache-control-rows 2026-09-14 17:58:17 -07:00
yucheng-berri
8481bc27f9
Merge pull request #41132 from BerriAI/litellm_bedrock_grounding_from_plain_messages
fix(bedrock guardrails): derive contextual grounding source and query from plain messages
2026-09-14 17:56:58 -07:00
kerry
e5845c17ff fix(vertex): bill image inputs at the image rate when usage lacks modality details
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 00:51:49 +00:00
kerry
0c91d9157c refactor(vertex): drop unused resolved_files from embed response parsing
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 00:45:08 +00:00
kerry
e26a4970dd fix(test): complete synthetic model metadata
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 00:42:21 +00:00
kerry
d4f2119b03 fix(cost): bill gemini-embedding-2 per token and stop double charging audio
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 00:41:46 +00:00
Yuneng Jiang
37bde0bdbe
test: keep provider request snapshots immutable 2026-09-14 17:27:18 -07:00
yucheng-berri
b67137b67f
Merge pull request #40702 from BerriAI/litellm_model_armor_logging_only_streaming
feat(model_armor): logging_only mode scans completed streams after delivery
2026-09-14 17:23:32 -07:00
Yuneng Jiang
51db940514
test: relocate strict replay harness coverage 2026-09-14 17:20:58 -07:00
Yuneng Jiang
b9c194076a
test: preserve strict replay numeric spelling 2026-09-14 17:11:56 -07:00
ryan
98ea14758f fix(proxy): only widen an id-only bulk member delete to its email when the user lists the team
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 00:09:57 +00:00
yucheng
5ceec4c21e fix(guardrails): cite message scoping only when an unscoped pass finds content
The not_run reason now says after message scoping only when the same messages carry text or tool calls without the skip flags applied. A request that is empty to begin with, whatever the flags, records no scannable content

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 00:03:02 +00:00
yucheng
52da64a45b fix(guardrails): only cite message scoping in the not_run reason when scoping is on
A request whose messages carry no scannable content at all, with no skip flag set, now records the neutral reason no scannable content instead of blaming configuration

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 23:56:32 +00:00
Yuneng Jiang
6604c78120
test: add strict stateless provider replay identity 2026-09-14 16:55:43 -07:00
ryan
5ace6fa731 fix(proxy): match an id-only bulk member delete against a legacy email-only roster entry
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 23:51:15 +00:00
yucheng
0519d86346 fix(guardrails): stop labelling image-only input as a not_run scoping skip
Images without text were never dispatched to guardrails before this change, so that gap is not a message scoping skip and must not get a not_run entry

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 23:49:03 +00:00
Yassin Kortam
d1fc231545
Merge pull request #41145 from BerriAI/litellm_health_check_skip_save_on_failed_read_main 2026-09-14 16:48:47 -07:00
yucheng
0d0b96ed06 Revert "refactor(guardrails): rename scoped-out evaluation status from not_run to skipped"
This reverts commit b37ce94075.
2026-09-14 23:46:09 +00:00
yucheng
bd9a87ea76 Revert "fix(guardrails): keep legacy not_run neutral and stop labelling image-only input as skipped"
This reverts commit f78dd921c9.
2026-09-14 23:46:09 +00:00
yucheng-berri
99245f9323
Merge pull request #41131 from BerriAI/litellm_prompt_security_created_status
fix(prompt_security): keep polling file sanitization through non-terminal statuses
2026-09-14 16:43:37 -07:00
yassin
cc872c760a test(responses): route the bridge regression through real providers instead of patching ProviderConfigManager
Drops the helper docstring and the test docstrings. The passthrough case now
uses together_ai, which has no native Responses config on main, so the test no
longer monkeypatches ProviderConfigManager at the class level

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 23:27:05 +00:00
tin-berri
c626ff098b
Merge pull request #40877 from BerriAI/litellm_lit7658_cache_cost_v0_fresh
feat(proxy): predict prompt-cache costs across deployments
2026-09-14 16:25:41 -07:00
kerry-berri
9481556b5d
Merge pull request #40955 from BerriAI/litellm-providers/price-sync
Some checks are pending
LiteLLM Rust / rust-test (push) Waiting to run
Unit Tests: Documentation Validation / documentation (push) Waiting to run
Unit Tests: Proxy DB Operations / assert-shard-coverage (push) Waiting to run
Unit Tests: Proxy DB Operations / auth-checks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / budgets (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / custom-logging (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / db-and-spend (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / endpoints-and-responses (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / guardrails-hooks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / jwt-and-keys (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / key-generation (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / logging-misc (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-runtime (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-server-core (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-utils (push) Blocked by required conditions
Unit Tests / caching-local (push) Waiting to run
Unit Tests / core-utils (push) Waiting to run
Unit Tests / enterprise-package (push) Waiting to run
Unit Tests / enterprise-routing (push) Waiting to run
Unit Tests / integrations (push) Waiting to run
Unit Tests / All Other Providers (push) Waiting to run
Unit Tests / Vertex AI (push) Waiting to run
Unit Tests / misc (push) Waiting to run
Unit Tests / proxy-auth (push) Waiting to run
Unit Tests / proxy-endpoints (push) Waiting to run
Unit Tests / proxy-extras (push) Waiting to run
Unit Tests / proxy-infra (push) Waiting to run
Unit Tests / proxy-server (push) Waiting to run
Unit Tests / responses-caching-types (push) Waiting to run
GitHub Actions Security Analysis / zizmor (push) Waiting to run
chore(prices): sync Vertex AI prices: 14 models
2026-09-14 16:16:07 -07:00
yassin
c6a36fb8b6 fix(health): skip background health check DB writes when the latest-row read fails
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit 6d2c4899b0)
2026-09-14 23:03:30 +00:00
yassin
ece2bbfed7 fix(proxy): resolve x-litellm-call-id from response metadata when routes omit call_id
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 23:03:27 +00:00
yuneng-jiang
63386d6cc6
Merge pull request #41070 from BerriAI/litellm_integration_management
test: cover persisted updates and warmed authorization policies
2026-09-14 15:57:32 -07:00
yassin
04c003c098 fix(responses): filter bridged kwargs like the native Responses path
A Responses request for a provider with a native Responses config that is served
through the chat-completions bridge (use_chat_completions_api or the
openai/chat_completions/ prefix) forwarded every raw kwarg, so a deployment-level
chat_template_kwargs reached OpenAI chat completions and got a 400. The bridge
now keeps only the keys a native dispatch would forward plus allowed_openai_params.
Providers with no native Responses config keep the passthrough

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 22:56:58 +00:00
Yassin Kortam
de55e22899
Merge pull request #41141 from BerriAI/litellm_lit7694_forwarded_headers_body_leak
fix(openai): keep extra_headers out of the chat request body on the httpx handler path
2026-09-14 15:52:31 -07:00
Yassin Kortam
1f46e58494
Merge pull request #41018 from BerriAI/litellm_fix_model_alias_map_leak
fix(utils): keep litellm params out of provider request bodies
2026-09-14 15:51:23 -07:00
yuneng-jiang
c57957f45d
Merge pull request #41066 from BerriAI/litellm_integration_foundation
test: add dedicated CircleCI integration contract foundation
2026-09-14 15:50:22 -07:00
kerry
b25b6eb5a4 test(cost_calc): pin vertex_ai gemini-3.5-flash-lite priority cache reads at the published 5.4e-08
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 22:48:59 +00:00
ryan
8e83e91275 fix(proxy): keep bulk user row errors free of the tuple length message
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 22:40:57 +00:00
yucheng
f78dd921c9 fix(guardrails): keep legacy not_run neutral and stop labelling image-only input as skipped
Some checks failed
ai-gateway image / ai-gateway release image (push) Has been cancelled
LiteLLM Rust / rust-lint (push) Has been cancelled
LiteLLM Rust / rust-test (push) Has been cancelled
Terraform Modules / fmt, validate, test (aws) (push) Has been cancelled
Terraform Modules / fmt, validate, test (gcp) (push) Has been cancelled
Terraform Provider / gofmt, vet, build, test (push) Has been cancelled
Terraform Provider / Provider endpoints vs proxy OpenAPI schema (push) Has been cancelled
Usage tracking, compliance and the dashboard now treat both not_run (older spend logs) and skipped as unevaluated through a shared UNEVALUATED_GUARDRAIL_STATUSES set, so old records stop counting as passed. The skipped record is no longer written when the request carried images, since images without text were never dispatched to guardrails before this change and that gap is not a message-scoping skip

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 22:34:33 +00:00
ryan
ca04b03c2c feat(proxy): move bulk user delete and team member delete under /management/v1
Replaces POST /user/bulk_delete and POST /team/bulk_member_delete with
POST /management/v1/users/bulk_delete and
POST /management/v1/teams/{team_id}/members/bulk_delete per the Management
API modernization one-pager: {data} envelopes, application/problem+json
errors with urn:litellm:error:* types, 422 on unknown body fields, 400 on
unknown query params, 403 on authorization failures, 404 on unknown team.

Route checks now match parametrized management/v1 paths so team-scoped
callers reach the endpoint's own authorization and get a 403 problem
instead of the generic 401.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 22:31:07 +00:00
yucheng
d8d7210b57 test(bedrock guardrails): cover tagged messages on the apply_guardrail response path with the flag off
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 22:28:58 +00:00
ryan
96332f75e8 refactor(proxy): move bulk user creation to POST /management/v1/users/bulk
Follows the Management API modernization design: plural resource under
/management/v1, {data, meta} response envelope, unknown request fields
rejected with 422, and RFC 9457 problem+json for request-level errors.
Body validation failures under /management/v1 now answer 422 instead of
the 400 query-parameter problem. /user/bulk_new is removed.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 22:28:39 +00:00
yassin
e2e1d36804 fix(openai): keep extra_headers out of the chat request body on the httpx handler path
Forwarded client headers on bridged /v1/responses calls were serialized into the
OpenAI JSON body as extra_headers when EXPERIMENTAL_OPENAI_BASE_LLM_HTTP_HANDLER
was set, and OpenAI rejected the request with unknown_parameter. The headers are
already merged into the outgoing HTTP headers, so only set the SDK-style
optional param on the SDK client path

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 22:27:35 +00:00
yucheng
b37ce94075 refactor(guardrails): rename scoped-out evaluation status from not_run to skipped
The per-guardrail status a scoped-out evaluation records is now skipped, matching the
skip_*_in_guardrail settings that cause it. Request-level rollup still maps it to not_run
so the StandardLoggingPayload status contract is unchanged

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 22:15:53 +00:00
yucheng
2e11f7bc7b test(bedrock guardrails): shorten grounding test docstrings
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 22:09:59 +00:00
Devin AI
6f9915b650 fix(utils): keep litellm params out of provider request bodies
get_litellm_params() keys such as model_alias_map, completion_call_id and
custom_prompt_dict were missing from all_litellm_params, so when a bridge
forwarded litellm_params as completion kwargs they were swept into extra_body
and OpenAI rejected the call with "Unknown parameter: 'model_alias_map'".

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 22:07:27 +00:00
ryan-crabbe-berri
7b6d223105
Merge pull request #40196 from BerriAI/litellm_team_admin_all_proxy_models
fix(ui): let team admins grant a team all proxy models
2026-09-14 15:05:01 -07:00
yucheng
442af3aab6 fix(bedrock guardrails): gate plain-message grounding behind contextual_grounding_from_messages
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 22:04:42 +00:00
yucheng
3a3075b8a2 fix(prompt_security): poll only on queued statuses, keep 500 for terminal or missing status
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 22:01:39 +00:00
ryan
ef71222349 Merge remote-tracking branch 'origin/main' into litellm_bulk_user_delete 2026-09-14 22:00:10 +00:00
Yassin Kortam
e766277846
Merge pull request #40991 from BerriAI/litellm_team_model_cooldown_siblings
fix(router): cool down team deployments on 429 when a sibling serves the same public model
2026-09-14 14:57:07 -07:00
Yassin Kortam
3e417fa6e0
Merge pull request #41113 from BerriAI/litellm_realtime_release_max_parallel_slot
fix(proxy): release max_parallel_requests slot when a realtime session ends without LLM callbacks
2026-09-14 14:56:11 -07:00