scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-19 00:01:29 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 88
49820 commits 2519 branches 1411 tags 3.1 GiB
Commit graph

17245 commits

Author SHA1 Message Date
kerry
c0c5044c45 fix(batches): keep modality token details in raw vertex batch usage 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-15 01:57:50 +00:00
kerry
ce83fac351 fix(cost): bill batch embeddings per modality token rate 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-15 01:50:21 +00:00
mateo-berri
16c326537f fix(guardrails): define UnappliableRequestRewrite in the shared guardrail translation utils 
The three guardrail translation handlers imported the exception from the proxy policy engine through a function-local import, which CodeQL flagged as a cyclic import. The exception and its helper now live next to the handlers in the shared guardrail translation utils, and the tests import it from there.

The Prompt Security modify-mode helper is also restructured into early-return TypedDict displays so the LIT002 budget stays at its limit
 2026-09-14 18:39:54 -07:00
Mateo Wang
d2859e18d7
Merge pull request #40988 from BerriAI/litellm_responses_stream_error_exception_mapping 
fix(responses): route mid-stream error events through exception_type so content_policy_fallbacks fire
 2026-09-14 18:36:20 -07:00
kerry
a28ea22ec1 fix(cost): move gemini-embedding-2-preview to per-token rates 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-15 01:34:22 +00:00
kerry
27a486e4d3 test(cost): cover modality guards and image detection fallbacks 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-15 01:34:17 +00:00
Mateo Wang
1ba97665b2
Merge pull request #41046 from BerriAI/devin/1789368982-cli-strenum-py310 
fix(cli): drop enum.StrEnum so the CLI imports on Python 3.10
 2026-09-14 18:32:30 -07:00
mateo-berri
931bdb8c0b fix(compression): protect part-level cache_control rows in compress() too 
compress() scores text-only copies of the rows, so a content-part cache_control marker was gone by the time get_protected_indices ran and the pinned row could still be stubbed. Read protection from the original rows, which are index-aligned with the normalized copies, and add a regression test that fails without the change.
 2026-09-14 18:19:51 -07:00
mateo-berri
6574d83eae Merge remote-tracking branch 'origin/main' into HEAD 
# Conflicts:
#	litellm/proxy/guardrails/guardrail_hooks/prompt_security/prompt_security.py
 2026-09-14 18:19:38 -07:00
kerry
4a8ec7b9d8 fix(cost): bill gemini-embedding-2-preview per token like the GA entries 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-15 01:18:50 +00:00
kerry
6a18105275 fix(vertex): only bill image rate without modality details when every input is an image 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-15 01:16:00 +00:00
jesus
9c59feee7c fix(headroom): protect the cached prefix through the last cache_control breakpoint 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-15 01:07:41 +00:00
kerry
ac8e1a355c test(vertex): load local pricing in embedding billing tests 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-15 01:00:32 +00:00
Mateo Wang
dad6e9be4a
Merge branch 'main' into fix/headroom-protect-cache-control-rows 2026-09-14 17:58:17 -07:00
yucheng-berri
8481bc27f9
Merge pull request #41132 from BerriAI/litellm_bedrock_grounding_from_plain_messages 
fix(bedrock guardrails): derive contextual grounding source and query from plain messages
 2026-09-14 17:56:58 -07:00
kerry
e5845c17ff fix(vertex): bill image inputs at the image rate when usage lacks modality details 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-15 00:51:49 +00:00
yassin
1943667fef fix(router): run sync pre-call checks when a model_name collides with a deployment id 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-15 00:51:13 +00:00
kerry
0c91d9157c refactor(vertex): drop unused resolved_files from embed response parsing 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-15 00:45:08 +00:00
kerry
e26a4970dd fix(test): complete synthetic model metadata 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-15 00:42:21 +00:00
yassin
bdc63d590e fix(router): keep weighted routing when a deployment id equals a model_name 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-15 00:42:18 +00:00
kerry
d4f2119b03 fix(cost): bill gemini-embedding-2 per token and stop double charging audio 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-15 00:41:46 +00:00
yucheng
0d03ea3154 Merge remote-tracking branch 'origin/main' into litellm_agent365_mcp_guardrail
Some checks failed
LiteLLM Rust / rust-lint (push) Has been cancelled
Details
ai-gateway image / ai-gateway release image (push) Has been cancelled
Details
LiteLLM Rust / rust-test (push) Has been cancelled
Details
2026-09-15 00:30:12 +00:00
yucheng
4bfaf74edc refactor(guardrails): keep Agent 365 PR to the guardrail, move MCP changes to stacked PRs 
The listed-tool metadata and per-caller catalog cache move to a follow-up PR
stacked on this one, and the RFC 9728 sign-in challenge and discovery metadata
move to a second PR stacked on that. On its own the guardrail evaluates the
tool name and arguments and the caller presents its Entra token

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-15 00:29:58 +00:00
Yuneng Jiang
37bde0bdbe
test: keep provider request snapshots immutable 2026-09-14 17:27:18 -07:00
yucheng
b030164f73 Merge remote-tracking branch 'origin/main' into litellm_agent365_mcp_guardrail 2026-09-15 00:26:03 +00:00
yucheng-berri
b67137b67f
Merge pull request #40702 from BerriAI/litellm_model_armor_logging_only_streaming 
feat(model_armor): logging_only mode scans completed streams after delivery
 2026-09-14 17:23:32 -07:00
Yuneng Jiang
51db940514
test: relocate strict replay harness coverage 2026-09-14 17:20:58 -07:00
Yuneng Jiang
b9c194076a
test: preserve strict replay numeric spelling 2026-09-14 17:11:56 -07:00
ryan
98ea14758f fix(proxy): only widen an id-only bulk member delete to its email when the user lists the team 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-15 00:09:57 +00:00
yucheng
5ceec4c21e fix(guardrails): cite message scoping only when an unscoped pass finds content 
The not_run reason now says after message scoping only when the same messages carry text or tool calls without the skip flags applied. A request that is empty to begin with, whatever the flags, records no scannable content

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-15 00:03:02 +00:00
yucheng
52da64a45b fix(guardrails): only cite message scoping in the not_run reason when scoping is on 
A request whose messages carry no scannable content at all, with no skip flag set, now records the neutral reason no scannable content instead of blaming configuration

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-14 23:56:32 +00:00
Yuneng Jiang
6604c78120
test: add strict stateless provider replay identity 2026-09-14 16:55:43 -07:00
ryan
5ace6fa731 fix(proxy): match an id-only bulk member delete against a legacy email-only roster entry 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-14 23:51:15 +00:00
yucheng
0519d86346 fix(guardrails): stop labelling image-only input as a not_run scoping skip 
Images without text were never dispatched to guardrails before this change, so that gap is not a message scoping skip and must not get a not_run entry

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-14 23:49:03 +00:00
Yassin Kortam
d1fc231545
Merge pull request #41145 from BerriAI/litellm_health_check_skip_save_on_failed_read_main 2026-09-14 16:48:47 -07:00
yucheng
0d0b96ed06 Revert "refactor(guardrails): rename scoped-out evaluation status from not_run to skipped" 
This reverts commit b37ce94075.
 2026-09-14 23:46:09 +00:00
yucheng
bd9a87ea76 Revert "fix(guardrails): keep legacy not_run neutral and stop labelling image-only input as skipped" 
This reverts commit f78dd921c9.
 2026-09-14 23:46:09 +00:00
yucheng-berri
99245f9323
Merge pull request #41131 from BerriAI/litellm_prompt_security_created_status 
fix(prompt_security): keep polling file sanitization through non-terminal statuses
 2026-09-14 16:43:37 -07:00
yassin
54f11b29a4 test(proxy): inject a fake prisma floor instead of patching SpendCounterReseed.from_db 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-14 23:32:25 +00:00
yassin
cc872c760a test(responses): route the bridge regression through real providers instead of patching ProviderConfigManager 
Drops the helper docstring and the test docstrings. The passthrough case now
uses together_ai, which has no native Responses config on main, so the test no
longer monkeypatches ProviderConfigManager at the class level

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-14 23:27:05 +00:00
tin-berri
c626ff098b
Merge pull request #40877 from BerriAI/litellm_lit7658_cache_cost_v0_fresh 
feat(proxy): predict prompt-cache costs across deployments
 2026-09-14 16:25:41 -07:00
yassin
e91c6ca789 fix(proxy): fall back to direct spend increments once the early reconcile has finalized the reservation 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-14 23:22:01 +00:00
kerry-berri
9481556b5d
Merge pull request #40955 from BerriAI/litellm-providers/price-sync
Some checks are pending
LiteLLM Rust / rust-test (push) Waiting to run
Details
Unit Tests: Documentation Validation / documentation (push) Waiting to run
Details
Unit Tests: Proxy DB Operations / assert-shard-coverage (push) Waiting to run
Details
Unit Tests: Proxy DB Operations / auth-checks (push) Blocked by required conditions
Details
Unit Tests: Proxy DB Operations / budgets (push) Blocked by required conditions
Details
Unit Tests: Proxy DB Operations / custom-logging (push) Blocked by required conditions
Details
Unit Tests: Proxy DB Operations / db-and-spend (push) Blocked by required conditions
Details
Unit Tests: Proxy DB Operations / endpoints-and-responses (push) Blocked by required conditions
Details
Unit Tests: Proxy DB Operations / guardrails-hooks (push) Blocked by required conditions
Details
Unit Tests: Proxy DB Operations / jwt-and-keys (push) Blocked by required conditions
Details
Unit Tests: Proxy DB Operations / key-generation (push) Blocked by required conditions
Details
Unit Tests: Proxy DB Operations / logging-misc (push) Blocked by required conditions
Details
Unit Tests: Proxy DB Operations / proxy-runtime (push) Blocked by required conditions
Details
Unit Tests: Proxy DB Operations / proxy-server-core (push) Blocked by required conditions
Details
Unit Tests: Proxy DB Operations / proxy-utils (push) Blocked by required conditions
Details
Unit Tests / caching-local (push) Waiting to run
Details
Unit Tests / core-utils (push) Waiting to run
Details
Unit Tests / enterprise-package (push) Waiting to run
Details
Unit Tests / enterprise-routing (push) Waiting to run
Details
Unit Tests / integrations (push) Waiting to run
Details
Unit Tests / All Other Providers (push) Waiting to run
Details
Unit Tests / Vertex AI (push) Waiting to run
Details
Unit Tests / misc (push) Waiting to run
Details
Unit Tests / proxy-auth (push) Waiting to run
Details
Unit Tests / proxy-extras (push) Waiting to run
Details
Unit Tests / proxy-infra (push) Waiting to run
Details
Unit Tests / proxy-server (push) Waiting to run
Details
Unit Tests / responses-caching-types (push) Waiting to run
Details
GitHub Actions Security Analysis / zizmor (push) Waiting to run
Details
Unit Tests / proxy-endpoints (push) Waiting to run
Details 
chore(prices): sync Vertex AI prices: 14 models
 2026-09-14 16:16:07 -07:00
yassin
c6a36fb8b6 fix(health): skip background health check DB writes when the latest-row read fails 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit 6d2c4899b0)
 2026-09-14 23:03:30 +00:00
yassin
ece2bbfed7 fix(proxy): resolve x-litellm-call-id from response metadata when routes omit call_id 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-14 23:03:27 +00:00
yassin
d08e43c6af fix(proxy): invalidate reserved counters when the early reconcile fails
Some checks failed
ai-gateway image / ai-gateway release image (push) Has been cancelled
Details
LiteLLM Rust / rust-lint (push) Has been cancelled
Details
LiteLLM Rust / rust-test (push) Has been cancelled
Details
Terraform Modules / fmt, validate, test (aws) (push) Has been cancelled
Details
Terraform Modules / fmt, validate, test (gcp) (push) Has been cancelled
Details
Terraform Provider / gofmt, vet, build, test (push) Has been cancelled
Details
Terraform Provider / Provider endpoints vs proxy OpenAPI schema (push) Has been cancelled
Details 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-14 23:01:30 +00:00
yassin
a3636acd0d fix(proxy): reconcile budget reservation before enqueuing spend to the DB 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-14 23:01:30 +00:00
yucheng
f235c8b3a2 fix(agent_365): keep sign-in discovery for servers forwarding an API-key header
Some checks are pending
ai-gateway image / ai-gateway release image (push) Waiting to run
Details
LiteLLM Rust / rust-lint (push) Waiting to run
Details
LiteLLM Rust / rust-test (push) Waiting to run
Details 
A forwarded x-api-key, api-key or apikey header is the upstream's own credential and leaves the caller's top-level Authorization with the gateway, so Agent 365 discovery and the connect-time challenge must not be suppressed for those servers. Only modes or forwarded headers that consume top-level Authorization opt a server out.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-14 22:59:26 +00:00
yuneng-jiang
63386d6cc6
Merge pull request #41070 from BerriAI/litellm_integration_management 
test: cover persisted updates and warmed authorization policies
 2026-09-14 15:57:32 -07:00
yassin
04c003c098 fix(responses): filter bridged kwargs like the native Responses path 
A Responses request for a provider with a native Responses config that is served
through the chat-completions bridge (use_chat_completions_api or the
openai/chat_completions/ prefix) forwarded every raw kwarg, so a deployment-level
chat_template_kwargs reached OpenAI chat completions and got a 400. The bridge
now keeps only the keys a native dispatch would forward plus allowed_openai_params.
Providers with no native Responses config keep the passthrough

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-14 22:56:58 +00:00