Commit graph

49344 commits

Author SHA1 Message Date
Yassin Kortam
cdb4eaac59
Merge pull request #40262 from BerriAI/litellm_lit_7190_inline_token_counts
fix(proxy): run the remaining inline token counts off the event loop
2026-09-15 13:38:00 -07:00
kerry-berri
7d3917dbf4
Merge pull request #41269 from BerriAI/litellm_ban_vendor_fact_pinning_tests
test: drop tests that pin vendor facts and add the CLAUDE.md rule
2026-09-15 13:16:55 -07:00
Yassin Kortam
367393405c
Merge pull request #41170 from BerriAI/litellm_prometheus_401_failed_requests_metric
fix(prometheus): count 401 auth failures in litellm_proxy_failed_requests_metric
2026-09-15 13:13:49 -07:00
Yassin Kortam
0e5be275b0
Merge pull request #41283 from BerriAI/litellm_writer_pinned_raw_sql_writes
fix(proxy): keep access-group raw SQL writes on the writer while writer_unavailable is stale
2026-09-15 13:13:30 -07:00
tin-berri
d07b2e87d2
Merge pull request #41270 from BerriAI/litellm_capability_classifier_pr
feat(router): add capability classifier as Fuse foundation
2026-09-15 13:04:21 -07:00
Yassin Kortam
435f0d22e8
Merge pull request #41156 from BerriAI/litellm_weighted_routing_model_id
fix(router): keep weighted routing when a deployment id equals a model_name
2026-09-15 12:55:45 -07:00
yassin
264305de23 fix(proxy): keep access-group raw SQL writes on the writer while writer_unavailable is stale
A stale RoutingPrismaWrapper.writer_unavailable flag made WriterPinnedClient hand back the routed wrapper, where query_raw is classified as a read, so the access-group UPDATE statements behind /key/regenerate, /key/generate with access_group_ids and model rename/delete went to the read replica and failed with SQLSTATE 25006. Route those raw statements through the underlying writer regardless of the flag; a raw SQL write has no replica fallback. WriterPinnedClient keeps yielding to the replica for degraded reads.

The model sync's backing-row count stays on the writer too: it runs right after the row delete/update on the writer and a lagging replica could still report the removed row, which would leave the group naming a model nobody serves.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 19:55:07 +00:00
Yassin Kortam
4d526ad747
Merge pull request #41216 from BerriAI/litellm_cache_custom_hf_tokenizer
fix(utils): cache custom HuggingFace tokenizers across /utils/token_counter requests
2026-09-15 12:51:18 -07:00
Yassin Kortam
d5b96648ea
Merge pull request #41161 from BerriAI/litellm_headroom_protect_cached_prefix
fix(headroom): protect the cached prefix through the last cache_control breakpoint
2026-09-15 12:49:50 -07:00
Yassin Kortam
c190f0f4b9
Merge pull request #41125 from BerriAI/litellm_alert_5xx_http_exceptions
fix(alerting): send llm_exceptions Slack alert for 5xx HTTPException and ProxyException
2026-09-15 12:48:58 -07:00
Yassin Kortam
a7b894f91b
Merge pull request #41271 from BerriAI/litellm_org_zero_max_budget
fix(proxy): enforce organization budgets when max_budget is 0
2026-09-15 12:47:18 -07:00
Yassin Kortam
60808520df
Merge pull request #40904 from BerriAI/litellm_jwt_agent_id_claim
feat(proxy): bind JWT claims to registered agents via agent_id_jwt_field
2026-09-15 12:46:18 -07:00
yassin
ecc2ed0c94 chore: merge main into litellm_lit_1795 and resolve test_router conflict
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 19:45:39 +00:00
yassin
d350aaf8bd Merge remote-tracking branch 'origin/main' into litellm_prometheus_401_failed_requests_metric 2026-09-15 19:43:47 +00:00
Yassin Kortam
1ca4579375
Merge pull request #41178 from BerriAI/litellm_request_override_selector_callbacks
fix(router): bind per-request routing_strategy override selectors to the request's callbacks
2026-09-15 12:39:20 -07:00
Tin Chi Lo
55fc0deabc style(router): format encrypted task selection 2026-09-15 12:09:05 -07:00
Tin Chi Lo
cadb7ee44d fix(router): preserve native encrypted capability tasks 2026-09-15 12:05:30 -07:00
Mateo Wang
d3929287fe
Merge pull request #41189 from BerriAI/litellm_per_turn_control_beta
fix(anthropic): add the per-turn-control beta when a message carries output_config
2026-09-15 11:53:01 -07:00
Tin Chi Lo
896f35c751 fix(router): extract capability tasks with request scoped markers 2026-09-15 11:52:20 -07:00
Yassin Kortam
501be3143d fix(proxy): enforce organization budgets when max_budget is 0
_organization_max_budget_check returned early whenever org_max_budget
was <= 0, so an organization with an explicit max_budget of 0 was
treated as unlimited instead of zero allowance. Key, team, and user
budget checks already skip only on None; align organization budgets
with that convention.

validate_team_org_change had the same defect in a different shape: it
used a truthy check on the org's max_budget when validating a team
move, so an explicit 0 there silently skipped the guard too.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-15 11:49:05 -07:00
Tin Chi Lo
e62f0d0376 fix(router): reject unknown capability policy fields 2026-09-15 11:43:04 -07:00
mateo-berri
f490172338 test(anthropic): drop docstrings and wrap a long line in the per-turn-control tests 2026-09-15 11:39:01 -07:00
Mateo Wang
2e06d195b2
Merge pull request #39857 from BerriAI/litellm_e2e_reliability_module_cells
test(e2e): cover the reliability retry, cooldown, fallback, and routing-strategy cells
2026-09-15 11:38:51 -07:00
Tin Chi Lo
914ae9b248 test(ui): use the current deployment affinity label 2026-09-15 11:38:28 -07:00
Yassin Kortam
41b5d47c71
Merge pull request #41144 from BerriAI/litellm_responses_bridge_filters_unknown_params
fix(responses): filter bridged kwargs like the native Responses path
2026-09-15 11:32:38 -07:00
Yassin Kortam
837423237a
Merge pull request #36775 from MvdB/litellm_presidio_new_entities
feat(guardrails): add new upstream presidio pii entities including german set
2026-09-15 11:26:21 -07:00
yujonglee
33000d7e25
Merge pull request #41180 from BerriAI/litellm_rust_bridge_native_stub
build(rust-bridge): add typed _native stub and validate it with mypy.stubtest
2026-09-15 11:25:56 -07:00
Devin AI
da7853c20a test: drop tests that pin vendor facts and add the CLAUDE.md rule
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 18:24:30 +00:00
Tin Chi Lo
2da9bbfc0f chore: merge main into capability classifier copy 2026-09-15 11:24:01 -07:00
tin-berri
3ac79757f4
Merge pull request #41175 from BerriAI/litellm_team_member_auto_routers
feat(auto-router): allow opted-in team members to manage their routers
2026-09-15 11:19:58 -07:00
mateo-berri
9bba2df58f chore(ui): regenerate dashboard API types after merging main 2026-09-15 11:19:50 -07:00
Yassin Kortam
dfcefd8298
Merge pull request #41256 from BerriAI/litellm_team_model_access_error_lists_all_models
fix(proxy): list directly assigned team models in model access errors
2026-09-15 11:16:25 -07:00
yassin
cbd72ff6d8 Merge remote-tracking branch 'origin/main' into litellm_responses_bridge_filters_unknown_params 2026-09-15 18:15:19 +00:00
ryan-crabbe-berri
8b6c398b92
Merge pull request #41039 from BerriAI/litellm_bulk_user_delete
feat(proxy): add POST /user/bulk_delete and POST /team/bulk_member_delete
2026-09-15 11:12:40 -07:00
mateo-berri
ce40b5773d Merge remote-tracking branch 'origin/main' into litellm_per_turn_control_beta 2026-09-15 11:11:52 -07:00
ryan
6ea1085bc3 Merge branch 'main' into litellm_bulk_user_delete
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 18:03:15 +00:00
mateo-berri
c0fd8f6012 fix(anthropic): merge a case-variant Anthropic-Beta client header instead of clobbering it 2026-09-15 11:03:10 -07:00
ryan-crabbe-berri
08b433267e
Merge pull request #40917 from BerriAI/litellm_credential_conflict_409
fix(credentials): answer 409 on a credential name collision, make Terraform adoption opt-in
2026-09-15 10:58:26 -07:00
ryan-crabbe-berri
6dcca8c4ae
Merge pull request #41028 from BerriAI/litellm_bulk_new_user
feat(proxy): add POST /management/v1/users/bulk for batched user and team membership creation
2026-09-15 10:57:46 -07:00
Mateo Wang
e5547a56a9
Merge pull request #41257 from BerriAI/litellm_pr_template_coding_tool_proof
docs(github): ask for interactive coding-tool proof in the PR template
2026-09-15 10:44:53 -07:00
ryan-crabbe-berri
a7f180fdd8 feat(terraform): make credential adoption opt-in, escape names in request URLs
Terraform's convention is that create does not seize a resource the
configuration never made, and credential_values holds secrets that are never
read back into state, so a silent takeover overwrites values no plan showed.
A name collision now fails with the terraform import command that adopts the
existing credential explicitly, and adopt_existing = true opts into taking it
over during create. The provider detects the conflict by the proxy's 409 and
keeps the Prisma string match as a fallback for older proxies

Credential names and model_id went into URLs raw, so a name with a slash or a
question mark hit the wrong route. Every credential URL is now built from a
package const through fmt.Sprintf with url.PathEscape or url.QueryEscape,
which the endpoint audit can resolve. Toggling adopt_existing alone no longer
sends a PATCH, so it does not rewrite the stored secret
2026-09-15 10:41:44 -07:00
ryan-crabbe-berri
81806f33cf fix(credentials): answer 409 on a name collision, let PATCH resolve values from model_id
POST /credentials let a duplicate name hit the unique index and handed back
Prisma's "Unique constraint failed" as a 500, so callers string-matched that
message to tell a caller mistake from a server fault. The unique violation now
maps to a 409 whose message names the PATCH route, two concurrent creates of
one name agree on it, and the detection lives in a repository helper the five
hand-rolled copies can move onto later

PATCH /credentials/{name} took a CredentialItem body, so the model_id the
Terraform adopt path sent was dropped. It now accepts UpdateCredentialItem and
shares the deployment lookup with create. Both handlers take the router as a
FastAPI dependency instead of reading the proxy global, which is what the
tests override
2026-09-15 10:41:43 -07:00
kerry-berri
b97bc10ec9
Merge pull request #41263 from BerriAI/litellm_usgov_rows_allow_price_list_source
test(pricing): let synced GovCloud Bedrock rows cite the AWS price list
2026-09-15 10:41:42 -07:00
Devin AI
9fce6e1989 test(pricing): let synced GovCloud Bedrock rows cite the AWS price list
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 17:09:12 +00:00
Devin AI
79450121f8 test(proxy): document access group test seam
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 17:00:11 +00:00
tin-berri
3ad9a7f336
Merge pull request #41174 from BerriAI/litellm_tier_model_affinity
fix(router): preserve session model choice within each complexity tier
2026-09-15 09:54:53 -07:00
mateo-berri
a502afe608 docs(github): ask for interactive coding-tool proof in the PR template 2026-09-15 09:34:45 -07:00
Devin AI
56d0f953f5 fix(proxy): list directly assigned team models in model access errors
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 16:33:41 +00:00
Mateo Wang
c8114ba41f
Merge pull request #40921 from BerriAI/litellm_unified_key_policy_hook
feat(proxy): unified custom_key_policy hook for key generate, update and regenerate
2026-09-15 06:34:28 -07:00
Mateo Wang
9496f16f12
Merge pull request #41173 from BerriAI/litellm_realtime_health_check_credential_name
fix(health): resolve litellm_credential_name in realtime health checks
2026-09-15 05:24:29 -07:00