scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-17 23:51:30 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 67
49465 commits 2515 branches 1410 tags 3 GiB
Commit graph

49465 commits

This branch
This branch
All branches
Author SHA1 Message Date
yassin
f60a603519 fix(proxy): log configured model access denials at the final response boundary 
Post-auth denials from can_key_call_resolved_model (per-request alias
rewrite, MCP sampling, realtime) never reach the auth exception handler,
so the internal allowlist reason was dropped when
model_access_denied_message was set. Log it once from the ProxyException
response handler and the realtime rejection path instead, and convert
JWT ModelAccessDeniedHTTPException into the specialized ProxyException so
the same boundary covers it

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-16 00:12:49 +00:00
yassin
7267c6bed7 Merge remote-tracking branch 'origin/main' into litellm_model_access_denied_message 2026-09-15 23:36:30 +00:00
yassin
b99f0c812f fix(proxy): log configured model access denial only at the auth error boundary 
Move the WARNING that carries the internal denial detail out of the message formatter and into the auth exception handler. The denial exceptions now carry internal_message so access-group probes and fallback paths that catch and recover from the denial no longer log a false denial for an allowed request

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-15 23:36:12 +00:00
tin-berri
8cd00d2d6e
Merge pull request #41282 from BerriAI/litellm_fast_mode_toggle_0915
Some checks failed
Unit Tests: Proxy DB Operations / auth-checks (push) Blocked by required conditions
Details
Unit Tests: Proxy DB Operations / budgets (push) Blocked by required conditions
Details
Unit Tests: Proxy DB Operations / custom-logging (push) Blocked by required conditions
Details
Unit Tests: Proxy DB Operations / proxy-utils (push) Blocked by required conditions
Details
Unit Tests / misc (push) Waiting to run
Details
Unit Tests / proxy-auth (push) Waiting to run
Details
Unit Tests / proxy-endpoints (push) Waiting to run
Details
Unit Tests: Proxy DB Operations / db-and-spend (push) Blocked by required conditions
Details
Unit Tests: Proxy DB Operations / endpoints-and-responses (push) Blocked by required conditions
Details
Unit Tests: Proxy DB Operations / guardrails-hooks (push) Blocked by required conditions
Details
Unit Tests: Proxy DB Operations / jwt-and-keys (push) Blocked by required conditions
Details
Unit Tests: Proxy DB Operations / key-generation (push) Blocked by required conditions
Details
Unit Tests: Proxy DB Operations / logging-misc (push) Blocked by required conditions
Details
Unit Tests: Proxy DB Operations / proxy-runtime (push) Blocked by required conditions
Details
Unit Tests: Proxy DB Operations / proxy-server-core (push) Blocked by required conditions
Details
Unit Tests / caching-local (push) Waiting to run
Details
Unit Tests / core-utils (push) Waiting to run
Details
Unit Tests / enterprise-package (push) Waiting to run
Details
Unit Tests / enterprise-routing (push) Waiting to run
Details
Unit Tests / integrations (push) Waiting to run
Details
Unit Tests / All Other Providers (push) Waiting to run
Details
Unit Tests / Vertex AI (push) Waiting to run
Details
Unit Tests / proxy-extras (push) Waiting to run
Details
Unit Tests / proxy-infra (push) Waiting to run
Details
Unit Tests / proxy-server (push) Waiting to run
Details
Unit Tests / responses-caching-types (push) Waiting to run
Details
GitHub Actions Security Analysis / zizmor (push) Waiting to run
Details
Terraform Provider / gofmt, vet, build, test (push) Has been cancelled
Details
Terraform Provider / Provider endpoints vs proxy OpenAPI schema (push) Has been cancelled
Details
ai-gateway image / ai-gateway release image (push) Has been cancelled
Details 
feat(auto-router): add per-model Fast mode toggle
 2026-09-15 16:10:06 -07:00
Mateo Wang
a056370e1e
Merge pull request #41179 from IToSSc/feat/TASK-2BK38Y-model-prices-docs 
feat: add aihubmix provider pricing entries
 2026-09-15 16:09:05 -07:00
yuneng-jiang
fa9177d5f2
Merge pull request #41321 from BerriAI/litellm_/release-version-bump-940c92 
chore: bump litellm-enterprise 0.1.67 -> 0.1.68, litellm-proxy-extras 0.4.97 -> 0.4.98, litellm 1.102.0 -> 1.103.0
 2026-09-15 16:06:39 -07:00
Mateo Wang
6bc821492b
Merge pull request #40476 from BerriAI/litellm_codex_model_catalog_sync 
feat(cli): sync Codex /model picker from proxy /v1/models in lite codex
 2026-09-15 15:59:04 -07:00
Yuneng Jiang
56685ddeae
Merge remote-tracking branch 'origin/main' into litellm_/release-version-bump-940c92 2026-09-15 15:56:29 -07:00
Yassin Kortam
0bb95e298c
Merge pull request #41309 from BerriAI/litellm_playground_custom_request_headers 2026-09-15 15:56:22 -07:00
Yuneng Jiang
818fd7bbb1
Merge remote-tracking branch 'origin/main' into litellm_/release-version-bump-940c92 2026-09-15 15:56:18 -07:00
Yassin Kortam
9d75cdd502
Merge pull request #41304 from BerriAI/litellm_lit1698_openai_system_messages_first 2026-09-15 15:56:13 -07:00
Yassin Kortam
9bb83fdaea
Merge pull request #41281 from BerriAI/litellm_lit_7417_jwt_key_mapping_issuer_scope 
fix(jwt-auth): scope JWT key mappings by issuer to prevent cross-issuer collisions
 2026-09-15 15:51:50 -07:00
Yuneng Jiang
ad74c3ede7
bump: litellm-enterprise 0.1.67 -> 0.1.68, litellm-proxy-extras 0.4.97 -> 0.4.98, litellm 1.102.0 -> 1.103.0 2026-09-15 15:48:54 -07:00
Yassin Kortam
eec954bf9a
Merge pull request #41296 from BerriAI/litellm_models_table_url_state 
feat(ui): persist Models table search, filters, sort and page in the URL
 2026-09-15 15:46:27 -07:00
Yassin Kortam
94a6dfaf52
Merge pull request #41307 from BerriAI/litellm_passthrough_deployment_model_id 
fix(passthrough): attribute Vertex passthrough successes to the resolved router deployment
 2026-09-15 15:44:38 -07:00
yassin
d4d8cc9092 fix(proxy): apply access denied message to JWT paths, sanitize denial log, await dashboard saves 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-15 22:44:03 +00:00
Mateo Wang
6ee1d72ae8
Merge pull request #40872 from BerriAI/litellm_bedrock_sanitize_tool_use_id 
fix(bedrock): sanitize client tool_call ids to Bedrock toolUseId constraints
 2026-09-15 15:41:39 -07:00
Mateo Wang
c3222ec110
Merge pull request #41171 from BerriAI/litellm_converted_stream_spend_tracking 
fix(logging): track spend for streams a deployment hook converted to non-streaming
 2026-09-15 15:34:13 -07:00
mateo-berri
fa116a0ecc Merge remote-tracking branch 'origin/main' into litellm_bedrock_sanitize_tool_use_id 2026-09-15 15:27:55 -07:00
yassin
754e87fa67 fix(ui): keep gateway auth and content-type headers ahead of playground custom headers 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-15 22:24:13 +00:00
yassin
9261a72eb7 test(passthrough): cover Vertex router resolution without a matching deployment 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-15 22:14:44 +00:00
yassin
c05af7a12f feat(ui): add custom request headers to the API Playground 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-15 22:12:06 +00:00
Yassin Kortam
226b1e1bb9
Merge pull request #41288 from BerriAI/litellm_langsmith_preserve_events_during_flush 
fix(langsmith): keep events appended during an in-flight flush instead of clearing them
 2026-09-15 15:06:36 -07:00
yassin
168b5bc4fb feat(proxy): configurable client-facing model access denied message 
Add litellm_settings.model_access_denied_message, a template ({model} placeholder) returned to clients instead of the detailed "can only access models=[...]" text on key/team/user/org/project and team-member model access denials. The full denial reason is still written to the proxy logs at WARNING. Unset keeps the existing detailed message, status codes and error types are unchanged.

Expose the new setting and the existing expose_router_debug_in_errors flag in the Admin UI general settings (String editor, Boolean toggle with an explicit True default) and allow both as safe DB overrides so they persist and propagate across workers.

Resolves LIT-5283

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-15 22:05:08 +00:00
Yassin Kortam
e36626174b
Merge pull request #41297 from BerriAI/litellm_return_400_on_lone_surrogate_input 
fix(proxy): return 400 instead of 500 for lone surrogate escapes in request body
 2026-09-15 15:04:08 -07:00
Yassin Kortam
579a30de83
Merge pull request #41300 from BerriAI/litellm_lit_1742_custom_provider_map_router 
fix(router): accept custom_provider_map providers before the first completion call
 2026-09-15 15:03:42 -07:00
Yassin Kortam
ee51be4db2
Merge pull request #41289 from BerriAI/litellm_fix_clientside_credential_deployment_scope 
fix(router): stop registering a caller-supplied credential as a router deployment
 2026-09-15 15:00:50 -07:00
yassin
c49fb1dd9d fix(openai): drop env var read for openai_system_messages_first, config and Admin UI set it 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-15 21:58:35 +00:00
yassin
5237fe4df3 refactor(passthrough): read the deployment model_info request state in two steps 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-15 21:57:12 +00:00
Yassin Kortam
1b8daf20e0
Merge pull request #38268 from BerriAI/litellm_fix_xai_web_search_nested_filters 
fix(xai): honor nested web_search filters on the xAI Responses API
 2026-09-15 14:56:36 -07:00
yassin
99fa38504a fix(ui): bound Models table page, page size and sort_by read from the URL 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-15 21:55:07 +00:00
yassin
77d913958d feat(openai): add openai_system_messages_first to put system messages first for prompt caching 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-15 21:53:29 +00:00
Yassin Kortam
fa09de9e45
Merge pull request #41279 from BerriAI/litellm_reset_budget_decrement_spend 
fix(proxy): reset budgets by decrementing pre-reset spend instead of zeroing rows
 2026-09-15 14:45:03 -07:00
Yassin Kortam
c1e1d62903
Merge pull request #41291 from BerriAI/litellm_lit1643_auth_failure_user_agent 
fix(proxy): keep client User-Agent on auth failure spend logs
 2026-09-15 14:44:32 -07:00
yassin
0cc6968495 fix(router): accept custom_provider_map providers before the first completion call 
get_llm_provider() and Router._add_deployment() only knew the built-in
provider_list and JSON providers, so a provider registered through
litellm.custom_provider_map was rejected until custom_llm_setup() had
run inside the first completion() call. Both now check the map directly.

Resolves LIT-1742

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-15 21:43:47 +00:00
tin-berri
ce1a4f896a
Merge pull request #41272 from BerriAI/litellm_fuse_v2_classifier_pr 
feat(router): add Fuse V2 classifier after capability forecasting
 2026-09-15 14:43:44 -07:00
yassin
e62ff9ebee fix(proxy): return 400 instead of 500 for lone surrogate escapes in request body 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-15 21:40:53 +00:00
yassin
96bffb1290 fix(passthrough): attribute Vertex passthrough successes to the resolved router deployment 
The Vertex passthrough route resolved a router deployment only to rewrite the
upstream URL and dropped its model_info, so the standard logging payload and
the Prometheus litellm_deployment_success_responses_total counter carried
model_id="". Carry the deployment's model_info through request.state into the
passthrough logging metadata, where it overrides any client-supplied model_info.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-15 21:39:39 +00:00
yassin
868d3855ab feat(ui): persist Models table search, filters, sort and page in the URL 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-15 21:37:09 +00:00
Yassin Kortam
140229bc4a
Merge pull request #41230 from BerriAI/litellm_client_timeout_408_skips_cooldown 
fix(router): stop counting caller-set timeout 408s toward deployment cooldown
 2026-09-15 14:31:11 -07:00
yassin
afb6f8be65 fix(xai): treat an explicit empty web_search filters object as unrestricted 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-15 21:24:32 +00:00
Yassin Kortam
a6127d2363
Merge pull request #35180 from BerriAI/litellm_lit4995_vertex_rerank_search_units 
fix(rerank): bill Vertex search_units from input records and give every rerank response a unique id
 2026-09-15 14:23:23 -07:00
Yassin Kortam
5df127d483 fix(router): stop registering a caller-supplied credential as a router deployment 
_handle_clientside_credential registered the per-request Deployment it built for
a client-supplied api_key/api_base via upsert_deployment, which added it to
self.model_list under the shared model_name. That made a request-scoped
credential a permanent, load-balanced deployment that any later caller of the
same model group could be routed onto, reaching the provider with someone
else's forwarded credential.

The per-request Deployment still gets its own stable id for cooldown and
logging identity; it is just never registered with the router.

Resolves LIT-7811
 2026-09-15 14:18:03 -07:00
mrinal
4c179f2f59 test(langsmith): type the flush race test and cancel its periodic task 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-15 21:17:36 +00:00
Yassin Kortam
d35af8d302
Merge pull request #38682 from BerriAI/devin_ai_1787937848_tf_team_member_fields 
feat(terraform): add tpm_limit, rpm_limit, budget_duration, allowed_models to litellm_team_member_add
 2026-09-15 14:12:25 -07:00
yassin
595bec46ff fix(router): time fallback-hop 408s against now, not the previous hop's end_time 
The failure logger skips fallback hops (has_logged_async_failure is already set), so
model_call_details.end_time still belongs to the previous hop and predates this hop's
api_call_start_time. The fallback cooldown guard measured a negative elapsed time and
cooled down deployments for caller-set timeouts.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-15 21:11:40 +00:00
yassin
2f719fec52 test(router): run the fallback provider-408 cooldown regression inside an event loop 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-15 21:11:40 +00:00
yassin
5da497f4ac fix(router): only exempt 408s that arrive after the caller's timeout from cooldown 
client_side_timeout records that the caller configured a timeout, not that
the timeout fired. A 408 the provider returns before that deadline is a
deployment failure and must still count toward cooldown.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-15 21:11:40 +00:00
yassin
67c17b68fa refactor(jwt-auth): extract issuer-scoped mapping lookup to satisfy C901 budget 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-15 21:09:01 +00:00
Yassin Kortam
e54b93017b fix(jwt-auth): scope JWT key mappings by issuer to prevent cross-issuer collisions 2026-09-15 21:02:17 +00:00