Commit graph

50950 commits

Author SHA1 Message Date
yucheng-berri
84df4c0d1b
Merge pull request #41783 from BerriAI/litellm_rate_limit_fallback_guardrails
fix(proxy): keep requested model guardrails and key disable_fallbacks on rate-limit fallback
2026-09-18 14:59:16 -07:00
Yassin Kortam
87694c26ef
Merge pull request #41324 from BerriAI/litellm_daily_global_spend_table
feat(proxy): add LiteLLM_DailyGlobalSpend key-free rollup for the usage dashboard
2026-09-18 14:53:08 -07:00
Yassin Kortam
47209d37f2
Merge pull request #41882 from BerriAI/litellm_azure_speech_api_base_prefix
fix(proxy): classify Azure Speech short audio behind a prefixed api base
2026-09-18 14:50:28 -07:00
yujonglee
59604b2b19
Merge pull request #41884 from BerriAI/litellm_ocr_test_matrix
test(ocr): declarative provider x auth x input matrix for tests/ocr_tests
2026-09-18 14:50:05 -07:00
Yassin Kortam
52d6aab421
Merge pull request #41554 from BerriAI/litellm_deepgram_listen_websocket_passthrough
feat(passthrough): deepgram streaming /v1/listen WebSocket passthrough with duration-based cost tracking
2026-09-18 14:48:37 -07:00
Mateo Wang
d45e04a9fd
Merge pull request #41062 from BerriAI/litellm_mistral_codex_reasoning_effort_client_metadata
fix(mistral): accept reasoning_effort on all models and drop client_metadata for Codex compatibility
2026-09-18 14:43:58 -07:00
Yujong Lee
f72b7155ac fix(ocr): map Rust upstream 401/403 to the public auth exceptions
The httpx.Response built for a Rust upstream failure had no request attached,
so constructing openai.AuthenticationError raised RuntimeError inside the
exception mapper and every bad-key OCR call surfaced as APIConnectionError 500
instead of AuthenticationError 401 (the Python path already returned 401)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 21:40:13 +00:00
Mateo Wang
a6bd779bd1
Merge pull request #39424 from emerzon/litellm_azure_ai_flux_2_flex
feat(azure_ai): support FLUX.2 flex images
2026-09-18 14:30:25 -07:00
yassin
0b5b69ea3a fix(deepgram): forward only the first model and language values to /listen
Authorization and pricing read the first model and language query value, but the raw query was forwarded, so Deepgram (which honours the last repeated value) could be sent a model the key was never allowed. Later duplicates of those two keys are now dropped before the upstream URL is built

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 21:29:29 +00:00
Yujong Lee
9767878425 test(ocr): replace per-provider OCR test classes with a declarative provider x auth x input matrix
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 21:27:03 +00:00
ryan-crabbe-berri
a43a4924a6
Merge pull request #40878 from BerriAI/litellm_null_cost_unpriced_deployments
fix(router): report null cost for unpriced deployments instead of 0
2026-09-18 14:20:52 -07:00
ryan-crabbe-berri
2332d3f183
Merge pull request #40737 from BerriAI/litellm_logs_user_email_display
fix(ui): show internal user email in logs table and log detail drawer
2026-09-18 14:20:16 -07:00
yujonglee
b0b2f13548
Merge pull request #41873 from BerriAI/litellm_rust_exception_type_port
feat(rust): port exception_type to litellm-core-utils
2026-09-18 14:17:17 -07:00
yassin
3449ae9d0d fix(proxy): advance the daily global spend marker in one conditional upsert so overlapping runs cannot rewind it
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 21:15:32 +00:00
yassin
f1b9642c41 fix(proxy): classify Azure Speech short audio behind a prefixed api base
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 21:15:01 +00:00
Mateo Wang
59c24abcbe
Merge pull request #33101 from BerriAI/litellm_fix_responses_ws_litellm_params_leak
fix(responses): stop managed Responses WebSocket from leaking litellm_params into provider request body
2026-09-18 14:11:19 -07:00
yassin
0897b663c5 Merge remote-tracking branch 'origin/litellm_deepgram_listen_websocket_passthrough' into litellm_deepgram_listen_websocket_passthrough 2026-09-18 21:08:45 +00:00
yassin
93d61abfa5 fix(deepgram): refuse /listen sessions that have no streaming price
A caller could pick a model with only a pre-recorded registry row, or no row at all, and the session would be billed at the pre-recorded rate or logged at zero cost, so budgets did not apply. The route now closes the WebSocket with 1008 before dialing Deepgram unless deepgram/streaming/<model> (or the -multilingual row for language=multi) is an exact registry hit, and the logging handler applies the same check so a registry change under a live session records the duration with no cost instead of a substitute rate

Regression tests cover the route refusal, an operator-supplied streaming row for another model being accepted, and the handler never substituting the pre-recorded rate

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 21:08:14 +00:00
Yassin Kortam
6759f28e73
Merge pull request #41557 from BerriAI/litellm_azure_speech_passthrough
feat(proxy): add Azure AI Speech pass-through route
2026-09-18 14:05:55 -07:00
ryan-crabbe-berri
006080ea6d
Merge pull request #41349 from BerriAI/litellm_team_member_spend_without_budget
fix(proxy): track team member spend when the member has no budget
2026-09-18 14:04:39 -07:00
Yujong Lee
9d134413c9 test(rust): rename the standalone 429 test to match the rule 2026-09-18 14:01:09 -07:00
Yujong Lee
2806d2546c Merge remote-tracking branch 'origin/main' into litellm_rust_exception_type_port 2026-09-18 21:00:11 +00:00
Yujong Lee
836bf7d897 refactor(rust): make the exception mapper a pure rule table
Rebuild exception_type around text rules per provider family and one shared
status table. The mapper takes the context, an injected redactor and the
original failure, and returns a PublicError with the message, the real
upstream response and the debug text. Every divergence from the Python
mapper and every known gap is listed in the module header

Match Python on a standalone 429 with an unknown status and on Cohere's
rules for failures without a status. Drop python_repr and the unread
public_failures fixtures
2026-09-18 13:59:36 -07:00
Mateo Wang
c553bc92bd
Merge pull request #41875 from BerriAI/litellm_passthrough_stream_timeout
fix(router): honor stream_timeout on the SDK-native passthrough route (/v1/messages, /converse)
2026-09-18 13:59:02 -07:00
Mateo Wang
20e1e6f2a9
Merge pull request #41384 from BerriAI/litellm_fix_azure_vector_store_search_url
fix(azure): keep api-version query after vector store search path
2026-09-18 13:56:05 -07:00
yassin
abf530fbeb fix(proxy): never rewind the daily global spend marker from an overlapping reconcile run
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 20:55:11 +00:00
ryan-crabbe-berri
c21e86a443 feat(ui): reset a team member's spend from the Members tab
Every member now carries a membership row, so a member who spent with no budget is over budget the moment a member budget is added later. The only fix was POST /team/{team_id}/member/{user_id}/reset_spend, which had no UI.

The Members tab gets a Reset spend action on rows that have current cycle spend. It confirms in a dialog, posts reset_to 0 through the typed client, and refreshes the team without remounting the page so the tab stays open. A team admin does not see it on their own row because the backend rejects that reset
2026-09-18 13:54:28 -07:00
ryan
61d4c5b9b5 fix(proxy): skip members already on the team before resolving a per-member budget
A mixed /team/member_add list that names an existing member used to run
add_new_member for them, which created or cloned a budget that the empty
upsert update branch never linked to their membership row. Filter the
requested members against the freshly locked roster first so budgets and
membership rows are only written for members who are actually new

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 13:54:17 -07:00
ryan
b6f4ad190e refactor(proxy): freeze the member spend arrays and budget link to stay within the type discipline budget
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 13:54:17 -07:00
ryan
2d61fa66b1 fix(proxy): lock teams in sorted team id order and keep existing member budgets on re-add
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 13:54:17 -07:00
ryan
499c334fce fix(proxy): lock each team in sorted order before the member spend upsert
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 13:54:17 -07:00
ryan
60077e90aa fix(proxy): take the team advisory lock before the member spend flush
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 13:54:17 -07:00
ryan
efef6ab684 fix(proxy): write team member spend as one roster checked upsert statement
Replaces the per team advisory lock and Pydantic roster parse in the spend flush
with a single INSERT ... ON CONFLICT statement that checks the stored roster in
SQL, so malformed roster JSON cannot fail the whole flush and large batches no
longer issue two queries per team inside the fixed transaction deadline

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 13:54:16 -07:00
ryan
7c068a4cf7 fix(repositories): import LiteralString from typing_extensions for python 3.10
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 13:54:16 -07:00
ryan
6dce85c728 fix(proxy): skip recreating membership rows for members removed before a spend flush
Take the team advisory lock in the spend flush transaction and read the roster
through it, so a delayed flush after /team/member_delete cannot recreate the
deleted LiteLLM_TeamMembership row. TEAM_ADVISORY_LOCK_SQL moves to
team_repository so the spend writer can import it without a circular import

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 13:54:16 -07:00
ryan
d9b48ac941 test(proxy): mock team membership upsert in team admin member add test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 13:54:16 -07:00
ryan
de4b520153 fix(proxy): track team member spend when the member has no budget
add_new_member only wrote a LiteLLM_TeamMembership row when a budget id
resolved, and the spend writer used update_many so a missing row failed
silently. Members without a budget therefore never accrued per-member spend.

The membership row is now always upserted (budget_id NULL when no budget
applies) and the spend write is an upsert so members added before this fix
start accruing on their next request.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 13:54:16 -07:00
yassin
b7d808e416 Merge remote-tracking branch 'origin/main' into litellm_deepgram_listen_websocket_passthrough
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 20:52:42 +00:00
kerry-berri
6fa34a299b
Merge pull request #41754 from BerriAI/litellm_qwen3_8_omni_flash
feat(models): add qwen3.8 flash rows, fix Cohere embed v3 context, Bedrock Mantle and OpenRouter pricing
2026-09-18 13:49:35 -07:00
Yassin Kortam
6e750e3d24
Merge pull request #41725 from BerriAI/litellm_mcp_admin_terminate_sessions_revoke_credentials
feat(mcp): let proxy admins force-close live MCP sessions and revoke stored user credentials
2026-09-18 13:44:37 -07:00
Mateo Wang
194e647e64
Merge pull request #41865 from BerriAI/litellm_vscode_extension
feat(vscode): add LiteLLM language model provider extension
2026-09-18 13:40:35 -07:00
mateo-berri
e75ad61fce fix(router): rank litellm_settings.request_timeout on the passthrough route like the completion route 2026-09-18 13:32:43 -07:00
yassin
57fd311f3a Merge remote-tracking branch 'origin/main' into litellm_azure_speech_passthrough 2026-09-18 20:32:32 +00:00
yassin
737c6e1b2d Merge remote-tracking branch 'origin/main' into litellm_mcp_admin_terminate_sessions_revoke_credentials
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

# Conflicts:
#	tests/test_litellm/proxy/config_resolvers/test_settings_store.py
2026-09-18 20:32:31 +00:00
mateo-berri
c984c329fe Merge remote-tracking branch 'origin/main' into litellm_fix_responses_ws_litellm_params_leak 2026-09-18 13:32:23 -07:00
yassin
1f3bee2e10 Merge remote-tracking branch 'origin/main' into litellm_deepgram_listen_websocket_passthrough
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

# Conflicts:
#	uv.lock
2026-09-18 20:32:05 +00:00
yucheng-berri
176ac4cc2b
Merge pull request #41862 from BerriAI/litellm_settings_store_clear_hang
* fix(proxy): make SettingsStore.clear() terminate when the config file owns a key

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(proxy): compare the cleared settings store against an unmutated refill mapping

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): keep the resolved values of config-owned keys across SettingsStore.clear()

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 13:29:53 -07:00
mateo-berri
eeca8b682b fix(vscode): raise the VS Code minimum to 1.115 for per-model configuration 2026-09-18 13:28:28 -07:00
Yujong Lee
e036e256ef revert: drop redundant anyio lockfile bump
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 20:23:53 +00:00
Yujong Lee
2703fab3c8 fix(deps): update anyio for osv scan
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 20:21:21 +00:00