Commit graph

50982 commits

Author SHA1 Message Date
Yujong Lee
41873e2bc7 fix(rust): box messages response output
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 15:45:08 -07:00
Yujong Lee
b7686d78b6 refactor(rust): move RouteMachine out of core into litellm-host next to the Machine trait
core/src/machine was route-neutral runtime code sitting among route surfaces, and the workspace had two modules named machine. It now lives in litellm-host beside the contract it implements, so core only holds routes. The OCR conversion from MachineFault moves to litellm-llms because the orphan rule no longer allows it in core
2026-09-18 15:45:08 -07:00
Yujong Lee
19ffb584eb refactor(rust): rename litellm-callbacks to litellm-host and HostOpError to InvokeError 2026-09-18 15:45:08 -07:00
Yujong Lee
c2679757b3 refactor(rust): put stream billing on the legacy surface 2026-09-18 15:45:08 -07:00
Yujong Lee
3a5b7c12ef refactor(rust): separate machine events from the Python lifecycle's events 2026-09-18 15:45:08 -07:00
Yujong Lee
1a52bae779 add streaming message 2026-09-18 15:45:08 -07:00
Yujong Lee
4627ec4ea8 test(rust): cover post-call header identity
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 15:43:08 -07:00
Yujong Lee
dd2e6c17bc fix(rust): preserve OCR callback headers
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 15:43:08 -07:00
Yujong Lee
ab27ee7efc test(rust): fix request count and header case in new OCR callback tests 2026-09-18 15:43:08 -07:00
Yujong Lee
30f7b8442b chore(rust): lock proptest 2026-09-18 15:43:08 -07:00
Yujong Lee
a737e3430a test(rust): property and parametrized tests for legacy callback contracts
The payload boundary of callbacks-legacy gets a model-based proptest: for any
JSON body, caller keywords and callback edit, keywords the route sends unchanged
reach pre_call as the caller's own objects, and the wire is the body pre_call
received as the callback left it. A parametrized test pins that a keyword the
bridge never reads keeps its identity through setup, the deployment hook,
check_limits and prepare.

Behaviour owned by the real Logging object is pinned end to end in the OCR
tests: a hypothesis version of the body property over HTTP, sync hooks seeing no
running event loop, retained payloads staying intact after the call, success
callbacks sharing one standard logging payload, and state stashed before a
blocking deployment hook raises reaching both failure callback families
2026-09-18 15:43:08 -07:00
Yujong Lee
b4bfd92a2a refactor(rust): route-neutral callback contract
Every legacy callback call from callbacks-legacy now goes through one typed
Python shim, litellm.rust_bridge.legacy_callbacks, the only Python module
the crate reaches. Before, the crate called Logging methods, litellm.utils
hooks, the logging worker, the executor and several litellm globals
directly, and its tests retyped those signatures by hand, so an outdated
fake could accept a call the real code rejects. python_contract.json lists
each shim function's parameters: a Python test pins it to the real
signatures and a Rust test pins it to the Rust enum.

The lifecycle contract changes to match the Python @client wrapper:
- the driver emits CallEvent::Started before begin, so every host sees one
  start time
- RequestContext carries the route-resolved api_key, so legacy pre_call and
  post_call receive it, and post_call's additional_args match the Python OCR
  path
- Passthrough and its re-aliasing are gone
- async deployment hooks always run, and the "no callbacks" shortcut that
  skipped the logging payload is removed, as in the Python path

The OCR api_key is a SecretValue from the wire request onward, so Debug
output upstream of the callback contract cannot leak it.

host-python's RouteHost now classifies native failures once through
classify, and host ops return HostOpError. The OCR route host keeps main's
public errors by sending both through the existing Python map_failure.
2026-09-18 15:43:08 -07:00
ryan-crabbe-berri
bc6b540205
Merge pull request #41888 from BerriAI/litellm_mcp_servers_my_connections_link
feat(ui): link MCP Servers page to the user's connected MCP servers
2026-09-18 15:27:05 -07:00
Yassin Kortam
b0887b63a5
Merge pull request #41838 from BerriAI/litellm_fix_tpm_window_reset_sibling_counters
fix(proxy): reset sibling tpm/rpm counters when the shared rate limit window rolls over
2026-09-18 15:23:05 -07:00
Yassin Kortam
5f83d97669
Merge pull request #41483 from BerriAI/litellm_v1_models_alias_metadata
fix(proxy): resolve model_group_alias to its target for /v1/models metadata
2026-09-18 15:22:02 -07:00
ryan-crabbe-berri
1d91fc232d
Merge pull request #41883 from BerriAI/litellm_duplicate_check_sandbox_network
ci(duplicate-check): let Codex reach GitHub from its sandbox
2026-09-18 15:12:20 -07:00
joshua
c9c4f21b55 fix(ui): let MCP Servers header actions wrap on narrow screens
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 22:10:43 +00:00
Mateo Wang
7283293d83
Merge pull request #41138 from BerriAI/litellm_bedrock_files_s3_endpoint_url
fix(bedrock): carry s3_endpoint_url and s3_region_name into file content downloads
2026-09-18 15:04:01 -07:00
joshua
19ef8e47a6 feat(ui): link MCP Servers page to the user's connected MCP servers
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 22:01:56 +00:00
yucheng-berri
84df4c0d1b
Merge pull request #41783 from BerriAI/litellm_rate_limit_fallback_guardrails
fix(proxy): keep requested model guardrails and key disable_fallbacks on rate-limit fallback
2026-09-18 14:59:16 -07:00
Yassin Kortam
87694c26ef
Merge pull request #41324 from BerriAI/litellm_daily_global_spend_table
feat(proxy): add LiteLLM_DailyGlobalSpend key-free rollup for the usage dashboard
2026-09-18 14:53:08 -07:00
Yassin Kortam
47209d37f2
Merge pull request #41882 from BerriAI/litellm_azure_speech_api_base_prefix
fix(proxy): classify Azure Speech short audio behind a prefixed api base
2026-09-18 14:50:28 -07:00
yujonglee
59604b2b19
Merge pull request #41884 from BerriAI/litellm_ocr_test_matrix
test(ocr): declarative provider x auth x input matrix for tests/ocr_tests
2026-09-18 14:50:05 -07:00
Yassin Kortam
52d6aab421
Merge pull request #41554 from BerriAI/litellm_deepgram_listen_websocket_passthrough
feat(passthrough): deepgram streaming /v1/listen WebSocket passthrough with duration-based cost tracking
2026-09-18 14:48:37 -07:00
Mateo Wang
d45e04a9fd
Merge pull request #41062 from BerriAI/litellm_mistral_codex_reasoning_effort_client_metadata
fix(mistral): accept reasoning_effort on all models and drop client_metadata for Codex compatibility
2026-09-18 14:43:58 -07:00
Yujong Lee
f72b7155ac fix(ocr): map Rust upstream 401/403 to the public auth exceptions
The httpx.Response built for a Rust upstream failure had no request attached,
so constructing openai.AuthenticationError raised RuntimeError inside the
exception mapper and every bad-key OCR call surfaced as APIConnectionError 500
instead of AuthenticationError 401 (the Python path already returned 401)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 21:40:13 +00:00
Devin AI
16d63eaf88 Merge remote-tracking branch 'origin/main' into litellm_fix_tpm_window_reset_sibling_counters 2026-09-18 21:38:26 +00:00
Mateo Wang
a6bd779bd1
Merge pull request #39424 from emerzon/litellm_azure_ai_flux_2_flex
feat(azure_ai): support FLUX.2 flex images
2026-09-18 14:30:25 -07:00
yassin
0b5b69ea3a fix(deepgram): forward only the first model and language values to /listen
Authorization and pricing read the first model and language query value, but the raw query was forwarded, so Deepgram (which honours the last repeated value) could be sent a model the key was never allowed. Later duplicates of those two keys are now dropped before the upstream URL is built

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 21:29:29 +00:00
Yujong Lee
9767878425 test(ocr): replace per-provider OCR test classes with a declarative provider x auth x input matrix
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 21:27:03 +00:00
ryan-crabbe-berri
1e8b8f7c33 ci(duplicate-check): describe the workspace-write sandbox accurately 2026-09-18 14:21:22 -07:00
ryan-crabbe-berri
a43a4924a6
Merge pull request #40878 from BerriAI/litellm_null_cost_unpriced_deployments
fix(router): report null cost for unpriced deployments instead of 0
2026-09-18 14:20:52 -07:00
ryan-crabbe-berri
2332d3f183
Merge pull request #40737 from BerriAI/litellm_logs_user_email_display
fix(ui): show internal user email in logs table and log detail drawer
2026-09-18 14:20:16 -07:00
yujonglee
b0b2f13548
Merge pull request #41873 from BerriAI/litellm_rust_exception_type_port
feat(rust): port exception_type to litellm-core-utils
2026-09-18 14:17:17 -07:00
ryan-crabbe-berri
1f27c442b4 ci(duplicate-check): let Codex reach GitHub from its sandbox
Every gh search in the first real runs failed with "error connecting to
api.github.com", so the verdict was always null. The legacy
sandbox_permissions key no longer grants network in read-only mode; the
workspace-write sandbox has a network_access switch that does. Pin the CLI
to the version the prompt was proven on
2026-09-18 14:17:00 -07:00
yassin
3449ae9d0d fix(proxy): advance the daily global spend marker in one conditional upsert so overlapping runs cannot rewind it
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 21:15:32 +00:00
yassin
f1b9642c41 fix(proxy): classify Azure Speech short audio behind a prefixed api base
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 21:15:01 +00:00
Mateo Wang
59c24abcbe
Merge pull request #33101 from BerriAI/litellm_fix_responses_ws_litellm_params_leak
fix(responses): stop managed Responses WebSocket from leaking litellm_params into provider request body
2026-09-18 14:11:19 -07:00
yassin
0897b663c5 Merge remote-tracking branch 'origin/litellm_deepgram_listen_websocket_passthrough' into litellm_deepgram_listen_websocket_passthrough 2026-09-18 21:08:45 +00:00
yassin
93d61abfa5 fix(deepgram): refuse /listen sessions that have no streaming price
A caller could pick a model with only a pre-recorded registry row, or no row at all, and the session would be billed at the pre-recorded rate or logged at zero cost, so budgets did not apply. The route now closes the WebSocket with 1008 before dialing Deepgram unless deepgram/streaming/<model> (or the -multilingual row for language=multi) is an exact registry hit, and the logging handler applies the same check so a registry change under a live session records the duration with no cost instead of a substitute rate

Regression tests cover the route refusal, an operator-supplied streaming row for another model being accepted, and the handler never substituting the pre-recorded rate

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 21:08:14 +00:00
Yassin Kortam
6759f28e73
Merge pull request #41557 from BerriAI/litellm_azure_speech_passthrough
feat(proxy): add Azure AI Speech pass-through route
2026-09-18 14:05:55 -07:00
ryan-crabbe-berri
006080ea6d
Merge pull request #41349 from BerriAI/litellm_team_member_spend_without_budget
fix(proxy): track team member spend when the member has no budget
2026-09-18 14:04:39 -07:00
Yujong Lee
9d134413c9 test(rust): rename the standalone 429 test to match the rule 2026-09-18 14:01:09 -07:00
Yujong Lee
2806d2546c Merge remote-tracking branch 'origin/main' into litellm_rust_exception_type_port 2026-09-18 21:00:11 +00:00
Yujong Lee
836bf7d897 refactor(rust): make the exception mapper a pure rule table
Rebuild exception_type around text rules per provider family and one shared
status table. The mapper takes the context, an injected redactor and the
original failure, and returns a PublicError with the message, the real
upstream response and the debug text. Every divergence from the Python
mapper and every known gap is listed in the module header

Match Python on a standalone 429 with an unknown status and on Cohere's
rules for failures without a status. Drop python_repr and the unread
public_failures fixtures
2026-09-18 13:59:36 -07:00
Mateo Wang
c553bc92bd
Merge pull request #41875 from BerriAI/litellm_passthrough_stream_timeout
fix(router): honor stream_timeout on the SDK-native passthrough route (/v1/messages, /converse)
2026-09-18 13:59:02 -07:00
Mateo Wang
20e1e6f2a9
Merge pull request #41384 from BerriAI/litellm_fix_azure_vector_store_search_url
fix(azure): keep api-version query after vector store search path
2026-09-18 13:56:05 -07:00
yassin
abf530fbeb fix(proxy): never rewind the daily global spend marker from an overlapping reconcile run
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 20:55:11 +00:00
ryan-crabbe-berri
c21e86a443 feat(ui): reset a team member's spend from the Members tab
Every member now carries a membership row, so a member who spent with no budget is over budget the moment a member budget is added later. The only fix was POST /team/{team_id}/member/{user_id}/reset_spend, which had no UI.

The Members tab gets a Reset spend action on rows that have current cycle spend. It confirms in a dialog, posts reset_to 0 through the typed client, and refreshes the team without remounting the page so the tab stays open. A team admin does not see it on their own row because the backend rejects that reset
2026-09-18 13:54:28 -07:00
ryan
61d4c5b9b5 fix(proxy): skip members already on the team before resolving a per-member budget
A mixed /team/member_add list that names an existing member used to run
add_new_member for them, which created or cloned a budget that the empty
upsert update branch never linked to their membership row. Filter the
requested members against the freshly locked roster first so budgets and
membership rows are only written for members who are actually new

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 13:54:17 -07:00