Commit graph

18968 commits

Author SHA1 Message Date
Devin AI
7dede188f8 test(timing): type the logging object test helper
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
2026-09-18 22:30:12 +00:00
yassin
2231a3ca43 feat(mcp): give each allowed MCP client an alias and a value
mcp_allowed_clients entries become {alias, value} objects: the value is what the JWT claim or header must equal, the alias is the name the dashboard and logs show. The Network Settings section is renamed Allowed Clients with one alias/value row per client

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 22:29:17 +00:00
mateo-berri
1adbfbfbb1 fix: strip eager_input_streaming for non-Claude providers next to input_examples 2026-09-18 15:28:45 -07:00
Yassin Kortam
b0887b63a5
Merge pull request #41838 from BerriAI/litellm_fix_tpm_window_reset_sibling_counters
fix(proxy): reset sibling tpm/rpm counters when the shared rate limit window rolls over
2026-09-18 15:23:05 -07:00
Yassin Kortam
5f83d97669
Merge pull request #41483 from BerriAI/litellm_v1_models_alias_metadata
fix(proxy): resolve model_group_alias to its target for /v1/models metadata
2026-09-18 15:22:02 -07:00
mateo-berri
d057e82e64 test(proxy): assert stored login throttle limits never outrank the config file 2026-09-18 15:13:51 -07:00
joshua
545bbeb001 test(mcp): update MCP suites for SDK 2 APIs
Rename McpError/isError/inputSchema-style references to the SDK 2
spellings, parse the JSONRPCMessage union with a TypeAdapter, and drive
the SDK transports off httpx2 MockTransport injection where respx can no
longer intercept. Adjust for SDK 2 behavior: the initialize handshake
negotiates handshake-era protocol versions only, an empty SSE stream
surfaces CONNECTION_CLOSED, non-2xx tool responses surface INTERNAL_ERROR
MCPError instead of HTTPStatusError, and the SDK read timeout carries the
JSON-RPC REQUEST_TIMEOUT code.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 22:13:18 +00:00
mateo-berri
417a88daed fix(responses): carry dict-valued reasoning_effort and keep the frame type on websocket defaults
A deployment whose reasoning_effort is an object is copied through as
reasoning the way the HTTP mapper does it instead of being dropped, and
the relay re-asserts the response.create frame type after merging
extra_body so a type key inside it can never replace it. The lazy
OpenAPI snapshot goes back to main: the earlier regeneration came from a
Python 3.14 interpreter dedenting docstrings, which CI on 3.12 rejects
2026-09-18 15:13:07 -07:00
joshua
4bc3f1d0fc build(deps): migrate MCP integration to MCP SDK 2.2.0
Replace the bespoke dependency-install CI gate with a real migration:
require mcp>=2.2.0,<3 alongside httpx2>=2.5.0,<3 and pydantic>=2.12.0,<3
in the proxy and mcp extras, drop langchain-mcp-adapters (pins mcp<2)
from the dev group, and remove the dependency-install workflow and
tests/mcp_dependency_tests that only exercised the old pins.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 22:12:40 +00:00
Moe Khalil
8e5f43f458 fix(auto-router): preserve JEV accounting and context bounds
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 22:09:27 +00:00
Devin AI
00ab2c1be3 fix(timing): anchor response duration and overhead at proxy receive time
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
2026-09-18 22:07:43 +00:00
Devin AI
665327249d fix(bedrock): extend maxTokens minimum clamp to xAI Grok models on Converse
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 22:06:09 +00:00
Mateo Wang
7283293d83
Merge pull request #41138 from BerriAI/litellm_bedrock_files_s3_endpoint_url
fix(bedrock): carry s3_endpoint_url and s3_region_name into file content downloads
2026-09-18 15:04:01 -07:00
Moe Khalil
b9e5bb3abb test(proxy): allow JEV dependency in budget fixtures
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 22:03:37 +00:00
yucheng-berri
84df4c0d1b
Merge pull request #41783 from BerriAI/litellm_rate_limit_fallback_guardrails
fix(proxy): keep requested model guardrails and key disable_fallbacks on rate-limit fallback
2026-09-18 14:59:16 -07:00
mateo-berri
55249c7128 fix: set vertex gemma-4-26b-a4b-it-maas context window to 262144 2026-09-18 14:58:58 -07:00
mateo-berri
3edbf60e9c fix(proxy): requeue the daily tag rollup on commit failure without the Redis buffer 2026-09-18 14:58:34 -07:00
mateo-berri
cf05466a27 fix(gemini): map every documented finishReason and reset per-candidate state
A content-less candidate is now kept as a choice whenever it carries a
finishReason, with the raw value on the choice's provider_specific_fields.
NO_IMAGE, IMAGE_RECITATION, IMAGE_OTHER and ESCALATION map to content_filter;
UNEXPECTED_TOOL_CALL and MISSING_THOUGHT_SIGNATURE map to stop. The
/v1/responses bridge reports content_filter and refusal as incomplete with
incomplete_details, and tool calls and reasoning no longer leak from one
candidate into the next.
2026-09-18 14:54:18 -07:00
Yassin Kortam
87694c26ef
Merge pull request #41324 from BerriAI/litellm_daily_global_spend_table
feat(proxy): add LiteLLM_DailyGlobalSpend key-free rollup for the usage dashboard
2026-09-18 14:53:08 -07:00
Moe Khalil
7c493ff3b9 test(auto-router): reconcile JEV integration checks
Co-authored-by: Moe Khalil <moe@berri.ai>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 21:52:24 +00:00
mateo-berri
e0a74dabd1 Merge remote-tracking branch 'origin/main' into litellm_config_update_rejects_config_owned_keys 2026-09-18 14:51:11 -07:00
Yassin Kortam
47209d37f2
Merge pull request #41882 from BerriAI/litellm_azure_speech_api_base_prefix
fix(proxy): classify Azure Speech short audio behind a prefixed api base
2026-09-18 14:50:28 -07:00
yassin
cca7ab8b1b test(mcp): type the REST allowlist test stubs
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 21:50:16 +00:00
yujonglee
59604b2b19
Merge pull request #41884 from BerriAI/litellm_ocr_test_matrix
test(ocr): declarative provider x auth x input matrix for tests/ocr_tests
2026-09-18 14:50:05 -07:00
Yassin Kortam
52d6aab421
Merge pull request #41554 from BerriAI/litellm_deepgram_listen_websocket_passthrough
feat(passthrough): deepgram streaming /v1/listen WebSocket passthrough with duration-based cost tracking
2026-09-18 14:48:37 -07:00
yucheng
2a7dcc77b2 test(team): mock the membership upsert the member add now issues on team create
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 21:47:32 +00:00
yucheng
0e74dd2811 test(team): drop the docstrings from the roster audit event tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 21:46:33 +00:00
yucheng
5e8247a1c0 fix(team): emit audit events for member_delete and role changes and carry the final roster on team create
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 21:46:33 +00:00
Mateo Wang
d45e04a9fd
Merge pull request #41062 from BerriAI/litellm_mistral_codex_reasoning_effort_client_metadata
fix(mistral): accept reasoning_effort on all models and drop client_metadata for Codex compatibility
2026-09-18 14:43:58 -07:00
Yujong Lee
f72b7155ac fix(ocr): map Rust upstream 401/403 to the public auth exceptions
The httpx.Response built for a Rust upstream failure had no request attached,
so constructing openai.AuthenticationError raised RuntimeError inside the
exception mapper and every bad-key OCR call surfaced as APIConnectionError 500
instead of AuthenticationError 401 (the Python path already returned 401)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 21:40:13 +00:00
Devin AI
16d63eaf88 Merge remote-tracking branch 'origin/main' into litellm_fix_tpm_window_reset_sibling_counters 2026-09-18 21:38:26 +00:00
Moe Khalil
86e079d7a8 feat(auto-router): integrate JEV context and usage accounting
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 21:38:15 +00:00
ryan-crabbe-berri
595768b54b fix(proxy): narrow project_id without a cast and fold the unbudgeted cases into the budget matrix test
Some checks failed
LiteLLM Rust / rust-lint (push) Has been cancelled
LiteLLM Rust / rust-test (push) Has been cancelled
LiteLLM Rust / rust-wheel (push) Has been cancelled
Terraform Modules / fmt, validate, test (aws) (push) Has been cancelled
Terraform Modules / fmt, validate, test (gcp) (push) Has been cancelled
The lint job failed on one new typing.cast (LIT006) in the cost callback, and the test-quality gate behind it would have failed next on a test whose only assertion inspected a mock (TQ002). project_id is now narrowed with isinstance, and the zero and negative max_budget cases run through the existing parametrized budget test, which asserts the raised error or a clean admit with no alert
2026-09-18 14:33:41 -07:00
Mateo Wang
a6bd779bd1
Merge pull request #39424 from emerzon/litellm_azure_ai_flux_2_flex
feat(azure_ai): support FLUX.2 flex images
2026-09-18 14:30:25 -07:00
yassin
0536fb3062 fix(mcp): fail closed on empty JWT claims and gate the REST tool routes on mcp_allowed_clients
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 21:29:59 +00:00
yassin
0b5b69ea3a fix(deepgram): forward only the first model and language values to /listen
Authorization and pricing read the first model and language query value, but the raw query was forwarded, so Deepgram (which honours the last repeated value) could be sent a model the key was never allowed. Later duplicates of those two keys are now dropped before the upstream URL is built

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 21:29:29 +00:00
Yujong Lee
9767878425 test(ocr): replace per-provider OCR test classes with a declarative provider x auth x input matrix
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 21:27:03 +00:00
ryan-crabbe-berri
a5f6ce7bb1 Merge remote-tracking branch 'origin/main' into litellm_lit_3269_project_spend_tracking
# Conflicts:
#	tests/test_litellm/proxy/db/test_db_spend_update_writer.py
2026-09-18 14:22:07 -07:00
ryan-crabbe-berri
a43a4924a6
Merge pull request #40878 from BerriAI/litellm_null_cost_unpriced_deployments
fix(router): report null cost for unpriced deployments instead of 0
2026-09-18 14:20:52 -07:00
mateo-berri
eb96d885ce fix(proxy): end failed responses streams with [DONE]
Emit data: [DONE] after event: response.failed, and after a late failure
when a terminal event already went out, so OpenAI SDK clients see the
same stream end as a completed response. Restore the lazy OpenAPI
snapshot to its Python 3.12 rendering, which is what CI regenerates.
2026-09-18 14:16:47 -07:00
mateo-berri
4a951847bb fix(responses): merge deployment litellm_params into native websocket response.create frames 2026-09-18 14:15:39 -07:00
yassin
3449ae9d0d fix(proxy): advance the daily global spend marker in one conditional upsert so overlapping runs cannot rewind it
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 21:15:32 +00:00
yassin
f1b9642c41 fix(proxy): classify Azure Speech short audio behind a prefixed api base
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 21:15:01 +00:00
Mateo Wang
59c24abcbe
Merge pull request #33101 from BerriAI/litellm_fix_responses_ws_litellm_params_leak
fix(responses): stop managed Responses WebSocket from leaking litellm_params into provider request body
2026-09-18 14:11:19 -07:00
yassin
0897b663c5 Merge remote-tracking branch 'origin/litellm_deepgram_listen_websocket_passthrough' into litellm_deepgram_listen_websocket_passthrough 2026-09-18 21:08:45 +00:00
yassin
93d61abfa5 fix(deepgram): refuse /listen sessions that have no streaming price
A caller could pick a model with only a pre-recorded registry row, or no row at all, and the session would be billed at the pre-recorded rate or logged at zero cost, so budgets did not apply. The route now closes the WebSocket with 1008 before dialing Deepgram unless deepgram/streaming/<model> (or the -multilingual row for language=multi) is an exact registry hit, and the logging handler applies the same check so a registry change under a live session records the duration with no cost instead of a substitute rate

Regression tests cover the route refusal, an operator-supplied streaming row for another model being accepted, and the handler never substituting the pre-recorded rate

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 21:08:14 +00:00
Yassin Kortam
6759f28e73
Merge pull request #41557 from BerriAI/litellm_azure_speech_passthrough
feat(proxy): add Azure AI Speech pass-through route
2026-09-18 14:05:55 -07:00
mateo-berri
9d39d7efaa chore: merge main into fix/gemini-candidate-finish-reason-no-content 2026-09-18 14:05:25 -07:00
ryan-crabbe-berri
006080ea6d
Merge pull request #41349 from BerriAI/litellm_team_member_spend_without_budget
fix(proxy): track team member spend when the member has no budget
2026-09-18 14:04:39 -07:00
Moe Khalil
0ac0362b42 fix(proxy): enforce virtual key budgets for JEV test routing
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 21:01:16 +00:00