Commit graph

51748 commits

Author SHA1 Message Date
yassin
da603c629b fix(ui): surface a malformed stored MCP allowlist as deny-all and let Save replace or remove it
Some checks failed
LiteLLM Rust / rust-lint (push) Has been cancelled
LiteLLM Rust / rust-test (push) Has been cancelled
LiteLLM Rust / rust-wheel (push) Has been cancelled
Terraform Modules / fmt, validate, test (aws) (push) Has been cancelled
Terraform Modules / fmt, validate, test (gcp) (push) Has been cancelled
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 22:37:30 +00:00
mateo-berri
7f9db61528 fix(vertex_ai): carry turns across stream rotation and route by model info
Rotating the Speech-to-Text stream at 240 s no longer ends the active turn:
the turn and its billed seconds continue on the new stream, forced at 280 s.
Bound the request and event queues (64 and 256) so a slow peer applies
backpressure instead of growing memory. Route a model to the Chirp realtime
path from its cost-map entry (mode audio_transcription plus /v1/realtime)
instead of a hardcoded name. Return on every branch of the recv and
transform helpers (CodeQL mixed returns), have the shared protocol helper take
the provider's error class so the Meta tests assert MuseProtocolError again,
and pin google-cloud-speech in the ci group so unit shards import it.
2026-09-18 15:35:59 -07:00
Devin AI
7dede188f8 test(timing): type the logging object test helper
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
2026-09-18 22:30:12 +00:00
yassin
2231a3ca43 feat(mcp): give each allowed MCP client an alias and a value
mcp_allowed_clients entries become {alias, value} objects: the value is what the JWT claim or header must equal, the alias is the name the dashboard and logs show. The Network Settings section is renamed Allowed Clients with one alias/value row per client

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 22:29:17 +00:00
mateo-berri
1adbfbfbb1 fix: strip eager_input_streaming for non-Claude providers next to input_examples 2026-09-18 15:28:45 -07:00
ryan-crabbe-berri
bc6b540205
Merge pull request #41888 from BerriAI/litellm_mcp_servers_my_connections_link
feat(ui): link MCP Servers page to the user's connected MCP servers
2026-09-18 15:27:05 -07:00
Yassin Kortam
b0887b63a5
Merge pull request #41838 from BerriAI/litellm_fix_tpm_window_reset_sibling_counters
fix(proxy): reset sibling tpm/rpm counters when the shared rate limit window rolls over
2026-09-18 15:23:05 -07:00
Yassin Kortam
5f83d97669
Merge pull request #41483 from BerriAI/litellm_v1_models_alias_metadata
fix(proxy): resolve model_group_alias to its target for /v1/models metadata
2026-09-18 15:22:02 -07:00
mateo-berri
d057e82e64 test(proxy): assert stored login throttle limits never outrank the config file 2026-09-18 15:13:51 -07:00
joshua
545bbeb001 test(mcp): update MCP suites for SDK 2 APIs
Rename McpError/isError/inputSchema-style references to the SDK 2
spellings, parse the JSONRPCMessage union with a TypeAdapter, and drive
the SDK transports off httpx2 MockTransport injection where respx can no
longer intercept. Adjust for SDK 2 behavior: the initialize handshake
negotiates handshake-era protocol versions only, an empty SSE stream
surfaces CONNECTION_CLOSED, non-2xx tool responses surface INTERNAL_ERROR
MCPError instead of HTTPStatusError, and the SDK read timeout carries the
JSON-RPC REQUEST_TIMEOUT code.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 22:13:18 +00:00
mateo-berri
417a88daed fix(responses): carry dict-valued reasoning_effort and keep the frame type on websocket defaults
A deployment whose reasoning_effort is an object is copied through as
reasoning the way the HTTP mapper does it instead of being dropped, and
the relay re-asserts the response.create frame type after merging
extra_body so a type key inside it can never replace it. The lazy
OpenAPI snapshot goes back to main: the earlier regeneration came from a
Python 3.14 interpreter dedenting docstrings, which CI on 3.12 rejects
2026-09-18 15:13:07 -07:00
joshua
5dc01319d7 refactor(mcp): port MCP client and server helpers to MCP SDK 2
McpError -> MCPError (new code/message/data constructor), camelCase model
attributes and constructor kwargs -> snake_case, RequestResponder ->
ClientSession message handler receiving ServerNotification | Exception,
RequestContext -> ClientRequestContext, read_timeout_seconds -> float,
server_capabilities property, JSONRPCMessage union parsed via TypeAdapter,
and httpx -> httpx2 for every object handed to the SDK transports
(MCPSigV4Auth, the httpx client factory, outbound_credentials auth
classes and resolver return types). Helpers that serve both litellm httpx
clients and the SDK's httpx2 transport accept both response types.

The SDK read-timeout code is now the JSON-RPC REQUEST_TIMEOUT (-32001)
instead of HTTP 408; as_mcp_read_timeout keeps the TimeoutError context
discriminator. Upstream transport exceptions and responses found in
exception trees are matched as httpx2 alongside httpx.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 22:13:04 +00:00
joshua
4bc3f1d0fc build(deps): migrate MCP integration to MCP SDK 2.2.0
Replace the bespoke dependency-install CI gate with a real migration:
require mcp>=2.2.0,<3 alongside httpx2>=2.5.0,<3 and pydantic>=2.12.0,<3
in the proxy and mcp extras, drop langchain-mcp-adapters (pins mcp<2)
from the dev group, and remove the dependency-install workflow and
tests/mcp_dependency_tests that only exercised the old pins.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 22:12:40 +00:00
ryan-crabbe-berri
1d91fc232d
Merge pull request #41883 from BerriAI/litellm_duplicate_check_sandbox_network
ci(duplicate-check): let Codex reach GitHub from its sandbox
2026-09-18 15:12:20 -07:00
joshua
c9c4f21b55 fix(ui): let MCP Servers header actions wrap on narrow screens
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 22:10:43 +00:00
Devin AI
00ab2c1be3 fix(timing): anchor response duration and overhead at proxy receive time
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
2026-09-18 22:07:43 +00:00
Devin AI
665327249d fix(bedrock): extend maxTokens minimum clamp to xAI Grok models on Converse
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 22:06:09 +00:00
Mateo Wang
7283293d83
Merge pull request #41138 from BerriAI/litellm_bedrock_files_s3_endpoint_url
fix(bedrock): carry s3_endpoint_url and s3_region_name into file content downloads
2026-09-18 15:04:01 -07:00
joshua
19ef8e47a6 feat(ui): link MCP Servers page to the user's connected MCP servers
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 22:01:56 +00:00
yucheng-berri
84df4c0d1b
Merge pull request #41783 from BerriAI/litellm_rate_limit_fallback_guardrails
fix(proxy): keep requested model guardrails and key disable_fallbacks on rate-limit fallback
2026-09-18 14:59:16 -07:00
mateo-berri
55249c7128 fix: set vertex gemma-4-26b-a4b-it-maas context window to 262144 2026-09-18 14:58:58 -07:00
mateo-berri
3edbf60e9c fix(proxy): requeue the daily tag rollup on commit failure without the Redis buffer 2026-09-18 14:58:34 -07:00
mateo-berri
76d1abba72 refactor(responses): map status codes to error codes with a lookup
Ends _response_error_code in an unconditional return so CodeQL stops flagging mixed explicit and implicit returns. No behavior change: every status maps as before.
2026-09-18 14:57:15 -07:00
mateo-berri
cf05466a27 fix(gemini): map every documented finishReason and reset per-candidate state
A content-less candidate is now kept as a choice whenever it carries a
finishReason, with the raw value on the choice's provider_specific_fields.
NO_IMAGE, IMAGE_RECITATION, IMAGE_OTHER and ESCALATION map to content_filter;
UNEXPECTED_TOOL_CALL and MISSING_THOUGHT_SIGNATURE map to stop. The
/v1/responses bridge reports content_filter and refusal as incomplete with
incomplete_details, and tool calls and reasoning no longer leak from one
candidate into the next.
2026-09-18 14:54:18 -07:00
Yassin Kortam
87694c26ef
Merge pull request #41324 from BerriAI/litellm_daily_global_spend_table
feat(proxy): add LiteLLM_DailyGlobalSpend key-free rollup for the usage dashboard
2026-09-18 14:53:08 -07:00
mateo-berri
e0a74dabd1 Merge remote-tracking branch 'origin/main' into litellm_config_update_rejects_config_owned_keys 2026-09-18 14:51:11 -07:00
Yassin Kortam
47209d37f2
Merge pull request #41882 from BerriAI/litellm_azure_speech_api_base_prefix
fix(proxy): classify Azure Speech short audio behind a prefixed api base
2026-09-18 14:50:28 -07:00
yassin
cca7ab8b1b test(mcp): type the REST allowlist test stubs
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 21:50:16 +00:00
yujonglee
59604b2b19
Merge pull request #41884 from BerriAI/litellm_ocr_test_matrix
test(ocr): declarative provider x auth x input matrix for tests/ocr_tests
2026-09-18 14:50:05 -07:00
Yassin Kortam
52d6aab421
Merge pull request #41554 from BerriAI/litellm_deepgram_listen_websocket_passthrough
feat(passthrough): deepgram streaming /v1/listen WebSocket passthrough with duration-based cost tracking
2026-09-18 14:48:37 -07:00
yucheng
2a7dcc77b2 test(team): mock the membership upsert the member add now issues on team create
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 21:47:32 +00:00
yucheng
0e74dd2811 test(team): drop the docstrings from the roster audit event tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 21:46:33 +00:00
yucheng
5e8247a1c0 fix(team): emit audit events for member_delete and role changes and carry the final roster on team create
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 21:46:33 +00:00
Mateo Wang
d45e04a9fd
Merge pull request #41062 from BerriAI/litellm_mistral_codex_reasoning_effort_client_metadata
fix(mistral): accept reasoning_effort on all models and drop client_metadata for Codex compatibility
2026-09-18 14:43:58 -07:00
Yujong Lee
f72b7155ac fix(ocr): map Rust upstream 401/403 to the public auth exceptions
The httpx.Response built for a Rust upstream failure had no request attached,
so constructing openai.AuthenticationError raised RuntimeError inside the
exception mapper and every bad-key OCR call surfaced as APIConnectionError 500
instead of AuthenticationError 401 (the Python path already returned 401)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 21:40:13 +00:00
Devin AI
16d63eaf88 Merge remote-tracking branch 'origin/main' into litellm_fix_tpm_window_reset_sibling_counters 2026-09-18 21:38:26 +00:00
ryan-crabbe-berri
595768b54b fix(proxy): narrow project_id without a cast and fold the unbudgeted cases into the budget matrix test
Some checks failed
LiteLLM Rust / rust-lint (push) Has been cancelled
LiteLLM Rust / rust-test (push) Has been cancelled
LiteLLM Rust / rust-wheel (push) Has been cancelled
Terraform Modules / fmt, validate, test (aws) (push) Has been cancelled
Terraform Modules / fmt, validate, test (gcp) (push) Has been cancelled
The lint job failed on one new typing.cast (LIT006) in the cost callback, and the test-quality gate behind it would have failed next on a test whose only assertion inspected a mock (TQ002). project_id is now narrowed with isinstance, and the zero and negative max_budget cases run through the existing parametrized budget test, which asserts the raised error or a clean admit with no alert
2026-09-18 14:33:41 -07:00
jesus
8983eefea5 fix(auth): drop redundant cast on team_object in centralized checks
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 21:30:32 +00:00
Mateo Wang
a6bd779bd1
Merge pull request #39424 from emerzon/litellm_azure_ai_flux_2_flex
feat(azure_ai): support FLUX.2 flex images
2026-09-18 14:30:25 -07:00
yassin
0536fb3062 fix(mcp): fail closed on empty JWT claims and gate the REST tool routes on mcp_allowed_clients
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 21:29:59 +00:00
yassin
0b5b69ea3a fix(deepgram): forward only the first model and language values to /listen
Authorization and pricing read the first model and language query value, but the raw query was forwarded, so Deepgram (which honours the last repeated value) could be sent a model the key was never allowed. Later duplicates of those two keys are now dropped before the upstream URL is built

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 21:29:29 +00:00
Yujong Lee
9767878425 test(ocr): replace per-provider OCR test classes with a declarative provider x auth x input matrix
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-18 21:27:03 +00:00
ryan-crabbe-berri
a5f6ce7bb1 Merge remote-tracking branch 'origin/main' into litellm_lit_3269_project_spend_tracking
# Conflicts:
#	tests/test_litellm/proxy/db/test_db_spend_update_writer.py
2026-09-18 14:22:07 -07:00
ryan-crabbe-berri
1e8b8f7c33 ci(duplicate-check): describe the workspace-write sandbox accurately 2026-09-18 14:21:22 -07:00
ryan-crabbe-berri
a43a4924a6
Merge pull request #40878 from BerriAI/litellm_null_cost_unpriced_deployments
fix(router): report null cost for unpriced deployments instead of 0
2026-09-18 14:20:52 -07:00
ryan-crabbe-berri
2332d3f183
Merge pull request #40737 from BerriAI/litellm_logs_user_email_display
fix(ui): show internal user email in logs table and log detail drawer
2026-09-18 14:20:16 -07:00
yujonglee
b0b2f13548
Merge pull request #41873 from BerriAI/litellm_rust_exception_type_port
feat(rust): port exception_type to litellm-core-utils
2026-09-18 14:17:17 -07:00
ryan-crabbe-berri
1f27c442b4 ci(duplicate-check): let Codex reach GitHub from its sandbox
Every gh search in the first real runs failed with "error connecting to
api.github.com", so the verdict was always null. The legacy
sandbox_permissions key no longer grants network in read-only mode; the
workspace-write sandbox has a network_access switch that does. Pin the CLI
to the version the prompt was proven on
2026-09-18 14:17:00 -07:00
mateo-berri
eb96d885ce fix(proxy): end failed responses streams with [DONE]
Emit data: [DONE] after event: response.failed, and after a late failure
when a terminal event already went out, so OpenAI SDK clients see the
same stream end as a completed response. Restore the lazy OpenAPI
snapshot to its Python 3.12 rendering, which is what CI regenerates.
2026-09-18 14:16:47 -07:00
mateo-berri
4a951847bb fix(responses): merge deployment litellm_params into native websocket response.create frames 2026-09-18 14:15:39 -07:00