Commit graph

48132 commits

Author SHA1 Message Date
joshua-berri
e11be9de21
Merge pull request #40525 from BerriAI/litellm_fix_mcp_optional_discovery
fix(mcp): respect optional discovery capabilities and quiet unsupported methods
2026-09-09 21:51:49 -07:00
joshua-berri
d44e52a3cb
Merge pull request #40453 from BerriAI/litellm_fix_mcp_auth_challenge_6635
fix(mcp): challenge and scope gateway-owned server authentication
2026-09-09 21:51:33 -07:00
joshua-berri
17e9b2329e
Merge pull request #40454 from BerriAI/litellm_fix_mcp_debug_auth_resolution_6892
fix(mcp): report resolved upstream authentication in debug headers
2026-09-09 21:51:22 -07:00
Joshua Valluru
95f6e96ef9 test(mcp): type auth diagnostics regression parameters 2026-09-09 20:57:33 -07:00
Mateo Wang
69245feff4
Merge pull request #39863 from BerriAI/litellm_lit_7022_azure_ai_passthrough_config
fix(azure_ai): add passthrough config so router-model relays reach the deployment's own endpoint
2026-09-09 20:56:57 -07:00
Mateo Wang
bf39643d39
Merge pull request #40497 from BerriAI/litellm_bump_smol_toml
chore(ui): bump smol-toml to fix GHSA-7w5x-hrqm-74c2 osv-scan failure
2026-09-09 20:44:42 -07:00
Joshua Valluru
a0bb021f02 fix(mcp): preserve gateway-owned resource scope through consent 2026-09-09 20:41:56 -07:00
Joshua Valluru
935611d25d fix(mcp): respect optional discovery capabilities and quiet unsupported methods 2026-09-09 20:21:19 -07:00
devin-ai-integration[bot]
87e2026f0a
fix(terraform): read key fields the proxy stores in metadata back from /key/info (#40513)
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-10 03:10:42 +00:00
devin-ai-integration[bot]
5264f8ed48
fix(terraform): drop litellm_key from state on 404 instead of failing the plan (#40443)
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-10 02:57:51 +00:00
devin-ai-integration[bot]
ea106fd8be
fix(terraform): send changed litellm_key duration on /key/update so expires is recomputed (#40511)
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-10 02:57:07 +00:00
devin-ai-integration[bot]
fb603fdb6b
fix(terraform): keep prior state when a virtual key update is rejected (#40512)
resourceKeyUpdate now calls d.Partial(true) on a failed /key/update so the SDK does not persist the rejected planned values into state, which made the next plan report no changes and silently dropped the update.

Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-09 19:48:35 -07:00
joshua-berri
083ddefa92
Merge pull request #40498 from BerriAI/litellm_fix_mcp_edit_tool_preview_7135
fix(mcp): refresh tool previews when editing connection settings
2026-09-09 19:48:00 -07:00
devin-ai-integration[bot]
cb9c3a9137
fix(terraform): send litellm_key model_max_budget as per-model BudgetConfig objects (#40450)
* fix(terraform): send litellm_key model_max_budget as per-model BudgetConfig objects

The key resource typed model_max_budget as map(number) and forwarded the
bare numbers to /key/generate, which the proxy rejects with a 500
('int' object is not iterable) because each model entry must be a
BudgetConfig object. The attribute is now a JSON string of per-model
budget objects, matching litellm_user, litellm_budget and litellm_tag,
with a schema version 1 state upgrader that drops the old map value so
existing state keeps loading

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(terraform): reject litellm_key model_max_budget JSON that is not per-model budget objects

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(terraform): reject unknown or empty per-model budget fields in litellm_key model_max_budget

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-09 19:46:26 -07:00
Mateo Wang
8ec2f00955
Merge pull request #40451 from BerriAI/litellm_responses_bridge_replay_encrypted_reasoning
fix(anthropic): replay OpenAI encrypted reasoning byte for byte through the /v1/messages bridge
2026-09-09 19:45:11 -07:00
devin-ai-integration[bot]
18eb6d7998
feat(terraform): allow custom team_id on litellm_team (#40459)
Add an optional, computed, force-new team_id argument to the
litellm_team resource. When set, it is sent to /team/new as the
team_id; when omitted the provider keeps generating a UUID. Read
mirrors the resource id into state so import and plan stay clean.

Resolves LIT-6399

Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-09 19:45:02 -07:00
devin-ai-integration[bot]
4ddc5e2c29
fix(terraform): read team per-model rpm/tpm limits from metadata and clear them on removal (#40439)
* fix(terraform): read team per-model rpm/tpm limits from metadata and clear them on removal

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* chore(terraform): drop explanatory comments from team per-model limit helper and tests

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-09 19:44:50 -07:00
devin-ai-integration[bot]
b64197c289
fix(terraform): preserve undeclared server-side key metadata on update (#40514)
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-09 19:44:36 -07:00
mateo-berri
966a58d1c8 Merge branch 'litellm_internal_staging' into litellm_lit_7022_azure_ai_passthrough_config 2026-09-09 19:42:28 -07:00
Mateo Wang
f0fac55fe1
Merge pull request #40114 from BerriAI/litellm_fix_background_polling_disconnect_guard
fix(responses): keep background polling alive after the client disconnects
2026-09-09 19:41:19 -07:00
mateo-berri
a6681950e8 Merge branch 'litellm_internal_staging' into litellm_lit_7022_azure_ai_passthrough_config 2026-09-09 19:35:55 -07:00
Mateo Wang
c005431cad
Merge pull request #40489 from BerriAI/litellm_lite_claude_apikeyhelper_conflict
fix(cli): let the apiKeyHelper supply Claude Code's key under lite claude
2026-09-09 19:33:59 -07:00
Mateo Wang
26626348f8
Merge pull request #40270 from BerriAI/litellm_bedrock_sign_request_off_loop
fix(bedrock): sign requests off the event loop on every async path
2026-09-09 19:31:58 -07:00
Mateo Wang
261aa2f16f
Merge pull request #40186 from BerriAI/litellm_lit_5546_count_tokens_offload
fix(token_counter): release the GIL for HuggingFace counts and cap exact counting per string
2026-09-09 19:29:17 -07:00
mateo-berri
5a1be56426 fix(router): drop bridge-tagged reasoning blocks whole when the minting deployment is not in the routed group
The cross-group branch of EncryptedContentAffinityCheck removed only the signature from
Anthropic-shaped thinking blocks, which left unsigned thinking blocks that Anthropic and
Bedrock reject (thinking.signature: Field required). Drop the whole block, the way #40280
drops undecryptable Responses input items, so the routed request carries the conversation
text with no reasoning item for those turns
2026-09-09 19:27:54 -07:00
Mateo Wang
9839bfcdbd
Merge pull request #40465 from BerriAI/litellm_e2e_spend_rows_join_virtual_key
test(e2e): every spend row a virtual key writes joins its token across all write paths
2026-09-09 19:25:41 -07:00
mateo-berri
69c8c70547 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_fix_background_polling_disconnect_guard 2026-09-09 19:21:47 -07:00
mateo-berri
3dcc09b26c fix(bedrock): keep the signing pool to AWS configs on the async-transform path
The async-transform path in the shared handler sent every provider's sign_and_log to the AWS pool, so Ollama, Snowflake, and watsonx queued behind Bedrock refreshes there. Only SignsRequestsWithAWS configs take run_aws_signing now, the rest keep the default-executor hop they had. The executor isolation test also runs on its own loop instead of pinning a one-thread default executor on the session-scoped pytest loop
2026-09-09 19:16:11 -07:00
Mateo Wang
2611f6420a
Merge pull request #33409 from BerriAI/litellm_fix_bedrock_deepseek_thinking_leak
fix(bedrock): stop leaking Anthropic `thinking`/`reasoning_effort` into DeepSeek Converse requests
2026-09-09 19:13:16 -07:00
Mateo Wang
1d18b61e3e
Merge pull request #40074 from mihidumh/fix/mai-image-unsupported-params
fix(azure_ai): reject unsupported n and size params on MAI image models
2026-09-09 19:11:11 -07:00
mateo-berri
8627576c9c fix(azure_ai): keep a deployment named like the first native path segment in the relayed URL 2026-09-09 19:11:04 -07:00
mateo-berri
ab9dc75411 merge: origin/litellm_internal_staging into litellm_lite_claude_apikeyhelper_conflict
Resolves the conflicts with the lite configure claude work from #40319: every persistent
writer and reader of Claude Code's settings file now resolves it through CLAUDE_CONFIG_DIR,
the lite up backup check only guards the default file, and each settings file keeps its own
undo receipt (the default file keeps ~/.litellm/claude_configure_state.json, any other file
gets ~/.litellm/claude_configure_state/<sha256 of its resolved path>.json).
2026-09-09 19:09:52 -07:00
mateo-berri
263c34fa26 Merge branch 'litellm_internal_staging' into litellm_fix_bedrock_deepseek_thinking_leak 2026-09-09 19:01:39 -07:00
mateo-berri
9c6a385efe docs(azure_ai): drop size from the MAI image edit router docstring 2026-09-09 18:57:37 -07:00
mateo-berri
d93baa3f2c fix(bedrock): sign on a dedicated executor instead of the shared default one
asyncio.to_thread puts every Bedrock signing on the loop's default executor, the same pool every provider's async entry point hops through, so signings parked on botocore's refresh lock queued unrelated providers behind Bedrock. run_aws_signing runs them on a 16-thread pool only AWS signing uses
2026-09-09 18:52:00 -07:00
Mateo Wang
db7ca65b69
Merge pull request #40462 from BerriAI/litellm_fix_responses_stream_named_tool_choice
fix(responses): echo a named tool_choice in the Responses API shape on the chat-completions bridge
2026-09-09 18:48:32 -07:00
kerry-berri
a268c3d274
Merge pull request #40257 from BerriAI/litellm_playground_null_cost_crash
fix(ui): guard playground cost metric against null and NaN
2026-09-09 18:47:41 -07:00
Mateo Wang
aefa1040a0
Merge pull request #40249 from BerriAI/litellm_fix_responses_bridge_reasoning_effort
fix: keep reasoning_effort for mode: responses bridge deployments
2026-09-09 18:46:58 -07:00
mateo-berri
71cf1c7901 test(e2e): drop the unused batch list client and fail loudly on a user teardown miss
The failed-batch redesign left list_batches, BatchList, BatchListQuery and
the batch object's metadata and created_at fields with no caller, and they
duplicated the batches suite's own client. delete_user discarded its result,
so a user that outlived the class fixture went unnoticed; unwrap turns that
into a teardown error like delete_key already does.
2026-09-09 18:34:15 -07:00
mateo-berri
708381948b fix(cli): compare the resolved settings path when deciding whether the lite up and autoroute backups guard it 2026-09-09 18:34:07 -07:00
mateo-berri
a25eccafc9 fix(azure_ai): drop the api_base path segments a relay already repeats 2026-09-09 18:32:13 -07:00
mateo-berri
4716b46c24 fix(router): read the affinity pin from the messages argument and strip bridge reasoning on a cross-group route
The encrypted_content_affinity check only read the Anthropic history from request_kwargs["messages"], so a caller that passes it through the callback's messages argument alone skipped the pin. Read the argument first and fall back to the kwargs.

When the minting deployment is not a candidate of the routed group, the base already strips the Responses input's encrypted reasoning; do the same for the bridge-tagged thinking blocks in Anthropic messages so the routed deployment gets the readable thinking text instead of ciphertext it cannot decrypt.
2026-09-09 18:31:08 -07:00
mateo-berri
4adf99557f fix(responses): echo "auto" for a tool_choice the bridge cannot express instead of failing after the provider call 2026-09-09 18:27:12 -07:00
mateo-berri
4d4906e94e Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_responses_bridge_replay_encrypted_reasoning
# Conflicts:
#	litellm/router_utils/pre_call_checks/encrypted_content_affinity_check.py
#	tests/test_litellm/router_utils/pre_call_checks/test_encrypted_content_affinity_check.py
2026-09-09 18:23:07 -07:00
Joshua Valluru
3169c80252 fix(mcp): preserve IPv6 hosts when comparing preview origins 2026-09-09 18:21:55 -07:00
mateo-berri
b3f2de058b fix(cli): only guard the default settings.json with the lite up and autoroute backups on login --config-claude 2026-09-09 18:19:56 -07:00
Kerry Lu
3aa9267df2 fix(ui): drop blank streamed costs instead of reading them as zero
Number("") and Number(" ") both return 0, which passed the finite check, so a
provider reporting an empty cost got a fabricated $0.000000 metric instead of
having the unusable value omitted.

Both ingestion sites carried the same inline parsing, so this pulls it into one
parseUsageCost helper that keeps finite numbers and non-blank numeric strings and
drops everything else, including booleans, arrays and breakdown objects.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YNw8WvkvCSeTcE5qvergu3
2026-09-09 18:19:52 -07:00
Mateo Wang
b22ca7ac6d
Merge pull request #40464 from BerriAI/litellm_internal_copy_35771
fix(azure): respect DEFAULT_MAX_RETRIES in initialize_azure_sdk_client (internal copy of #35771)
2026-09-09 18:19:08 -07:00
mateo-berri
c1ca963d75 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_lit_5546_count_tokens_offload 2026-09-09 18:18:17 -07:00
mateo-berri
47eb7f4349 fix(token_counter): give each event loop its own count limiter
A single process-wide anyio.CapacityLimiter wakes waiters with the
asyncio.Event of whichever loop created it, so a fifth loop in another
thread (asyncio.run per request under Celery, gunicorn sync workers, or
run_async_function from function_with_fallbacks) waited forever once four
counts were in flight, and building it at import time raised
AsyncLibraryNotFoundError on anyio below 4.2. The limiter now lives in a
RunVar and is created on first use inside the running loop.

TOKEN_COUNTER_MAX_CONCURRENT_COUNTS reads from the environment like
TOKEN_COUNTER_MAX_EXACT_CHARS, and the proxy's tokenizer lookup runs on the
shared thread pool so a Hub download no longer holds a count slot.
2026-09-09 18:17:32 -07:00