Commit graph

48082 commits

Author SHA1 Message Date
devin-ai-integration[bot]
ea106fd8be
fix(terraform): send changed litellm_key duration on /key/update so expires is recomputed (#40511)
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-10 02:57:07 +00:00
devin-ai-integration[bot]
fb603fdb6b
fix(terraform): keep prior state when a virtual key update is rejected (#40512)
resourceKeyUpdate now calls d.Partial(true) on a failed /key/update so the SDK does not persist the rejected planned values into state, which made the next plan report no changes and silently dropped the update.

Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-09 19:48:35 -07:00
joshua-berri
083ddefa92
Merge pull request #40498 from BerriAI/litellm_fix_mcp_edit_tool_preview_7135
fix(mcp): refresh tool previews when editing connection settings
2026-09-09 19:48:00 -07:00
devin-ai-integration[bot]
cb9c3a9137
fix(terraform): send litellm_key model_max_budget as per-model BudgetConfig objects (#40450)
* fix(terraform): send litellm_key model_max_budget as per-model BudgetConfig objects

The key resource typed model_max_budget as map(number) and forwarded the
bare numbers to /key/generate, which the proxy rejects with a 500
('int' object is not iterable) because each model entry must be a
BudgetConfig object. The attribute is now a JSON string of per-model
budget objects, matching litellm_user, litellm_budget and litellm_tag,
with a schema version 1 state upgrader that drops the old map value so
existing state keeps loading

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(terraform): reject litellm_key model_max_budget JSON that is not per-model budget objects

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(terraform): reject unknown or empty per-model budget fields in litellm_key model_max_budget

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-09 19:46:26 -07:00
Mateo Wang
8ec2f00955
Merge pull request #40451 from BerriAI/litellm_responses_bridge_replay_encrypted_reasoning
fix(anthropic): replay OpenAI encrypted reasoning byte for byte through the /v1/messages bridge
2026-09-09 19:45:11 -07:00
devin-ai-integration[bot]
18eb6d7998
feat(terraform): allow custom team_id on litellm_team (#40459)
Add an optional, computed, force-new team_id argument to the
litellm_team resource. When set, it is sent to /team/new as the
team_id; when omitted the provider keeps generating a UUID. Read
mirrors the resource id into state so import and plan stay clean.

Resolves LIT-6399

Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-09 19:45:02 -07:00
devin-ai-integration[bot]
4ddc5e2c29
fix(terraform): read team per-model rpm/tpm limits from metadata and clear them on removal (#40439)
* fix(terraform): read team per-model rpm/tpm limits from metadata and clear them on removal

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* chore(terraform): drop explanatory comments from team per-model limit helper and tests

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-09 19:44:50 -07:00
devin-ai-integration[bot]
b64197c289
fix(terraform): preserve undeclared server-side key metadata on update (#40514)
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-09 19:44:36 -07:00
Mateo Wang
f0fac55fe1
Merge pull request #40114 from BerriAI/litellm_fix_background_polling_disconnect_guard
fix(responses): keep background polling alive after the client disconnects
2026-09-09 19:41:19 -07:00
Mateo Wang
c005431cad
Merge pull request #40489 from BerriAI/litellm_lite_claude_apikeyhelper_conflict
fix(cli): let the apiKeyHelper supply Claude Code's key under lite claude
2026-09-09 19:33:59 -07:00
Mateo Wang
26626348f8
Merge pull request #40270 from BerriAI/litellm_bedrock_sign_request_off_loop
fix(bedrock): sign requests off the event loop on every async path
2026-09-09 19:31:58 -07:00
Mateo Wang
261aa2f16f
Merge pull request #40186 from BerriAI/litellm_lit_5546_count_tokens_offload
fix(token_counter): release the GIL for HuggingFace counts and cap exact counting per string
2026-09-09 19:29:17 -07:00
mateo-berri
5a1be56426 fix(router): drop bridge-tagged reasoning blocks whole when the minting deployment is not in the routed group
The cross-group branch of EncryptedContentAffinityCheck removed only the signature from
Anthropic-shaped thinking blocks, which left unsigned thinking blocks that Anthropic and
Bedrock reject (thinking.signature: Field required). Drop the whole block, the way #40280
drops undecryptable Responses input items, so the routed request carries the conversation
text with no reasoning item for those turns
2026-09-09 19:27:54 -07:00
Mateo Wang
9839bfcdbd
Merge pull request #40465 from BerriAI/litellm_e2e_spend_rows_join_virtual_key
test(e2e): every spend row a virtual key writes joins its token across all write paths
2026-09-09 19:25:41 -07:00
mateo-berri
69c8c70547 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_fix_background_polling_disconnect_guard 2026-09-09 19:21:47 -07:00
mateo-berri
3dcc09b26c fix(bedrock): keep the signing pool to AWS configs on the async-transform path
The async-transform path in the shared handler sent every provider's sign_and_log to the AWS pool, so Ollama, Snowflake, and watsonx queued behind Bedrock refreshes there. Only SignsRequestsWithAWS configs take run_aws_signing now, the rest keep the default-executor hop they had. The executor isolation test also runs on its own loop instead of pinning a one-thread default executor on the session-scoped pytest loop
2026-09-09 19:16:11 -07:00
Mateo Wang
2611f6420a
Merge pull request #33409 from BerriAI/litellm_fix_bedrock_deepseek_thinking_leak
fix(bedrock): stop leaking Anthropic `thinking`/`reasoning_effort` into DeepSeek Converse requests
2026-09-09 19:13:16 -07:00
Mateo Wang
1d18b61e3e
Merge pull request #40074 from mihidumh/fix/mai-image-unsupported-params
fix(azure_ai): reject unsupported n and size params on MAI image models
2026-09-09 19:11:11 -07:00
mateo-berri
ab9dc75411 merge: origin/litellm_internal_staging into litellm_lite_claude_apikeyhelper_conflict
Resolves the conflicts with the lite configure claude work from #40319: every persistent
writer and reader of Claude Code's settings file now resolves it through CLAUDE_CONFIG_DIR,
the lite up backup check only guards the default file, and each settings file keeps its own
undo receipt (the default file keeps ~/.litellm/claude_configure_state.json, any other file
gets ~/.litellm/claude_configure_state/<sha256 of its resolved path>.json).
2026-09-09 19:09:52 -07:00
mateo-berri
263c34fa26 Merge branch 'litellm_internal_staging' into litellm_fix_bedrock_deepseek_thinking_leak 2026-09-09 19:01:39 -07:00
mateo-berri
9c6a385efe docs(azure_ai): drop size from the MAI image edit router docstring 2026-09-09 18:57:37 -07:00
mateo-berri
d93baa3f2c fix(bedrock): sign on a dedicated executor instead of the shared default one
asyncio.to_thread puts every Bedrock signing on the loop's default executor, the same pool every provider's async entry point hops through, so signings parked on botocore's refresh lock queued unrelated providers behind Bedrock. run_aws_signing runs them on a 16-thread pool only AWS signing uses
2026-09-09 18:52:00 -07:00
Mateo Wang
db7ca65b69
Merge pull request #40462 from BerriAI/litellm_fix_responses_stream_named_tool_choice
fix(responses): echo a named tool_choice in the Responses API shape on the chat-completions bridge
2026-09-09 18:48:32 -07:00
kerry-berri
a268c3d274
Merge pull request #40257 from BerriAI/litellm_playground_null_cost_crash
fix(ui): guard playground cost metric against null and NaN
2026-09-09 18:47:41 -07:00
Mateo Wang
aefa1040a0
Merge pull request #40249 from BerriAI/litellm_fix_responses_bridge_reasoning_effort
fix: keep reasoning_effort for mode: responses bridge deployments
2026-09-09 18:46:58 -07:00
mateo-berri
71cf1c7901 test(e2e): drop the unused batch list client and fail loudly on a user teardown miss
The failed-batch redesign left list_batches, BatchList, BatchListQuery and
the batch object's metadata and created_at fields with no caller, and they
duplicated the batches suite's own client. delete_user discarded its result,
so a user that outlived the class fixture went unnoticed; unwrap turns that
into a teardown error like delete_key already does.
2026-09-09 18:34:15 -07:00
mateo-berri
708381948b fix(cli): compare the resolved settings path when deciding whether the lite up and autoroute backups guard it 2026-09-09 18:34:07 -07:00
mateo-berri
4716b46c24 fix(router): read the affinity pin from the messages argument and strip bridge reasoning on a cross-group route
The encrypted_content_affinity check only read the Anthropic history from request_kwargs["messages"], so a caller that passes it through the callback's messages argument alone skipped the pin. Read the argument first and fall back to the kwargs.

When the minting deployment is not a candidate of the routed group, the base already strips the Responses input's encrypted reasoning; do the same for the bridge-tagged thinking blocks in Anthropic messages so the routed deployment gets the readable thinking text instead of ciphertext it cannot decrypt.
2026-09-09 18:31:08 -07:00
mateo-berri
4adf99557f fix(responses): echo "auto" for a tool_choice the bridge cannot express instead of failing after the provider call 2026-09-09 18:27:12 -07:00
mateo-berri
4d4906e94e Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_responses_bridge_replay_encrypted_reasoning
# Conflicts:
#	litellm/router_utils/pre_call_checks/encrypted_content_affinity_check.py
#	tests/test_litellm/router_utils/pre_call_checks/test_encrypted_content_affinity_check.py
2026-09-09 18:23:07 -07:00
Joshua Valluru
3169c80252 fix(mcp): preserve IPv6 hosts when comparing preview origins 2026-09-09 18:21:55 -07:00
mateo-berri
b3f2de058b fix(cli): only guard the default settings.json with the lite up and autoroute backups on login --config-claude 2026-09-09 18:19:56 -07:00
Kerry Lu
3aa9267df2 fix(ui): drop blank streamed costs instead of reading them as zero
Number("") and Number(" ") both return 0, which passed the finite check, so a
provider reporting an empty cost got a fabricated $0.000000 metric instead of
having the unusable value omitted.

Both ingestion sites carried the same inline parsing, so this pulls it into one
parseUsageCost helper that keeps finite numbers and non-blank numeric strings and
drops everything else, including booleans, arrays and breakdown objects.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YNw8WvkvCSeTcE5qvergu3
2026-09-09 18:19:52 -07:00
Mateo Wang
b22ca7ac6d
Merge pull request #40464 from BerriAI/litellm_internal_copy_35771
fix(azure): respect DEFAULT_MAX_RETRIES in initialize_azure_sdk_client (internal copy of #35771)
2026-09-09 18:19:08 -07:00
mateo-berri
c1ca963d75 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_lit_5546_count_tokens_offload 2026-09-09 18:18:17 -07:00
mateo-berri
47eb7f4349 fix(token_counter): give each event loop its own count limiter
A single process-wide anyio.CapacityLimiter wakes waiters with the
asyncio.Event of whichever loop created it, so a fifth loop in another
thread (asyncio.run per request under Celery, gunicorn sync workers, or
run_async_function from function_with_fallbacks) waited forever once four
counts were in flight, and building it at import time raised
AsyncLibraryNotFoundError on anyio below 4.2. The limiter now lives in a
RunVar and is created on first use inside the running loop.

TOKEN_COUNTER_MAX_CONCURRENT_COUNTS reads from the environment like
TOKEN_COUNTER_MAX_EXACT_CHARS, and the proxy's tokenizer lookup runs on the
shared thread pool so a Hub download no longer holds a count slot.
2026-09-09 18:17:32 -07:00
Mateo Wang
21ae29b759
Merge pull request #40284 from BerriAI/litellm_legacy_hook_streaming_pipeline_step
feat(guardrails): run legacy post-call hooks as streaming pipeline steps
2026-09-09 18:14:37 -07:00
Mateo Wang
53f3e70f02
Merge pull request #40461 from BerriAI/litellm_lit_7373_responses_custom_tool_call_guardrail
fix(guardrails): scan and rewrite Responses custom_tool_call output items
2026-09-09 18:13:00 -07:00
Joshua Valluru
493c98f9bf fix(mcp): normalize default ports in preview origins 2026-09-09 18:11:52 -07:00
Mateo Wang
0e088337a2
Merge pull request #31884 from BerriAI/litellm_add-claude-sonnet-5-pricing
fix(pricing): rolling model registry update: Bedrock gpt-6-astra, gpt-image-2.5, Cohere rerank 4, Vertex Grok 4.3/4.6/4.20, Gemini 3.5 audio, OpenAI web search fee, xAI Imagine video, Lyria 3.5, Voyage, ChatGPT GPT-5.5/5.6, Bedrock Mantle, Scaleway dates
2026-09-09 18:11:37 -07:00
tin-berri
e34c4c8edc
fix(ui): scope shadow eval models to configured chat groups (#40488)
Co-authored-by: Claude Code <noreply@anthropic.com>
2026-09-09 18:09:28 -07:00
Joshua Valluru
3523f3731c fix(mcp): bind inherited preview credentials to the saved origin 2026-09-09 18:04:36 -07:00
tin-berri
317b29e69d
feat(cli): add lite configure claude and lite unconfigure claude (#40319)
Persistently route Claude Code through a LiteLLM proxy with a long-lived virtual key or the stored lite login, turn on gateway model discovery so /model lists the proxy's models, optionally pick the model Claude Code starts on, and record what changed so unconfigure restores only the keys the user has not touched since. lite login --config-claude writes through the same receipt and is undoable too. The two settings merges (lite up / --config-claude and lite autoroute) collapse into one credential-aware merge
2026-09-09 18:02:23 -07:00
mateo-berri
b554d79914 fix(cli): resolve Claude Code's settings file through CLAUDE_CONFIG_DIR
Claude Code reads settings.json from CLAUDE_CONFIG_DIR when it is set,
while lite login --config-claude wrote to ~/.claude/settings.json and
lite claude checked that same file for its apiKeyHelper. With the
override set, lite could drop ANTHROPIC_AUTH_TOKEN because the helper
lives in a file Claude Code never reads, leaving it with no key at all.

claude_settings_path(environ) now picks the file the way Claude Code
does, and both commands go through it. The CLI test directory gets an
autouse conftest that isolates HOME, USERPROFILE, and CLAUDE_CONFIG_DIR
per test and fails any test that writes the developer's real
settings.json, restoring it first.
2026-09-09 18:01:11 -07:00
Joshua Valluru
245369764e fix(mcp): preserve edited settings in static connection previews 2026-09-09 17:49:52 -07:00
kerry-berri
b8d7f68aeb
Apply suggestion from @greptile-apps[bot]
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-09-09 17:46:12 -07:00
joshua-berri
a8bb2f93b9
Merge pull request #40494 from BerriAI/litellm_fix_mcp_dotted_properties_5777
fix(ui): preserve dotted MCP tool argument names
2026-09-09 17:45:00 -07:00
Joshua Valluru
892d20d86f fix(mcp): refresh tools when editing server connection settings 2026-09-09 17:34:24 -07:00
tin-berri
6b721de3e5
fix(databricks): route Unity model services through AI Gateway (#40492)
Co-authored-by: Claude Code <noreply@anthropic.com>
2026-09-09 17:15:48 -07:00
Joshua Valluru
0b21b99ebd fix(ui): type MCP argument resolver results explicitly 2026-09-09 16:57:15 -07:00