Commit graph

43876 commits

Author SHA1 Message Date
Mateo Wang
da9d406e8d
Merge pull request #34887 from RayJueWang/litellm_fix_spend_deadlock_retry
fix(proxy): retry spend updates on Postgres deadlock instead of dropping them
2026-08-19 15:53:45 -07:00
Mateo Wang
449bf68498
Merge pull request #36987 from BerriAI/litellm_infer_single_worker_redis_banner
feat(proxy): auto-suppress the no-Redis banner for confirmed single-worker deployments
2026-08-19 15:52:58 -07:00
ryan-crabbe-berri
5d6033f8b4
refactor(ui): migrate the remaining dashboard pages off antd (#37524)
* refactor(ui): migrate the remaining dashboard pages off antd

Converts the teams, usage, guardrails, vector stores, cost tracking,
agents, policies, login and onboarding screens onto the shadcn
primitives, including the team info tab shell and the virtual keys
hover cards.

* fix(ui): close out the antd migration's failing type checks and tests

Alert and Badge were missing the success and info variants their call
sites already used. Combobox dropped disabled because Base UI merges the
primitive's own props over the render child, so the flag never reached
the input, and the guardrails status filter had no accessible name, which
left two comboboxes indistinguishable to the tests.

The remaining test updates swap antd's title-based queries for the roles
the shadcn controls expose.
2026-08-19 22:46:55 +00:00
Mateo Wang
07c61387fc
Merge pull request #37504 from BerriAI/litellm_fix_stale_member_search_results
fix(ui): drop stale user search answers so Enter commits the current match
2026-08-19 15:46:19 -07:00
tin-berri
dfeb12649b
feat(complexity-router): make the reasoning override floor configurable (#37537)
The reasoning override's floor was pinned to tier_boundaries.simple_medium,
so an operator could not restore the unconditional promotion nor raise the bar
independently of the SIMPLE/MEDIUM cut. Setting reasoning_override_min_score
was accepted and echoed back by /model/info, because the config model allows
extra keys, while routing ignored it.

Resolve the floor through one accessor that falls back to simple_medium when
the field is unset, so moving that boundary still moves the floor with it, and
an explicit 0 is a real floor rather than an absent one. Record the resolved
value on the routing decision so a logged row states the floor that applied,
which is also what lets the Admin UI stop hardcoding the copy PR #37500 added.
2026-08-19 15:45:39 -07:00
mateo-berri
7e27e211a1 Merge branch 'litellm_internal_staging' of https://github.com/BerriAI/litellm into litellm_lit_5785_vertex_regional_pricing
# Conflicts:
#	type-discipline-budget.json
2026-08-19 15:44:05 -07:00
mateo-berri
160d3dac42 fix(proxy): issue enqueued-token Lua calls one key at a time for Redis Cluster compatibility 2026-08-19 15:40:15 -07:00
Mateo Wang
aee48f5264
Merge pull request #37528 from BerriAI/litellm_contributing_scoped_unit_tests
docs: run only the tests covering your change, leave suites to CI
2026-08-19 15:39:54 -07:00
mateo-berri
9bb5483991 fix(proxy): apply db-backed max_batch_file_size_mb on config reload 2026-08-19 15:37:44 -07:00
mateo-berri
b477d0967a fix(proxy): lift standard_logging_object onto request_data before the logging object is popped 2026-08-19 15:34:38 -07:00
ryan-crabbe-berri
74b279bc44
fix(auth): resolve bare model names against wildcard deployments in model access groups (#37492)
* fix(auth): resolve bare model names against wildcard deployments in model access groups

* test(e2e): cover model access group permission checks on keys and teams
2026-08-19 15:33:29 -07:00
Yassin Kortam
6ca48efc8b
feat(cli): add lite login --config-claude to wire Claude Code at login (#37507)
`lite up` already patches ~/.claude/settings.json, but only for as long as it
runs in the foreground, and it restores the original file on exit. Users
proxying Claude Code through LiteLLM therefore have to re-wire it by hand after
every login.

--config-claude makes that write persistent. It reuses the settings shape
`lite up` writes (env.ANTHROPIC_BASE_URL plus an apiKeyHelper invocation),
preserves every unrelated key, creates the file when missing, and writes it
atomically with owner-only permissions. Plain `lite login` is unchanged.

Reaching the credential through apiKeyHelper rather than copying it into the
file means a later login refreshes it with no further action, and keeps the
short-lived CLI token out of settings.json entirely.

The shared parts of the settings-file handling move from up.py into a new
claude_settings.py, since up.py imports auth.py and so auth.py cannot import
up.py back. That module now also owns the registry of commands that can be
temporarily managing the file, so the persistent write refuses while either
`lite up` or `lite autoroute up` holds a backup it would later restore over
this write.

Because this write has no backup and no `lite down`, it is stricter than
`lite up` about the user's file: it writes through a symlinked settings.json
rather than replacing the link with a regular file, and it refuses rather than
silently discarding a non-object `env` value.

Also fixes the apiKeyHelper command itself: --base-url belongs to the
top-level `lite` group, so `lite auth print-token --base-url X` is rejected by
click with "No such option". Every settings file `lite up` has written carries
that malformed command, which makes the helper return nothing and every Claude
Code request lose its token. The existing tests only string-matched the
generated command, so the new tests parse it through the real CLI instead.
2026-08-19 15:32:14 -07:00
mateo-berri
a870d45a8a Merge branch 'litellm_internal_staging' of https://github.com/BerriAI/litellm into litellm_lit_5785_vertex_regional_pricing
# Conflicts:
#	litellm/model_prices_and_context_window_backup.json
#	model_prices_and_context_window.json
2026-08-19 15:31:42 -07:00
Mateo Wang
7e247c1477
Merge pull request #37525 from BerriAI/litellm_lit_5741_replay_match_keys
feat(e2e): canonical content-based match keys for record-and-replay
2026-08-19 15:27:16 -07:00
Mateo Wang
d192ceec73
Merge pull request #37457 from BerriAI/litellm_batch_empty_line_cost
fix(batches): stop one bad output line from zeroing an entire batch's spend
2026-08-19 15:26:37 -07:00
mateo-berri
0f4c2d71fb test(files): hoist shared batch line fixture into one constant 2026-08-19 15:22:43 -07:00
mateo-berri
b39a339b7d fix(vertex_ai): apply regional endpoint uplift to cost tracking 2026-08-19 15:21:06 -07:00
mateo-berri
708ff0b910 fix(proxy): retry end-user spend updates on Postgres deadlock instead of dropping them 2026-08-19 15:20:59 -07:00
Mateo Wang
c76223a3be
chore: make it concise 2026-08-19 15:19:43 -07:00
Mateo Wang
f80b5e3cbb
Merge pull request #35998 from BerriAI/litellm_fix_bedrock_adaptive_thinking_token_accounting
fix(anthropic,bedrock): report provider thinking tokens instead of classifying them as text
2026-08-19 15:14:49 -07:00
mateo
bd0c2fdb90 docs(pr-template): run only the tests covering your change, leave suites to CI
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-19 22:14:42 +00:00
mateo-berri
7744b9100e docs(search): stop advertising yaml litellm_params knobs the search router drops 2026-08-19 15:14:20 -07:00
mateo-berri
103830ee86 fix(ui): drop the error toast for a superseded user search 2026-08-19 15:13:56 -07:00
mateo-berri
31090d122e Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_infer_single_worker_redis_banner
# Conflicts:
#	litellm/proxy/proxy_server.py
2026-08-19 15:12:57 -07:00
Mateo Wang
6b62b0b386
chore: make it more concise 2026-08-19 15:11:37 -07:00
Mateo Wang
d58b1c8558
Merge pull request #37516 from BerriAI/litellm_gemini_prompt_cache_min_tokens_4096
fix(model_prices): set prompt_cache_min_tokens=4096 for Gemini 3.5/3.6/3.7 Flash and 3.1 Pro Preview
2026-08-19 15:11:00 -07:00
mateo-berri
7a6a677b72 feat(proxy): enqueued-token rate limiting for batches with refund on completion and cancellation 2026-08-19 15:09:05 -07:00
mateo-berri
680e4a5736 fix(files): treat nonpositive max_batch_file_size_mb as no cap 2026-08-19 15:05:17 -07:00
mateo-berri
710ef81a80 fix(usage): keep responses usage SDK-parseable and complete streamed reasoning splits
An unknown reasoning split now falls back to reasoning_tokens=0 in the
chat-to-responses usage translation, since the OpenAI SDK requires
output_tokens_details with an int reasoning_tokens, and the streaming
chunk builder caps the tokenized reasoning estimate at completion_tokens
and fills text_tokens with the remainder
2026-08-19 14:57:19 -07:00
mateo-berri
c5194a8507 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_fix_spend_deadlock_retry 2026-08-19 14:55:58 -07:00
mateo-berri
975e79dcef fix(e2e): make concurrent replay consumption race-free
Greptile flagged that lazy per-slug pool initialization could double-build
under concurrent replay calls, splitting consumption across a discarded
pool. Pools are now built once at ReplaySource construction and per-key
consumption is a single atomic deque pop, with a barrier-synchronized
regression test that fails 10/10 under the lazy-init mutant
2026-08-19 14:55:53 -07:00
mateo-berri
45884b9bd3 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_gemini_prompt_cache_min_tokens_4096
# Conflicts:
#	litellm/model_prices_and_context_window_backup.json
#	model_prices_and_context_window.json
2026-08-19 14:55:44 -07:00
mateo-berri
70a4f9a73a fix(search): refuse AgentCore credentials over plaintext HTTP
A trusted hostname over plain http would expose the bearer token or a
replayable SigV4 signature to network observers. Credentials now only ride
https, with localhost exempt so local MCP stubs keep working.
2026-08-19 14:50:58 -07:00
Mateo Wang
b8d5139701
Merge pull request #37473 from BerriAI/litellm_model_registry_audit_20260819
fix(model_prices): correct gemini 3.1 flash image and deepseek v4 pricing, add openai deprecation dates
2026-08-19 14:50:49 -07:00
mateo-berri
81c975cff8 test(ui): cover the loading state while a newer user search is in flight 2026-08-19 14:47:18 -07:00
mateo
03ccfe9f98 docs(contributing): scope local unit test runs to the change
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-19 21:45:20 +00:00
Mateo Wang
f4b46c81da
Merge pull request #37283 from BerriAI/devin/1787058723-registry-deprecation-dates
fix(model_prices): add provider-announced deprecation_date to 205 registry entries
2026-08-19 14:43:45 -07:00
mateo-berri
2a4598219d feat(proxy): fast-fail validation for batch input files at /v1/files 2026-08-19 14:43:22 -07:00
yassin
8ef522a2a0 fix(search): read AgentCore structuredContent results
Web-search connector 1.1.0 and later return the machine-readable results in result.structuredContent and may leave the text block as prose, which the parser dropped.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-19 21:41:57 +00:00
Mateo Wang
ccffb77b0e
Merge pull request #37515 from BerriAI/litellm_project_key_all_team_models_sentinel
fix(proxy): accept inherited model sentinels in project key limits
2026-08-19 14:40:34 -07:00
mateo-berri
ed84e82428 docs(search): use the latest Bedrock sonnet in the agentcore example config 2026-08-19 14:35:25 -07:00
mateo-berri
5eeccf69b6 fix(batches): skip undecodable batch output lines when costing 2026-08-19 14:35:01 -07:00
mateo-berri
125587d286 feat(e2e): canonical content-based match keys for record-and-replay
Replay previously matched interactions by transport verb and path in
recorded order, so a request whose body drifted from the recording
silently replayed the stale response, and reordering two independent
calls broke replay even though both were recorded. Match keys are now
canonical: fixture_canonical.py strips volatile headers and credential
fields, replaces unique markers, generated ids, uuids, and timestamps
with fixed placeholders, sorts object keys, and hashes what remains, so
a key is stable across runs and machines while any real content drift
is a hard ReplayMiss naming the computed key, the closest recorded key
with its file, and a content diff, with no fallthrough to a live call.
Matching is order-independent across distinct keys and FIFO within one
key. Recording now also redacts credential body and form fields (not
just auth headers) so provider keys never land in bundles.

Resolves LIT-5741
2026-08-19 14:33:22 -07:00
Mateo Wang
59c7e7a17d
Merge pull request #37360 from BerriAI/litellm_lit_5729_e2e_record_replay_seam
feat(e2e): add record/replay transport seam and fixture bundle format
2026-08-19 14:28:03 -07:00
tin-berri
0de60a2ff2
fix(mcp): stop reporting failed OpenAPI tool calls as successes (#37496)
An OpenAPI-backed MCP tool whose upstream answered 401 came back as a
successful tool result carrying the upstream's rejection as its content, so a
caller saw {"error":"invalid_token"} presented as data and the gateway recorded
the request in its own spend log as call_mcp_tool | success.

Three layers each erased the outcome. The request function returned
response.text whatever the status, _handle_local_mcp_tool caught every exception
and returned it as ordinary TextContent, and both dispatch sites then stamped
isError=False unconditionally. Fixing only the first, which is the obvious fix,
changes nothing, because the two above it still map failure onto the
success-shaped value.

The status is now classified where the response is held: a 401 becomes
MCPUpstreamAuthError so the caller is told to re-authenticate, and every other
non-2xx becomes MCPOpenApiUpstreamError, which carries the status and drops the
upstream body rather than serving it as tool content. _handle_local_mcp_tool no
longer swallows, and the call_tool arm keeps the auth error's type. Nothing new
renders these: call_mcp_tool and call_tool_rest_api already turn them into an
isError result naming the status and into a real 401 with WWW-Authenticate, and
the OpenAPI path simply never reached them.

The result is now byte-identical to the regular MCP path for the same failure.
2026-08-19 14:26:02 -07:00
mateo-berri
20a3a16c2f fix(proxy): populate deployment fields on failed-request spend logs from the standard logging payload 2026-08-19 14:25:43 -07:00
mateo-berri
b3c3e6ebb8 fix(search): default the AgentCore MCP protocol version to the gateway default 2026-08-19 14:25:34 -07:00
tin-berri
afbfc3f8fa
fix(complexity-router): gate the reasoning override on a non-SIMPLE score (#37500)
Two or more reasoning keyword matches promoted a request straight to the
REASONING tier no matter what the weighted score said, so "hi, step by step,
pros and cons" scored 0.100 and still bought the most expensive tier.

Require the score to clear the simple_medium boundary before the override
applies. Promotion from MEDIUM or COMPLEX is unchanged; only prompts the
scorer already placed in the cheapest band stay there.
2026-08-19 14:19:24 -07:00
Mateo Wang
eec27a9cb3
Merge pull request #36593 from BerriAI/devin_ai_lit_5445_perplexity_stream_dict_cost
fix(streaming): accept provider cost objects when propagating usage cost
2026-08-19 14:18:36 -07:00
mateo-berri
4606ea3f12 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_fix_stale_member_search_results
# Conflicts:
#	ui/litellm-dashboard/src/components/common_components/user_search_modal.tsx
2026-08-19 14:18:21 -07:00