Commit graph

46534 commits

Author SHA1 Message Date
mateo
7bdcfa65b7 chore: gitignore CLAUDE.local.md
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-19 23:09:25 +00:00
Yassin Kortam
3f2e0badb4
fix(helm): default litellm-helm to the ghcr.io/berriai/litellm image (#37491)
The chart shipped ghcr.io/berriai/litellm-database as its image default,
with a comment offering it as the "optimized image with database". That
distinction no longer exists: Dockerfile and docker/Dockerfile.database
differ only in comment text and one builder-stage apk package, and both
published images bake the prisma CLI, engines, schema.prisma and
prisma_migration.py into /app, so either one runs the migrations job.

Point the default at the canonical image the release notes, the cosign
verification instructions and the chart's own README already name, and
update the chart's tests and README so nothing still refers to the
legacy repository.
2026-08-19 16:04:20 -07:00
Mateo Wang
634e699555
Merge pull request #36331 from BerriAI/devin_ai_agentcore_search
feat(search): add Amazon Bedrock AgentCore web search provider
2026-08-19 15:59:17 -07:00
mateo-berri
8494a4deee fix(vertex_ai): read the served location from optional_params when pricing proxy calls 2026-08-19 15:58:12 -07:00
Mateo Wang
da9d406e8d
Merge pull request #34887 from RayJueWang/litellm_fix_spend_deadlock_retry
fix(proxy): retry spend updates on Postgres deadlock instead of dropping them
2026-08-19 15:53:45 -07:00
Mateo Wang
449bf68498
Merge pull request #36987 from BerriAI/litellm_infer_single_worker_redis_banner
feat(proxy): auto-suppress the no-Redis banner for confirmed single-worker deployments
2026-08-19 15:52:58 -07:00
ryan-crabbe-berri
5d6033f8b4
refactor(ui): migrate the remaining dashboard pages off antd (#37524)
* refactor(ui): migrate the remaining dashboard pages off antd

Converts the teams, usage, guardrails, vector stores, cost tracking,
agents, policies, login and onboarding screens onto the shadcn
primitives, including the team info tab shell and the virtual keys
hover cards.

* fix(ui): close out the antd migration's failing type checks and tests

Alert and Badge were missing the success and info variants their call
sites already used. Combobox dropped disabled because Base UI merges the
primitive's own props over the render child, so the flag never reached
the input, and the guardrails status filter had no accessible name, which
left two comboboxes indistinguishable to the tests.

The remaining test updates swap antd's title-based queries for the roles
the shadcn controls expose.
2026-08-19 22:46:55 +00:00
Mateo Wang
07c61387fc
Merge pull request #37504 from BerriAI/litellm_fix_stale_member_search_results
fix(ui): drop stale user search answers so Enter commits the current match
2026-08-19 15:46:19 -07:00
tin-berri
dfeb12649b
feat(complexity-router): make the reasoning override floor configurable (#37537)
The reasoning override's floor was pinned to tier_boundaries.simple_medium,
so an operator could not restore the unconditional promotion nor raise the bar
independently of the SIMPLE/MEDIUM cut. Setting reasoning_override_min_score
was accepted and echoed back by /model/info, because the config model allows
extra keys, while routing ignored it.

Resolve the floor through one accessor that falls back to simple_medium when
the field is unset, so moving that boundary still moves the floor with it, and
an explicit 0 is a real floor rather than an absent one. Record the resolved
value on the routing decision so a logged row states the floor that applied,
which is also what lets the Admin UI stop hardcoding the copy PR #37500 added.
2026-08-19 15:45:39 -07:00
mateo-berri
7e27e211a1 Merge branch 'litellm_internal_staging' of https://github.com/BerriAI/litellm into litellm_lit_5785_vertex_regional_pricing
# Conflicts:
#	type-discipline-budget.json
2026-08-19 15:44:05 -07:00
ljogeiger
db50e123d5 test(vertex_ai): parametrize placeholder scoping across gemini-3 model variants 2026-08-19 22:42:24 +00:00
mateo-berri
160d3dac42 fix(proxy): issue enqueued-token Lua calls one key at a time for Redis Cluster compatibility 2026-08-19 15:40:15 -07:00
Mateo Wang
aee48f5264
Merge pull request #37528 from BerriAI/litellm_contributing_scoped_unit_tests
docs: run only the tests covering your change, leave suites to CI
2026-08-19 15:39:54 -07:00
mateo-berri
9bb5483991 fix(proxy): apply db-backed max_batch_file_size_mb on config reload 2026-08-19 15:37:44 -07:00
mateo-berri
b477d0967a fix(proxy): lift standard_logging_object onto request_data before the logging object is popped 2026-08-19 15:34:38 -07:00
ryan-crabbe-berri
74b279bc44
fix(auth): resolve bare model names against wildcard deployments in model access groups (#37492)
* fix(auth): resolve bare model names against wildcard deployments in model access groups

* test(e2e): cover model access group permission checks on keys and teams
2026-08-19 15:33:29 -07:00
ljogeiger
d5af42717e test(vertex_ai): cover id-embedded, tool-level, and end-to-end parallel signature replay 2026-08-19 22:32:24 +00:00
Yassin Kortam
6ca48efc8b
feat(cli): add lite login --config-claude to wire Claude Code at login (#37507)
`lite up` already patches ~/.claude/settings.json, but only for as long as it
runs in the foreground, and it restores the original file on exit. Users
proxying Claude Code through LiteLLM therefore have to re-wire it by hand after
every login.

--config-claude makes that write persistent. It reuses the settings shape
`lite up` writes (env.ANTHROPIC_BASE_URL plus an apiKeyHelper invocation),
preserves every unrelated key, creates the file when missing, and writes it
atomically with owner-only permissions. Plain `lite login` is unchanged.

Reaching the credential through apiKeyHelper rather than copying it into the
file means a later login refreshes it with no further action, and keeps the
short-lived CLI token out of settings.json entirely.

The shared parts of the settings-file handling move from up.py into a new
claude_settings.py, since up.py imports auth.py and so auth.py cannot import
up.py back. That module now also owns the registry of commands that can be
temporarily managing the file, so the persistent write refuses while either
`lite up` or `lite autoroute up` holds a backup it would later restore over
this write.

Because this write has no backup and no `lite down`, it is stricter than
`lite up` about the user's file: it writes through a symlinked settings.json
rather than replacing the link with a regular file, and it refuses rather than
silently discarding a non-object `env` value.

Also fixes the apiKeyHelper command itself: --base-url belongs to the
top-level `lite` group, so `lite auth print-token --base-url X` is rejected by
click with "No such option". Every settings file `lite up` has written carries
that malformed command, which makes the helper return nothing and every Claude
Code request lose its token. The existing tests only string-matched the
generated command, so the new tests parse it through the real CLI instead.
2026-08-19 15:32:14 -07:00
mateo-berri
a870d45a8a Merge branch 'litellm_internal_staging' of https://github.com/BerriAI/litellm into litellm_lit_5785_vertex_regional_pricing
# Conflicts:
#	litellm/model_prices_and_context_window_backup.json
#	model_prices_and_context_window.json
2026-08-19 15:31:42 -07:00
Mateo Wang
7e247c1477
Merge pull request #37525 from BerriAI/litellm_lit_5741_replay_match_keys
feat(e2e): canonical content-based match keys for record-and-replay
2026-08-19 15:27:16 -07:00
Mateo Wang
d192ceec73
Merge pull request #37457 from BerriAI/litellm_batch_empty_line_cost
fix(batches): stop one bad output line from zeroing an entire batch's spend
2026-08-19 15:26:37 -07:00
mateo-berri
0f4c2d71fb test(files): hoist shared batch line fixture into one constant 2026-08-19 15:22:43 -07:00
ljogeiger
677ef1e317 docs(vertex_ai): drop stale note about the removed model argument 2026-08-19 22:21:31 +00:00
mateo-berri
b39a339b7d fix(vertex_ai): apply regional endpoint uplift to cost tracking 2026-08-19 15:21:06 -07:00
mateo-berri
708ff0b910 fix(proxy): retry end-user spend updates on Postgres deadlock instead of dropping them 2026-08-19 15:20:59 -07:00
Mateo Wang
c76223a3be
chore: make it concise 2026-08-19 15:19:43 -07:00
Mateo Wang
f80b5e3cbb
Merge pull request #35998 from BerriAI/litellm_fix_bedrock_adaptive_thinking_token_accounting
fix(anthropic,bedrock): report provider thinking tokens instead of classifying them as text
2026-08-19 15:14:49 -07:00
mateo
bd0c2fdb90 docs(pr-template): run only the tests covering your change, leave suites to CI
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-19 22:14:42 +00:00
mateo-berri
7744b9100e docs(search): stop advertising yaml litellm_params knobs the search router drops 2026-08-19 15:14:20 -07:00
mateo-berri
103830ee86 fix(ui): drop the error toast for a superseded user search 2026-08-19 15:13:56 -07:00
mateo-berri
31090d122e Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_infer_single_worker_redis_banner
# Conflicts:
#	litellm/proxy/proxy_server.py
2026-08-19 15:12:57 -07:00
Mateo Wang
6b62b0b386
chore: make it more concise 2026-08-19 15:11:37 -07:00
Mateo Wang
d58b1c8558
Merge pull request #37516 from BerriAI/litellm_gemini_prompt_cache_min_tokens_4096
fix(model_prices): set prompt_cache_min_tokens=4096 for Gemini 3.5/3.6/3.7 Flash and 3.1 Pro Preview
2026-08-19 15:11:00 -07:00
mateo-berri
7a6a677b72 feat(proxy): enqueued-token rate limiting for batches with refund on completion and cancellation 2026-08-19 15:09:05 -07:00
ljogeiger
aa832d81e9 fix(vertex_ai): only fall back to a placeholder thought signature on the first parallel function call
Gemini returns a thoughtSignature on the first function call of a parallel
batch and leaves the siblings bare. When replaying that assistant turn,
litellm gave every unsigned call the skip_thought_signature_validator
placeholder, so a three-call turn went back with three signatures where
Gemini had produced one. Keep the placeholder for the first call only and
forward the siblings with whatever signature they actually carry, which is
usually none.
2026-08-19 22:08:42 +00:00
mateo-berri
680e4a5736 fix(files): treat nonpositive max_batch_file_size_mb as no cap 2026-08-19 15:05:17 -07:00
mateo-berri
710ef81a80 fix(usage): keep responses usage SDK-parseable and complete streamed reasoning splits
An unknown reasoning split now falls back to reasoning_tokens=0 in the
chat-to-responses usage translation, since the OpenAI SDK requires
output_tokens_details with an int reasoning_tokens, and the streaming
chunk builder caps the tokenized reasoning estimate at completion_tokens
and fills text_tokens with the remainder
2026-08-19 14:57:19 -07:00
mateo-berri
c5194a8507 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_fix_spend_deadlock_retry 2026-08-19 14:55:58 -07:00
mateo-berri
975e79dcef fix(e2e): make concurrent replay consumption race-free
Greptile flagged that lazy per-slug pool initialization could double-build
under concurrent replay calls, splitting consumption across a discarded
pool. Pools are now built once at ReplaySource construction and per-key
consumption is a single atomic deque pop, with a barrier-synchronized
regression test that fails 10/10 under the lazy-init mutant
2026-08-19 14:55:53 -07:00
mateo-berri
45884b9bd3 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_gemini_prompt_cache_min_tokens_4096
# Conflicts:
#	litellm/model_prices_and_context_window_backup.json
#	model_prices_and_context_window.json
2026-08-19 14:55:44 -07:00
mateo-berri
70a4f9a73a fix(search): refuse AgentCore credentials over plaintext HTTP
A trusted hostname over plain http would expose the bearer token or a
replayable SigV4 signature to network observers. Credentials now only ride
https, with localhost exempt so local MCP stubs keep working.
2026-08-19 14:50:58 -07:00
Mateo Wang
b8d5139701
Merge pull request #37473 from BerriAI/litellm_model_registry_audit_20260819
fix(model_prices): correct gemini 3.1 flash image and deepseek v4 pricing, add openai deprecation dates
2026-08-19 14:50:49 -07:00
mateo-berri
81c975cff8 test(ui): cover the loading state while a newer user search is in flight 2026-08-19 14:47:18 -07:00
mateo
03ccfe9f98 docs(contributing): scope local unit test runs to the change
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-19 21:45:20 +00:00
Mateo Wang
f4b46c81da
Merge pull request #37283 from BerriAI/devin/1787058723-registry-deprecation-dates
fix(model_prices): add provider-announced deprecation_date to 205 registry entries
2026-08-19 14:43:45 -07:00
mateo-berri
2a4598219d feat(proxy): fast-fail validation for batch input files at /v1/files 2026-08-19 14:43:22 -07:00
yassin
8ef522a2a0 fix(search): read AgentCore structuredContent results
Web-search connector 1.1.0 and later return the machine-readable results in result.structuredContent and may leave the text block as prose, which the parser dropped.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-19 21:41:57 +00:00
Mateo Wang
ccffb77b0e
Merge pull request #37515 from BerriAI/litellm_project_key_all_team_models_sentinel
fix(proxy): accept inherited model sentinels in project key limits
2026-08-19 14:40:34 -07:00
mateo-berri
ed84e82428 docs(search): use the latest Bedrock sonnet in the agentcore example config 2026-08-19 14:35:25 -07:00
mateo-berri
5eeccf69b6 fix(batches): skip undecodable batch output lines when costing 2026-08-19 14:35:01 -07:00