Commit graph

43817 commits

Author SHA1 Message Date
Yuneng Jiang
05ec5cce8d
chore(ci): remove CodSpeed benchmarks
Drops the CodSpeed workflow along with everything that existed only to feed
it: the tests/benchmarks suite, the pytest-codspeed CI dependency, its
uv.lock and license-cache entries, and the README badge.
2026-08-20 12:45:21 -07:00
Yassin Kortam
3f2e0badb4
fix(helm): default litellm-helm to the ghcr.io/berriai/litellm image (#37491)
The chart shipped ghcr.io/berriai/litellm-database as its image default,
with a comment offering it as the "optimized image with database". That
distinction no longer exists: Dockerfile and docker/Dockerfile.database
differ only in comment text and one builder-stage apk package, and both
published images bake the prisma CLI, engines, schema.prisma and
prisma_migration.py into /app, so either one runs the migrations job.

Point the default at the canonical image the release notes, the cosign
verification instructions and the chart's own README already name, and
update the chart's tests and README so nothing still refers to the
legacy repository.
2026-08-19 16:04:20 -07:00
Mateo Wang
634e699555
Merge pull request #36331 from BerriAI/devin_ai_agentcore_search
feat(search): add Amazon Bedrock AgentCore web search provider
2026-08-19 15:59:17 -07:00
Mateo Wang
da9d406e8d
Merge pull request #34887 from RayJueWang/litellm_fix_spend_deadlock_retry
fix(proxy): retry spend updates on Postgres deadlock instead of dropping them
2026-08-19 15:53:45 -07:00
Mateo Wang
449bf68498
Merge pull request #36987 from BerriAI/litellm_infer_single_worker_redis_banner
feat(proxy): auto-suppress the no-Redis banner for confirmed single-worker deployments
2026-08-19 15:52:58 -07:00
ryan-crabbe-berri
5d6033f8b4
refactor(ui): migrate the remaining dashboard pages off antd (#37524)
* refactor(ui): migrate the remaining dashboard pages off antd

Converts the teams, usage, guardrails, vector stores, cost tracking,
agents, policies, login and onboarding screens onto the shadcn
primitives, including the team info tab shell and the virtual keys
hover cards.

* fix(ui): close out the antd migration's failing type checks and tests

Alert and Badge were missing the success and info variants their call
sites already used. Combobox dropped disabled because Base UI merges the
primitive's own props over the render child, so the flag never reached
the input, and the guardrails status filter had no accessible name, which
left two comboboxes indistinguishable to the tests.

The remaining test updates swap antd's title-based queries for the roles
the shadcn controls expose.
2026-08-19 22:46:55 +00:00
Mateo Wang
07c61387fc
Merge pull request #37504 from BerriAI/litellm_fix_stale_member_search_results
fix(ui): drop stale user search answers so Enter commits the current match
2026-08-19 15:46:19 -07:00
tin-berri
dfeb12649b
feat(complexity-router): make the reasoning override floor configurable (#37537)
The reasoning override's floor was pinned to tier_boundaries.simple_medium,
so an operator could not restore the unconditional promotion nor raise the bar
independently of the SIMPLE/MEDIUM cut. Setting reasoning_override_min_score
was accepted and echoed back by /model/info, because the config model allows
extra keys, while routing ignored it.

Resolve the floor through one accessor that falls back to simple_medium when
the field is unset, so moving that boundary still moves the floor with it, and
an explicit 0 is a real floor rather than an absent one. Record the resolved
value on the routing decision so a logged row states the floor that applied,
which is also what lets the Admin UI stop hardcoding the copy PR #37500 added.
2026-08-19 15:45:39 -07:00
Mateo Wang
aee48f5264
Merge pull request #37528 from BerriAI/litellm_contributing_scoped_unit_tests
docs: run only the tests covering your change, leave suites to CI
2026-08-19 15:39:54 -07:00
ryan-crabbe-berri
74b279bc44
fix(auth): resolve bare model names against wildcard deployments in model access groups (#37492)
* fix(auth): resolve bare model names against wildcard deployments in model access groups

* test(e2e): cover model access group permission checks on keys and teams
2026-08-19 15:33:29 -07:00
Yassin Kortam
6ca48efc8b
feat(cli): add lite login --config-claude to wire Claude Code at login (#37507)
`lite up` already patches ~/.claude/settings.json, but only for as long as it
runs in the foreground, and it restores the original file on exit. Users
proxying Claude Code through LiteLLM therefore have to re-wire it by hand after
every login.

--config-claude makes that write persistent. It reuses the settings shape
`lite up` writes (env.ANTHROPIC_BASE_URL plus an apiKeyHelper invocation),
preserves every unrelated key, creates the file when missing, and writes it
atomically with owner-only permissions. Plain `lite login` is unchanged.

Reaching the credential through apiKeyHelper rather than copying it into the
file means a later login refreshes it with no further action, and keeps the
short-lived CLI token out of settings.json entirely.

The shared parts of the settings-file handling move from up.py into a new
claude_settings.py, since up.py imports auth.py and so auth.py cannot import
up.py back. That module now also owns the registry of commands that can be
temporarily managing the file, so the persistent write refuses while either
`lite up` or `lite autoroute up` holds a backup it would later restore over
this write.

Because this write has no backup and no `lite down`, it is stricter than
`lite up` about the user's file: it writes through a symlinked settings.json
rather than replacing the link with a regular file, and it refuses rather than
silently discarding a non-object `env` value.

Also fixes the apiKeyHelper command itself: --base-url belongs to the
top-level `lite` group, so `lite auth print-token --base-url X` is rejected by
click with "No such option". Every settings file `lite up` has written carries
that malformed command, which makes the helper return nothing and every Claude
Code request lose its token. The existing tests only string-matched the
generated command, so the new tests parse it through the real CLI instead.
2026-08-19 15:32:14 -07:00
Mateo Wang
7e247c1477
Merge pull request #37525 from BerriAI/litellm_lit_5741_replay_match_keys
feat(e2e): canonical content-based match keys for record-and-replay
2026-08-19 15:27:16 -07:00
Mateo Wang
d192ceec73
Merge pull request #37457 from BerriAI/litellm_batch_empty_line_cost
fix(batches): stop one bad output line from zeroing an entire batch's spend
2026-08-19 15:26:37 -07:00
mateo-berri
708ff0b910 fix(proxy): retry end-user spend updates on Postgres deadlock instead of dropping them 2026-08-19 15:20:59 -07:00
Mateo Wang
c76223a3be
chore: make it concise 2026-08-19 15:19:43 -07:00
Mateo Wang
f80b5e3cbb
Merge pull request #35998 from BerriAI/litellm_fix_bedrock_adaptive_thinking_token_accounting
fix(anthropic,bedrock): report provider thinking tokens instead of classifying them as text
2026-08-19 15:14:49 -07:00
mateo
bd0c2fdb90 docs(pr-template): run only the tests covering your change, leave suites to CI
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-19 22:14:42 +00:00
mateo-berri
7744b9100e docs(search): stop advertising yaml litellm_params knobs the search router drops 2026-08-19 15:14:20 -07:00
mateo-berri
103830ee86 fix(ui): drop the error toast for a superseded user search 2026-08-19 15:13:56 -07:00
mateo-berri
31090d122e Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_infer_single_worker_redis_banner
# Conflicts:
#	litellm/proxy/proxy_server.py
2026-08-19 15:12:57 -07:00
Mateo Wang
6b62b0b386
chore: make it more concise 2026-08-19 15:11:37 -07:00
Mateo Wang
d58b1c8558
Merge pull request #37516 from BerriAI/litellm_gemini_prompt_cache_min_tokens_4096
fix(model_prices): set prompt_cache_min_tokens=4096 for Gemini 3.5/3.6/3.7 Flash and 3.1 Pro Preview
2026-08-19 15:11:00 -07:00
mateo-berri
710ef81a80 fix(usage): keep responses usage SDK-parseable and complete streamed reasoning splits
An unknown reasoning split now falls back to reasoning_tokens=0 in the
chat-to-responses usage translation, since the OpenAI SDK requires
output_tokens_details with an int reasoning_tokens, and the streaming
chunk builder caps the tokenized reasoning estimate at completion_tokens
and fills text_tokens with the remainder
2026-08-19 14:57:19 -07:00
mateo-berri
c5194a8507 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_fix_spend_deadlock_retry 2026-08-19 14:55:58 -07:00
mateo-berri
975e79dcef fix(e2e): make concurrent replay consumption race-free
Greptile flagged that lazy per-slug pool initialization could double-build
under concurrent replay calls, splitting consumption across a discarded
pool. Pools are now built once at ReplaySource construction and per-key
consumption is a single atomic deque pop, with a barrier-synchronized
regression test that fails 10/10 under the lazy-init mutant
2026-08-19 14:55:53 -07:00
mateo-berri
45884b9bd3 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_gemini_prompt_cache_min_tokens_4096
# Conflicts:
#	litellm/model_prices_and_context_window_backup.json
#	model_prices_and_context_window.json
2026-08-19 14:55:44 -07:00
mateo-berri
70a4f9a73a fix(search): refuse AgentCore credentials over plaintext HTTP
A trusted hostname over plain http would expose the bearer token or a
replayable SigV4 signature to network observers. Credentials now only ride
https, with localhost exempt so local MCP stubs keep working.
2026-08-19 14:50:58 -07:00
Mateo Wang
b8d5139701
Merge pull request #37473 from BerriAI/litellm_model_registry_audit_20260819
fix(model_prices): correct gemini 3.1 flash image and deepseek v4 pricing, add openai deprecation dates
2026-08-19 14:50:49 -07:00
mateo-berri
81c975cff8 test(ui): cover the loading state while a newer user search is in flight 2026-08-19 14:47:18 -07:00
mateo
03ccfe9f98 docs(contributing): scope local unit test runs to the change
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-19 21:45:20 +00:00
Mateo Wang
f4b46c81da
Merge pull request #37283 from BerriAI/devin/1787058723-registry-deprecation-dates
fix(model_prices): add provider-announced deprecation_date to 205 registry entries
2026-08-19 14:43:45 -07:00
yassin
8ef522a2a0 fix(search): read AgentCore structuredContent results
Web-search connector 1.1.0 and later return the machine-readable results in result.structuredContent and may leave the text block as prose, which the parser dropped.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-19 21:41:57 +00:00
Mateo Wang
ccffb77b0e
Merge pull request #37515 from BerriAI/litellm_project_key_all_team_models_sentinel
fix(proxy): accept inherited model sentinels in project key limits
2026-08-19 14:40:34 -07:00
mateo-berri
ed84e82428 docs(search): use the latest Bedrock sonnet in the agentcore example config 2026-08-19 14:35:25 -07:00
mateo-berri
5eeccf69b6 fix(batches): skip undecodable batch output lines when costing 2026-08-19 14:35:01 -07:00
mateo-berri
125587d286 feat(e2e): canonical content-based match keys for record-and-replay
Replay previously matched interactions by transport verb and path in
recorded order, so a request whose body drifted from the recording
silently replayed the stale response, and reordering two independent
calls broke replay even though both were recorded. Match keys are now
canonical: fixture_canonical.py strips volatile headers and credential
fields, replaces unique markers, generated ids, uuids, and timestamps
with fixed placeholders, sorts object keys, and hashes what remains, so
a key is stable across runs and machines while any real content drift
is a hard ReplayMiss naming the computed key, the closest recorded key
with its file, and a content diff, with no fallthrough to a live call.
Matching is order-independent across distinct keys and FIFO within one
key. Recording now also redacts credential body and form fields (not
just auth headers) so provider keys never land in bundles.

Resolves LIT-5741
2026-08-19 14:33:22 -07:00
Mateo Wang
59c7e7a17d
Merge pull request #37360 from BerriAI/litellm_lit_5729_e2e_record_replay_seam
feat(e2e): add record/replay transport seam and fixture bundle format
2026-08-19 14:28:03 -07:00
tin-berri
0de60a2ff2
fix(mcp): stop reporting failed OpenAPI tool calls as successes (#37496)
An OpenAPI-backed MCP tool whose upstream answered 401 came back as a
successful tool result carrying the upstream's rejection as its content, so a
caller saw {"error":"invalid_token"} presented as data and the gateway recorded
the request in its own spend log as call_mcp_tool | success.

Three layers each erased the outcome. The request function returned
response.text whatever the status, _handle_local_mcp_tool caught every exception
and returned it as ordinary TextContent, and both dispatch sites then stamped
isError=False unconditionally. Fixing only the first, which is the obvious fix,
changes nothing, because the two above it still map failure onto the
success-shaped value.

The status is now classified where the response is held: a 401 becomes
MCPUpstreamAuthError so the caller is told to re-authenticate, and every other
non-2xx becomes MCPOpenApiUpstreamError, which carries the status and drops the
upstream body rather than serving it as tool content. _handle_local_mcp_tool no
longer swallows, and the call_tool arm keeps the auth error's type. Nothing new
renders these: call_mcp_tool and call_tool_rest_api already turn them into an
isError result naming the status and into a real 401 with WWW-Authenticate, and
the OpenAPI path simply never reached them.

The result is now byte-identical to the regular MCP path for the same failure.
2026-08-19 14:26:02 -07:00
mateo-berri
b3c3e6ebb8 fix(search): default the AgentCore MCP protocol version to the gateway default 2026-08-19 14:25:34 -07:00
tin-berri
afbfc3f8fa
fix(complexity-router): gate the reasoning override on a non-SIMPLE score (#37500)
Two or more reasoning keyword matches promoted a request straight to the
REASONING tier no matter what the weighted score said, so "hi, step by step,
pros and cons" scored 0.100 and still bought the most expensive tier.

Require the score to clear the simple_medium boundary before the override
applies. Promotion from MEDIUM or COMPLEX is unchanged; only prompts the
scorer already placed in the cheapest band stay there.
2026-08-19 14:19:24 -07:00
Mateo Wang
eec27a9cb3
Merge pull request #36593 from BerriAI/devin_ai_lit_5445_perplexity_stream_dict_cost
fix(streaming): accept provider cost objects when propagating usage cost
2026-08-19 14:18:36 -07:00
mateo-berri
4606ea3f12 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_fix_stale_member_search_results
# Conflicts:
#	ui/litellm-dashboard/src/components/common_components/user_search_modal.tsx
2026-08-19 14:18:21 -07:00
ryan-crabbe-berri
f1e143a87c
chore(ui): upgrade the dashboard to React 19 (#37411)
* chore(ui): upgrade the dashboard to React 19

Bumps react and react-dom from 18.3.1 to 19.2.8 with matching @types. Next 16 already required a React 19 peer, so this aligns the dashboard with what the framework expects and unblocks Base UI and shadcn work that assumes the React 19 ref model.

React 19 passes ref through as a regular prop, so the setup file's forwardRef tripwire and the ref-forwarding test's forwardRef case no longer describe real behavior; both now assert the React 19 contract instead. useRef<T>(null) now yields RefObject<T | null>, which is the one prop type MessageList had to widen.

* test(ui): wait for a Base UI select popup to open before clicking an option

The option lands in the DOM one render before the popup finishes entering, while its positioner still carries pointer-events: none, so clicking it throws. Waiting on the option's text alone was a race that React 19's flush timing loses, which is why four ToolPolicies cases went red on the bump.

chooseSelectOption in test-utils opens the trigger, finds the option by role, waits for it to stop being pointer-blocked, then clicks. It also replaces the last-match-by-text hack, which only worked because the popup happens to portal after the table.
2026-08-19 21:18:08 +00:00
mateo-berri
77716eeaed fix(model_prices): set prompt_cache_min_tokens=4096 for Gemini 3.5/3.6/3.7 Flash and 3.1 Pro Preview 2026-08-19 14:13:00 -07:00
mateo-berri
fe26f5a541 fix(ui): clear the user search spinner when the box is emptied 2026-08-19 14:11:47 -07:00
mateo-berri
46a4eda19e Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_fix_bedrock_adaptive_thinking_token_accounting
# Conflicts:
#	litellm/llms/bedrock/chat/invoke_handler.py
#	litellm/responses/litellm_completion_transformation/transformation.py
#	tests/test_litellm/llms/bedrock/chat/test_converse_transformation.py
#	tests/test_litellm/responses/litellm_completion_transformation/test_reasoning_content_transformation.py
2026-08-19 14:10:53 -07:00
mateo-berri
74f12bf6ef fix(proxy): accept inherited model sentinels in project key limits 2026-08-19 14:09:09 -07:00
yuneng-jiang
3d51eb378a
refactor(ui): migrate the antd Alert call sites onto the shared Alert (#37513)
Moves all 33 antd Alert usages across 20 dashboard files onto
src/components/shared/Alert, following the composition the rest of the
dashboard already uses: message becomes AlertTitle, description becomes
AlertDescription, showIcon becomes a lucide icon child, and closable
becomes an AlertAction ghost button.

antd type="success" has no counterpart on the shared Alert, so the two
success sites land on the default variant with a CircleCheck icon, which
is what cloudzero_export_modal and CloudZeroIntegrationSettings already
do for the same case.

LoginPage's dismissible SSO notice moves into its own SsoEnabledNotice
component in the same file: antd's closable carried its own dismiss
state, and inlining it pushed LoginPageContent past the complexity
budget.

Four files lose their last antd symbol, so their no-restricted-imports
suppressions are pruned by hand. antd import sites drop from 115 to 111
across 107 to 103 files, and the no-restricted-imports ratchet drops
from 119 to 115 over 110 to 106 files.

One test asserted antd's own ant-alert-info class; it is repointed to
the shared Alert's text-info variant class, which keeps the same
"info, not warning" check. Every other colocated test passes untouched.
2026-08-19 21:07:50 +00:00
tin-berri
a613773fca
feat(auto-router)!: scope shadow eval jobs to multiple keys (#37251)
* feat(auto-router): scope shadow eval jobs to multiple keys

A shadow eval job now covers a set of keys instead of exactly one, and each
key carries its own max_turns budget, so one key exhausting its budget leaves
its siblings sampling. The existing job row already is the per-key unit
(api_key_id, max_turns, stopped_at, and the one-active-per-key-and-direction
partial unique index all live on it), so multi-key is grouping rather than
schema surgery: a new group_id column ties N sibling rows written atomically
by one create_many, the API's job id becomes the group id, and pre-existing
jobs backfill group_id = id so their ids keep resolving. The sampler hot path
is untouched; its test file has a zero-line diff

Results come back pooled plus a per-key breakdown and responses list every key
with its own budget, stop state and read-time labels. The dashboard is adapted
minimally to the new shapes (the picker stays single-key and submits a one-key
list); the multi-select picker and per-key table land in the stacked UI PR

* fix(shadow_eval): derive completed from spent budgets and record operator stops

* fix(shadow_eval): stamp stops atomically and freeze counts at the stamp

The stop endpoint wrote stopped_by and stopped_at as two separate updates, so
a failure between them left a job reading stopped while its unstamped legs
kept sampling, and the retry got 400 already stopped. One UPDATE now stamps
stopped_by and every missing stopped_at together, preserving the stopped_at a
leg earned from its own budget via COALESCE

Attempt counts now exclude attempts that land after a leg's stopped_at, so an
in-flight attempt finishing just after an operator stop can never push a
legacy pre-stopped_by job over its budget and flip it from stopped to
completed at read time

* fix(shadow_eval): backfill stopped_by so legacy stops never read as completions

* chore(ui): regenerate api types for the shadow eval stop fields

* fix(shadow_eval): let the stop statement pick one winner under racing stops

Two operators can both pass the derived-status guard in the race window. The
stop UPDATE now claims only legs with stopped_by still null and the endpoint
judges by its row count, so exactly one caller ever gets the 200 and the loser
gets the same already-stopped 400 a late caller gets

* refactor(shadow_eval): make the stop statement the whole state machine

The status guard ran before the UPDATE, so a stop racing the last budgeted
attempt still claimed the job and it read stopped forever instead of
completed. The statement now claims the job only while a leg still samples
inside the window with no stop recorded, and the endpoint reads once after
writing: a racing operator, a same-instant budget spend, and a repeat stop all
get the 400 naming the status the job actually holds. The pre-write guard and
the hand-built response go away

* chore(ui): regenerate api types for the stop route description
2026-08-19 14:02:15 -07:00
mateo-berri
51cafe4365 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_batch_empty_line_cost
# Conflicts:
#	type-discipline-budget.json
2026-08-19 13:54:57 -07:00