Commit graph

50181 commits

Author SHA1 Message Date
Mateo Wang
390ab45448
Merge pull request #37506 from BerriAI/litellm_dashscope_reasoning_effort
Some checks are pending
LiteLLM Rust / rust-wheel (push) Waiting to run
Unit Tests: Documentation Validation / documentation (push) Waiting to run
Unit Tests: Proxy DB Operations / budgets (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / assert-shard-coverage (push) Waiting to run
Unit Tests: Proxy DB Operations / auth-checks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / endpoints-and-responses (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / jwt-and-keys (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / key-generation (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / logging-misc (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-runtime (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-utils (push) Blocked by required conditions
Unit Tests / proxy-server (push) Waiting to run
Unit Tests: Proxy DB Operations / custom-logging (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / db-and-spend (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / guardrails-hooks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-server-core (push) Blocked by required conditions
Unit Tests / caching-local (push) Waiting to run
Unit Tests / core-utils (push) Waiting to run
Unit Tests / enterprise-package (push) Waiting to run
Unit Tests / enterprise-routing (push) Waiting to run
Unit Tests / integrations (push) Waiting to run
Unit Tests / All Other Providers (push) Waiting to run
Unit Tests / Vertex AI (push) Waiting to run
Unit Tests / misc (push) Waiting to run
Unit Tests / proxy-auth (push) Waiting to run
Unit Tests / proxy-endpoints (push) Waiting to run
Unit Tests / proxy-extras (push) Waiting to run
Unit Tests / proxy-infra (push) Waiting to run
Unit Tests / responses-caching-types (push) Waiting to run
GitHub Actions Security Analysis / zizmor (push) Waiting to run
fix(dashscope): forward reasoning_effort to the provider
2026-09-16 16:29:25 -07:00
mateo-berri
2f186055e6 fix(logging): decide keep-or-scrub for a log extra by comparing it to its scrubbed copy
Some checks are pending
ai-gateway image / ai-gateway release image (push) Waiting to run
The code-quality check refuses recursive functions and the walk that inspected
extras was one, so the filter no longer walks anything itself. safe_dumps now
builds its JSON-native structure through safe_json_structure, the filter scrubs
the extra through that, and the original object is kept only when the scrubbed
copy compares equal to it. Anything the serializer skipped (non-string keys,
nests past its depth, fields a repr hides) makes the copy differ, so the copy
wins. A host object whose equality raises, as numpy arrays and torch tensors
do, counts as changed instead of breaking the caller's log call
2026-09-16 16:29:23 -07:00
yujonglee
1c5d4d0be8
Merge pull request #41501 from BerriAI/litellm_1789598099-cache-redis
feat(rust): scaffold Redis cache crate
2026-09-16 16:28:54 -07:00
kerry
aea9678f61 refactor(streaming): compute the reasoning token cap once
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 23:28:28 +00:00
kerry
884e96958c fix(streaming): cap estimated reasoning tokens to the provider total and cover dict chunks
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 23:27:48 +00:00
Moe Khalil
1c9525fdab test(router): cover deployment replacement during metadata discovery
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 23:27:31 +00:00
kerry
15c18ad6cd fix(bedrock_mantle): accept verbosity on gpt-5.x chat completions
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 23:27:07 +00:00
ryan-crabbe-berri
ab92a6637d test(e2e): cover team admin editable fields on /team/update
Team admins are refused until a proxy admin enables a field, then limited to the enabled fields, and resending unchanged budget settings keeps the team's budget reset times
2026-09-16 16:26:52 -07:00
joshua-berri
41410e9556
Merge pull request #41364 from BerriAI/litellm_fix_mcp_auth_fail_closed_4501
fix(mcp): fail closed on missing upstream credentials
2026-09-16 23:26:38 +00:00
Moe Khalil
29d1a191a1 refactor(router): isolate per-deployment metadata refresh
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 23:24:03 +00:00
Moe Khalil
b7c6befb37 feat(router): discover token limits for hosted OpenAI-compatible models
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 23:19:54 +00:00
Yujong Lee
93ba409adf fix(rust): address Redis cache review findings
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 23:19:28 +00:00
mateo-berri
3712d8de92 Merge remote-tracking branch 'origin/main' into litellm_model_group_info_proxy_admin_all_models
Some checks failed
LiteLLM Rust / rust-wheel (push) Has been cancelled
LiteLLM Rust / rust-lint (push) Has been cancelled
LiteLLM Rust / rust-test (push) Has been cancelled
Terraform Provider / gofmt, vet, build, test (push) Has been cancelled
Terraform Provider / Provider endpoints vs proxy OpenAPI schema (push) Has been cancelled
2026-09-16 16:18:45 -07:00
mateo-berri
2a9fa48730 fix: expand wildcard deployments for proxy admins on /model_group/info 2026-09-16 16:18:44 -07:00
Yujong Lee
0b3c3885bc fix(rust): scope Redis dependency to cache crate
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 23:15:00 +00:00
Yujong Lee
8ae1f76339 feat(rust): scaffold redis cache crate
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 23:15:00 +00:00
mateo-berri
a1ad95dbbd fix(gemini): read the minimal thinking floor from the cost map and cover the /v1/messages bridge
Some checks failed
ai-gateway image / ai-gateway release image (push) Has been cancelled
LiteLLM Rust / rust-lint (push) Has been cancelled
LiteLLM Rust / rust-test (push) Has been cancelled
2026-09-16 16:13:41 -07:00
kerry
43514a7ffe Merge remote-tracking branch 'origin/main' into litellm_fix_interrupted_anthropic_reasoning_usage 2026-09-16 23:13:06 +00:00
kerry
02bccfd89f fix(streaming): fill text_tokens when reasoning is counted from stream content
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 23:13:05 +00:00
ryan-crabbe-berri
249a23b09c chore(ui): prune stale eslint suppressions for projects page 2026-09-16 16:12:48 -07:00
ryan-crabbe-berri
ef8e066c77 fix(ui): address review on orgs-projects url state
Keep /projects list paging as a pushed history entry, validate the
project key table page size against its offered options, and clear the
key table params through the table-state setters instead of a copied
key list.
2026-09-16 16:12:48 -07:00
ryan-crabbe-berri
df2dd9b7f2 feat(ui): keep projects search and project key table state in the URL
The projects list search lives in ?project_search= and its pagination now goes through useUrlTableState, keeping the page and page_size keys. The key table inside a project reads keys_search, keys_page and keys_page_size, resets to its first page on a new search, and no longer snaps a deep-linked page while the key fetch is failing. Closing a project drops its keys_ params so they do not leak into the next project
2026-09-16 16:12:47 -07:00
ryan-crabbe-berri
417cc5c4fb feat(ui): keep organizations list and detail tab state in the URL
The organizations list now reads its search (org_search), org ID filter (filter_org_id), sort (sort_by, sort_order) and pagination (page, page_size) from the URL through useUrlTableState. The detail view tabs are controlled by ?org_tab=, and the Edit row action opens ?org=<id>&org_tab=settings in one history entry instead of passing an editOrg flag
2026-09-16 16:12:47 -07:00
yuneng-jiang
765e6e498d
Merge pull request #41402 from BerriAI/litellm_/buildkite-litellm-e2e-setup-ff714d
feat(e2e): make the provider cache reusable across builds and mount Bedrock behind it
2026-09-16 16:12:29 -07:00
ryan-crabbe-berri
81ae5caa7e fix(proxy): pass only a team admin's changed fields on to the team update 2026-09-16 16:11:52 -07:00
yassin
acd4f0eb04 fix(spend_tracking): attribute router-rejected requests to the model group provider
A request for a configured model group that the router rejects before picking a deployment (all deployments in cooldown, no healthy deployment) never gets a custom_llm_provider in its logging kwargs. The spend log payload persisted an empty provider, the daily spend tables carried it through, and the Admin UI Usage page rendered those requests under unknown even though every model in the group has a provider

get_logging_payload now takes the proxy router and, when the logged provider is missing, infers it from the model group's deployments. It only attributes when every deployment in the group resolves to the same provider; mixed groups, unknown groups and a missing router leave the value empty as before. Explicitly logged providers keep precedence

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 23:10:47 +00:00
mateo-berri
610249c663 fix(bedrock): carry the tool_call_id into the rewritten tool call text 2026-09-16 16:08:46 -07:00
ryan-crabbe-berri
8bb4317c75
Merge pull request #41331 from BerriAI/litellm_ui_url_state_foundation
feat(ui): shared URL-state layer for tables and tabs
2026-09-16 16:07:45 -07:00
yuneng-jiang
030901211e
Merge pull request #41360 from BerriAI/litellm_unpin_together_ai_serverless_model
test(together_ai): move request-shape checks to the mapped file, drop the live ones
2026-09-16 16:07:33 -07:00
yuneng-jiang
fddf83a2ac
Merge pull request #41487 from BerriAI/litellm_isolate_generic_api_ndjson_test
test(logging): pick this test's own records out of the shared log batch
2026-09-16 16:06:47 -07:00
ryan-crabbe-berri
be6e7ef173
Merge pull request #41345 from BerriAI/litellm_remove_duplicate_user_budget_hook
fix(proxy): remove duplicate user budget hook that 429'd zero-cost models
2026-09-16 16:06:41 -07:00
yucheng-berri
659c4ce5a1
Merge pull request #40669 from BerriAI/litellm_otel_passthrough_trace_propagation
fix(otel): propagate W3C trace context on HTTP and WebSocket passthrough
2026-09-16 16:02:46 -07:00
Yuneng Jiang
939757ceff
Merge remote-tracking branch 'origin/litellm_/buildkite-litellm-e2e-setup-ff714d' into litellm_/buildkite-litellm-e2e-setup-ff714d 2026-09-16 16:02:09 -07:00
mateo-berri
02d9aae8c8 fix(proxy): stop forwarding LiteLLM credential headers on Bedrock agent-runtime passthrough
The agent-runtime branch of /bedrock/{endpoint} (agents, knowledgebases, flows,
retrieveAndGenerate, rerank, generateQuery, optimize-prompt) forwarded every
caller header to AWS next to the SigV4 signature, so a LiteLLM key presented in
x-api-key or x-litellm-api-key reached bedrock-agent-runtime verbatim. Build the
upstream header set explicitly: drop LiteLLM credential headers by name and any
authenticated secret by value, keep the remaining caller headers, and let the
signed headers win on collisions.
2026-09-16 16:01:52 -07:00
Yuneng Jiang
afb28540bb
fix(e2e): keep the CLI determinism test out of the in-cluster suite
It drives the real CLI for several seconds. The edge stamps every
upstream call with PYTEST_CURRENT_TEST, a process-global that names
whichever test the worker is in when the call arrives rather than the one
that made it, so a test that holds a worker that long collects other
tests' in-flight calls. Build 234's key report credits this test with 20
Bedrock and 7 Anthropic misses, and it makes no provider call at all.

Those misattributed calls take the wrong test id into the cache key and
write recordings under it, so the test was polluting the shared corpus it
exists to protect.

Deselected unless E2E_CLI_DETERMINISM is set, the same opt-in shape the
managed-files, prompt-caching and redis-chaos markers already use. The
attribution bug itself is older than this branch and is reported, not
fixed here.
2026-09-16 16:01:32 -07:00
yuneng-jiang
fa35300266
Merge branch 'main' into litellm_/buildkite-litellm-e2e-setup-ff714d 2026-09-16 16:00:45 -07:00
mateo-berri
8045794c37 Merge remote-tracking branch 'origin/main' into litellm_dashscope_reasoning_effort 2026-09-16 15:59:08 -07:00
yuneng-jiang
ceaa0adef8
Merge branch 'main' into litellm_unpin_together_ai_serverless_model 2026-09-16 15:57:17 -07:00
yuneng-jiang
7f83650a41
Merge branch 'main' into litellm_isolate_generic_api_ndjson_test 2026-09-16 15:56:53 -07:00
ryan-crabbe-berri
36eb9cdb35 test(proxy): drop the route-list membership test that the PATCH gate tests already cover 2026-09-16 15:56:27 -07:00
ryan
789b8285bc Merge remote-tracking branch 'origin/main' into litellm_remove_duplicate_user_budget_hook 2026-09-16 22:56:19 +00:00
yuneng-jiang
680233fa8b
Merge pull request #41496 from BerriAI/litellm_/litellm-ci-failure-be9267
fix(prices): dedupe Nova cache_read_input_token_cost keys left by a text merge
2026-09-16 15:56:07 -07:00
ryan-crabbe-berri
d29753c521 fix(ui): let another tab's column save replace a toggle this tab could not save
A column toggle that localStorage refused was kept in memory and read ahead of storage, so a later save from another tab stayed hidden until this tab saved again. A storage event now drops the in-memory copy for its key, or all of them when another tab clears storage
2026-09-16 15:51:45 -07:00
ryan-crabbe-berri
adc937c493 fix(ui): read persisted column visibility from storage instead of a mounted copy
usePersistedColumnVisibility kept a useState copy seeded once at mount, so a later tableId or defaults change showed the old table's columns and saved them under the new key. It now reads localStorage through useSyncExternalStore, keeping only writes that storage refused in memory, so the hook has no copy to go stale. Stored choices are layered over the defaults on every read, and changes saved in another tab show up.
2026-09-16 15:51:45 -07:00
ryan-crabbe-berri
4d30bbce45 fix(ui): keep controlled client-side table pages across data reloads
Controlled client-mode DataTables no longer let TanStack reset the page index when rows change, since the owner of the pagination state decides the page. Once rows settle, a page past the end snaps back to the last page, matching server mode
2026-09-16 15:51:45 -07:00
ryan-crabbe-berri
eda81fff59 feat(ui): shared URL-state layer for tables and tabs
Add useUrlTableState (search, sort, page, page size and filter_<column>
in the query string via one nuqs useQueryStates call, with keyPrefix and
urlKeys for routes that host two tables or need legacy key names) and
useUrlTab (validated ?tab= param with a role-aware fallback). Migrate
the Virtual Keys table onto useUrlTableState with a byte-identical URL
contract and bind the Playground tab strip to ?tab=.

DataTable gains controlled columnVisibility/onColumnVisibilityChange
plus usePersistedColumnVisibility (localStorage per table id), and an
isError prop that keeps the server page clamp from rewriting a
deep-linked ?page= after a failed fetch. Virtual Keys uses both.

The expired-session redirect in handleError now keeps the query string
and hash so the return URL captured on re-login restores the filtered
view instead of the bare list.

Delete useTabRouting and tabRoutes, the pathname tab router left over
from the reverted path-per-tab attempt (#34327, reverted in #34629);
tab persistence has to be a query param on the static export.
2026-09-16 15:51:45 -07:00
ryan-crabbe-berri
c37d0a2e66 fix(proxy): read general_settings without a cast and test the /team/update gate by behavior only 2026-09-16 15:51:04 -07:00
kerry
4b70696afa fix(streaming): estimate interrupted Anthropic stream usage from reasoning_content
Interrupted Anthropic streams that die before message_delta were billed at
the message_start placeholder (any value above 1 was trusted) or at 0 when
the partial response was reasoning-only, because the token_counter fallback
only looked at visible text. Reset the placeholder whenever no finish_reason
or second usage event arrived, fold the already-counted reasoning tokens into
the fallback estimate, and drop the stale completion_tokens_details so cost
is computed from the recovered count

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 22:49:25 +00:00
mateo-berri
ebae692a0d refactor(responses): drop the api_base cast and mark the header merge mutable-ok
Some checks failed
Terraform Modules / fmt, validate, test (aws) (push) Has been cancelled
Terraform Modules / fmt, validate, test (gcp) (push) Has been cancelled
LiteLLM Rust / rust-lint (push) Has been cancelled
LiteLLM Rust / rust-test (push) Has been cancelled
LiteLLM Rust / rust-wheel (push) Has been cancelled
Terraform Provider / gofmt, vet, build, test (push) Has been cancelled
Terraform Provider / Provider endpoints vs proxy OpenAPI schema (push) Has been cancelled
2026-09-16 15:45:50 -07:00
ryan
dae16264c1 test(auth): cover over-budget user on zero-cost vs paid model in common_checks
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 15:42:15 -07:00