Commit graph

49968 commits

Author SHA1 Message Date
kerry
bba15b1382 style(responses_bridge): apply ruff format
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 23:52:53 +00:00
kerry
b7a9042f1b style(responses_bridge): suppress type-discipline flags with reasons
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 23:43:04 +00:00
kerry
9212124266 style(responses_bridge): use dict.get in text merge helper
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 23:37:45 +00:00
kerry
255ef3bc26 refactor(bedrock_mantle): build supported params without mutation
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 23:37:45 +00:00
kerry
438681be1a refactor(responses_bridge): share text merge helper
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 23:33:28 +00:00
kerry
e8f246bb6b fix(responses_bridge): forward verbosity as text.verbosity
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 23:32:32 +00:00
kerry
15c18ad6cd fix(bedrock_mantle): accept verbosity on gpt-5.x chat completions
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 23:27:07 +00:00
yuneng-jiang
765e6e498d
Merge pull request #41402 from BerriAI/litellm_/buildkite-litellm-e2e-setup-ff714d
feat(e2e): make the provider cache reusable across builds and mount Bedrock behind it
2026-09-16 16:12:29 -07:00
ryan-crabbe-berri
8bb4317c75
Merge pull request #41331 from BerriAI/litellm_ui_url_state_foundation
feat(ui): shared URL-state layer for tables and tabs
2026-09-16 16:07:45 -07:00
yuneng-jiang
030901211e
Merge pull request #41360 from BerriAI/litellm_unpin_together_ai_serverless_model
test(together_ai): move request-shape checks to the mapped file, drop the live ones
2026-09-16 16:07:33 -07:00
yuneng-jiang
fddf83a2ac
Merge pull request #41487 from BerriAI/litellm_isolate_generic_api_ndjson_test
test(logging): pick this test's own records out of the shared log batch
2026-09-16 16:06:47 -07:00
ryan-crabbe-berri
be6e7ef173
Merge pull request #41345 from BerriAI/litellm_remove_duplicate_user_budget_hook
fix(proxy): remove duplicate user budget hook that 429'd zero-cost models
2026-09-16 16:06:41 -07:00
yucheng-berri
659c4ce5a1
Merge pull request #40669 from BerriAI/litellm_otel_passthrough_trace_propagation
fix(otel): propagate W3C trace context on HTTP and WebSocket passthrough
2026-09-16 16:02:46 -07:00
Yuneng Jiang
939757ceff
Merge remote-tracking branch 'origin/litellm_/buildkite-litellm-e2e-setup-ff714d' into litellm_/buildkite-litellm-e2e-setup-ff714d 2026-09-16 16:02:09 -07:00
Yuneng Jiang
afb28540bb
fix(e2e): keep the CLI determinism test out of the in-cluster suite
It drives the real CLI for several seconds. The edge stamps every
upstream call with PYTEST_CURRENT_TEST, a process-global that names
whichever test the worker is in when the call arrives rather than the one
that made it, so a test that holds a worker that long collects other
tests' in-flight calls. Build 234's key report credits this test with 20
Bedrock and 7 Anthropic misses, and it makes no provider call at all.

Those misattributed calls take the wrong test id into the cache key and
write recordings under it, so the test was polluting the shared corpus it
exists to protect.

Deselected unless E2E_CLI_DETERMINISM is set, the same opt-in shape the
managed-files, prompt-caching and redis-chaos markers already use. The
attribution bug itself is older than this branch and is reported, not
fixed here.
2026-09-16 16:01:32 -07:00
yuneng-jiang
fa35300266
Merge branch 'main' into litellm_/buildkite-litellm-e2e-setup-ff714d 2026-09-16 16:00:45 -07:00
yuneng-jiang
ceaa0adef8
Merge branch 'main' into litellm_unpin_together_ai_serverless_model 2026-09-16 15:57:17 -07:00
yuneng-jiang
7f83650a41
Merge branch 'main' into litellm_isolate_generic_api_ndjson_test 2026-09-16 15:56:53 -07:00
ryan
789b8285bc Merge remote-tracking branch 'origin/main' into litellm_remove_duplicate_user_budget_hook 2026-09-16 22:56:19 +00:00
yuneng-jiang
680233fa8b
Merge pull request #41496 from BerriAI/litellm_/litellm-ci-failure-be9267
fix(prices): dedupe Nova cache_read_input_token_cost keys left by a text merge
2026-09-16 15:56:07 -07:00
ryan-crabbe-berri
d29753c521 fix(ui): let another tab's column save replace a toggle this tab could not save
A column toggle that localStorage refused was kept in memory and read ahead of storage, so a later save from another tab stayed hidden until this tab saved again. A storage event now drops the in-memory copy for its key, or all of them when another tab clears storage
2026-09-16 15:51:45 -07:00
ryan-crabbe-berri
adc937c493 fix(ui): read persisted column visibility from storage instead of a mounted copy
usePersistedColumnVisibility kept a useState copy seeded once at mount, so a later tableId or defaults change showed the old table's columns and saved them under the new key. It now reads localStorage through useSyncExternalStore, keeping only writes that storage refused in memory, so the hook has no copy to go stale. Stored choices are layered over the defaults on every read, and changes saved in another tab show up.
2026-09-16 15:51:45 -07:00
ryan-crabbe-berri
4d30bbce45 fix(ui): keep controlled client-side table pages across data reloads
Controlled client-mode DataTables no longer let TanStack reset the page index when rows change, since the owner of the pagination state decides the page. Once rows settle, a page past the end snaps back to the last page, matching server mode
2026-09-16 15:51:45 -07:00
ryan-crabbe-berri
eda81fff59 feat(ui): shared URL-state layer for tables and tabs
Add useUrlTableState (search, sort, page, page size and filter_<column>
in the query string via one nuqs useQueryStates call, with keyPrefix and
urlKeys for routes that host two tables or need legacy key names) and
useUrlTab (validated ?tab= param with a role-aware fallback). Migrate
the Virtual Keys table onto useUrlTableState with a byte-identical URL
contract and bind the Playground tab strip to ?tab=.

DataTable gains controlled columnVisibility/onColumnVisibilityChange
plus usePersistedColumnVisibility (localStorage per table id), and an
isError prop that keeps the server page clamp from rewriting a
deep-linked ?page= after a failed fetch. Virtual Keys uses both.

The expired-session redirect in handleError now keeps the query string
and hash so the return URL captured on re-login restores the filtered
view instead of the bare list.

Delete useTabRouting and tabRoutes, the pathname tab router left over
from the reverted path-per-tab attempt (#34327, reverted in #34629);
tab persistence has to be a query param on the static export.
2026-09-16 15:51:45 -07:00
ryan
dae16264c1 test(auth): cover over-budget user on zero-cost vs paid model in common_checks
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 15:42:15 -07:00
ryan
287bbaa6c1 fix(proxy): remove duplicate user budget hook that 429'd zero-cost models
_PROXY_MaxBudgetLimiter re-checked spend:user:{id} against user_max_budget in
async_pre_call_hook without the zero-cost model exemption that
_user_max_budget_check applies in auth, so free models were rejected with
"Max budget limit reached." once a user was over budget. Auth already owns
this check, so the hook is deleted rather than taught the exemption again

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 15:42:15 -07:00
Yuneng Jiang
b9751a38ab
Revert "fix(prices): dedupe Nova cache_read_input_token_cost keys left by a text merge"
This reverts commit a9fc6d255b.
2026-09-16 15:36:54 -07:00
ryan-crabbe-berri
9357491f91
Merge pull request #35418 from BerriAI/litellm_skills_ssh_sources
feat(ui): accept ssh clone urls when registering a skill
2026-09-16 15:36:18 -07:00
Yuneng Jiang
a9fc6d255b
fix(prices): dedupe Nova cache_read_input_token_cost keys left by a text merge
Six Amazon Nova entries define cache_read_input_token_cost twice, which
is what a clean text merge of two branches that both added the field
looks like. JSON parsers keep the last occurrence, so this turned
test_price_map_has_no_duplicate_keys red on every open PR's merge commit,
including this one, which touches neither file.

Both occurrences in all six entries carry the same value, so dropping the
later one leaves every parsed price identical.

Same change as #41496, carried here so this branch is not blocked on it.
Identical deletions, so the two merge cleanly in either order.
2026-09-16 15:15:42 -07:00
Yuneng Jiang
99545b5f26
Merge remote-tracking branch 'origin/main' into litellm_/buildkite-litellm-e2e-setup-ff714d 2026-09-16 15:15:22 -07:00
Yassin Kortam
a6e87526f8
Merge pull request #40843 from elifozdamar/litellm_fix_nonstream_parallel_slot_release
fix(proxy): release completed max-parallel slots promptly
2026-09-16 15:13:59 -07:00
Yuneng Jiang
76c0f8db1d
chore(e2e): report the key components behind a mount that never converges
Builds 232 and 233 held the Bedrock hit rate at 9% with the Claude Code
driver already sending byte-identical requests and headers, so something
between the proxy's ingress and the upstream still moves per build and
the flat key cannot say what.

Emit a digest per key component next to the counters: the test id, the
method, the URL, each keyed header, the whole body, and one digest per
top-level JSON body field. Values are digested, so no payload or
credential reaches the artifact. Diffing two builds' artifacts names the
field that moved.

Diagnostic, to be removed once it has answered.
2026-09-16 15:01:08 -07:00
Yuneng Jiang
a6b10ad654
fix(prices): dedupe Nova cache_read_input_token_cost keys left by a text merge
PR #41343 and PR #41112 both added cache_read_input_token_cost to the six amazon.nova-{micro,lite,pro}-v1:0 and us.amazon.nova-* entries, one at the top of each entry and one at the bottom. The merge kept both, so every PR now fails test_price_map_has_no_duplicate_keys. Both copies carried the same value, so this only removes the trailing duplicate in both price files
2026-09-16 14:58:57 -07:00
Yassin Kortam
f62f140f75
Merge pull request #41403 from BerriAI/litellm_key_total_spend
feat(proxy): expose lifetime total_spend on virtual keys
2026-09-16 14:49:02 -07:00
Yassin Kortam
2bfa10b6d9
Merge pull request #41311 from BerriAI/litellm_key_status_filter_deleted_lookup
feat(keys): filter /key/list by active, expired, revoked or deleted status and serve deleted keys from /key/info
2026-09-16 14:48:34 -07:00
Yassin Kortam
95abc9fb0b
Merge pull request #41330 from BerriAI/litellm_team_model_max_budget_v2
feat(team): team-level model_max_budget with key-level overrides
2026-09-16 14:48:29 -07:00
Yassin Kortam
314c0d71a5
Merge pull request #41327 from BerriAI/litellm_s3_log_prompts_only
feat(s3): add s3_log_prompts_only option to log prompts without responses
2026-09-16 14:48:18 -07:00
yujonglee
cc99ad00dd
Merge pull request #41489 from BerriAI/litellm_ocr_fileio_core
refactor(ocr): move file preparation from the python bridge into litellm-core
2026-09-16 14:45:09 -07:00
ryan-crabbe-berri
4439631e05
Merge pull request #41379 from runjivu/fix/fallback-budget-check
fix!: re-check budget on router fallback targets
2026-09-16 14:39:56 -07:00
Yassin Kortam
cd08c65002
Merge pull request #41425 from BerriAI/litellm_streaming_buffer_release_on_scan
feat(guardrails): release buffered stream chunks after each passing scan
2026-09-16 14:37:56 -07:00
Yassin Kortam
16bbff6643
Merge pull request #41474 from BerriAI/litellm_router_tpm_rpm_count_before_headers_v2
fix(router): count TPM/RPM usage before building rate-limit headers
2026-09-16 14:37:04 -07:00
Yuneng Jiang
0de187e76c style(test): annotate the new locals as Final 2026-09-16 14:34:13 -07:00
Yuneng Jiang
515bf8c9d5 refactor(test): validate cost-map entries into a typed model
The selector read raw cost-map dicts as `Mapping[str, Any]`. It now validates
each together_ai entry into a frozen Pydantic model and takes the two
capabilities as keyword booleans, so nothing in the helper is coarsely typed
or stringly addressed.
2026-09-16 14:32:08 -07:00
Mateo Wang
6edb549dbd
Merge pull request #41112 from BerriAI/litellm_registry_audit_2026_09_14
fix(models): rolling registry audit: Gemini latest aliases, Nova cache pricing, OpenRouter/Together sync, Mistral GLM 5.3, Azure snapshots, Grok caching
2026-09-16 14:31:50 -07:00
Yuneng Jiang
ba6c9fa61d fix(test): drop the redundant sys.path.insert
CI runs these lanes as `python -m pytest` from the repo root, so the root is
already on sys.path and `tests._live_test_helpers` imports without help. The
insert only tripped the TQ003 test-quality budget.
2026-09-16 14:26:16 -07:00
Yuneng Jiang
b478131701 test(together_ai): select the live model from the cost map
`together_ai/openai/gpt-oss-20b` was hardcoded in two live tests and is no
longer served, so both failed on a vendor catalog change rather than on
anything litellm did.

Both call sites now resolve the cheapest non-deprecated together_ai chat
entry at runtime, filtered on the capabilities the tests actually exercise,
mirroring what tests/e2e/llm_translation/test_together_ai_e2e.py already
does. The selector lives in tests/_live_test_helpers.py so both lanes share
one implementation.
2026-09-16 14:20:49 -07:00
ryan-crabbe-berri
17844cf14b Merge remote-tracking branch 'origin/main' into pr41379 2026-09-16 14:17:23 -07:00
ryan-crabbe-berri
cfe65f7b55 fix(proxy)!: enforce fallback budget by default
A budget bypass that ships off by default stays open for every deployment
that does not know to look for the flag, so `enforce_fallback_budget` now
defaults to true and `general_settings.enforce_fallback_budget: false` is
the opt-out for anyone who wants the old unguarded behaviour back.

BREAKING CHANGE: a paid fallback target is now refused for callers who are
over their key or user `max_budget`. Deployments relying on fallbacks to
keep serving over-budget callers must set enforce_fallback_budget: false.
2026-09-16 14:15:27 -07:00
kerry-berri
930ec9643a
Merge pull request #41446 from BerriAI/litellm_fix_anthropic_stream_served_model
fix(anthropic): carry the served model from message_start onto stream chunks
2026-09-16 14:14:54 -07:00
Yujong Lee
c621435ef7 refactor(ocr): move file preparation from the python bridge into litellm-core
Delete litellm/ocr/input.py and the native _ocr_file_document, _ocr_upload_document
and _ocr_mime_type helpers. File documents now project to a typed OcrDocumentInput
and the core lifecycle reads local paths, encodes bytes and asks the host to read
file-like objects through a ReadDocument operation before the provider request

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-16 21:02:30 +00:00