Commit graph

46618 commits

Author SHA1 Message Date
Yuneng Jiang
4a3dcd5e8e
refactor(ui): give the collapsed log drawer its own sidebar toggle
Open, the trace sidebar is byte-identical to before: the toggle sits over its
header exactly where it did, and the header keeps the padding that makes room
for it. Collapsed, that button has nowhere to live, so the drawer header shows
one instead, on the model row or the request id row when the log names no
model.

Both come from SidebarToggle, so they cannot drift in design. The chevrons now
point the way the sidebar will move: right while it is open, left while it is
collapsed.
2026-08-29 11:53:14 -07:00
Mateo Wang
c24f821652
Merge pull request #34849 from BerriAI/litellm_keyless_key_managed_resource_owner
fix(managed resources): let keys with no user_id or team_id read their own batches and files
2026-08-29 11:50:00 -07:00
ryan-crabbe-berri
e8994e8ce1 Merge branch 'litellm_window_spend_writer' into litellm_window_spend_reader 2026-08-29 11:49:42 -07:00
Yuneng Jiang
20e6d6457a
refactor(ui): show the log drawer's sidebar toggle only where it has a row
Putting the toggle in the drawer header unconditionally stranded it on its own
line: the model row renders empty for a log that names no model or provider, so
the chevron sat alone above the request id.

The sidebar keeps the toggle whenever it is open, in its own header. Collapsed,
the toggle moves into the drawer header and joins the model row, or the request
id row when there is no model to join. Shared between both through
SidebarToggle so the two call sites cannot drift.
2026-08-29 11:48:04 -07:00
ryan-crabbe-berri
4e22a5ef5a Merge branch 'litellm_window_spend_schema' into litellm_window_spend_writer 2026-08-29 11:47:37 -07:00
mateo-berri
f2f988bd57 test: cover forged _headroom_interception_converted_stream strip at the proxy boundary 2026-08-29 11:46:23 -07:00
Mateo Wang
5e60ec5c31
Merge pull request #38743 from BerriAI/litellm_techdebt_20260829
refactor: clean up tech debt that landed on 2026-08-29
2026-08-29 11:46:05 -07:00
mateo-berri
6d3e687ce4 fix(db): let the writer pin yield to the replica while the writer is degraded 2026-08-29 11:44:45 -07:00
Yuneng Jiang
b27a1a13a2
refactor(ui): move the trace sidebar toggle into the log drawer header
The collapsed rail kept the toggle in flow but left a 40px stub of empty
sidebar on screen. The toggle now leads the drawer header's first row, ahead of
the provider logo and the model name, so it reads as part of the header and the
sidebar goes back to unmounting when collapsed.

Still no absolute positioning and no stacking level: the button is a normal
in-flow child of the header row it sits in. DrawerHeader takes the collapsed
state and the toggle handler as props rather than reaching for the drawer's
state.
2026-08-29 11:42:45 -07:00
ryan-crabbe-berri
65e535e089 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_window_spend_schema 2026-08-29 11:42:39 -07:00
yuneng-jiang
06852ca290
Merge pull request #38779 from BerriAI/litellm_/release-version-bump-71ce15
chore: bump litellm-enterprise 0.1.61 -> 0.1.62, litellm-proxy-extras 0.4.90 -> 0.4.91
2026-08-29 11:42:18 -07:00
yuneng-jiang
207893db9c
Merge pull request #38778 from BerriAI/litellm_/json-readability-logs-661421
fix(ui): make the logs JSON viewer follow the theme in dark mode
2026-08-29 11:42:04 -07:00
mateo-berri
9e01bd1441 fix(azure): send the deployment name as the body model on v1 image routes 2026-08-29 11:41:58 -07:00
ryan-crabbe-berri
144fbe9fff Merge branch 'litellm_window_spend_writer' into litellm_window_spend_reader 2026-08-29 11:37:11 -07:00
ryan-crabbe-berri
21b201d526 Merge branch 'litellm_window_spend_schema' into litellm_window_spend_writer 2026-08-29 11:37:09 -07:00
ryan-crabbe-berri
a6273bb332 chore(migrations): drop the generated comment from the budget window spend migration 2026-08-29 11:36:59 -07:00
Yuneng Jiang
34355778e6
bump: litellm-enterprise 0.1.61 -> 0.1.62, litellm-proxy-extras 0.4.90 -> 0.4.91 2026-08-29 11:33:32 -07:00
mateo-berri
68f891fd2b Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_keyless_key_managed_resource_owner 2026-08-29 11:33:08 -07:00
Yuneng Jiang
2e3ae43b6f
Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_/json-readability-logs-661421 2026-08-29 11:31:08 -07:00
mateo-berri
a8c36e8307 Merge remote-tracking branch 'origin/litellm_internal_staging' into devin_ai_fix_model_new_read_replica_lag_38556 2026-08-29 11:29:07 -07:00
Mateo Wang
fee8619708 test: add batch request count keys to gcs pub sub spend logs fixture 2026-08-29 11:26:39 -07:00
yuneng-jiang
fa25ff2a2e
Merge pull request #38626 from BerriAI/litellm_ui_model_links_team_key_info
feat(ui): link team and key model chips to the models page filtered to that group
2026-08-29 11:26:15 -07:00
mateo-berri
b65592e623 test: drop stray trailing blank line in test_main.py 2026-08-29 11:18:19 -07:00
mateo-berri
c254605e92 test(embeddings): move encoding_format default coverage to wire-level assertions
Consolidate the new regression tests into
test_openai_embedding_encoding_format_default.py, replacing mocks that
pinned the old float default with respx captures of the request body,
and update the stale local_testing default-float test to assert
omission
2026-08-29 11:10:21 -07:00
mateo-berri
e22744c439 fix(embeddings): omit encoding_format when the client omits it on OpenAI-compatible calls
When no encoding_format is set on the call, the model config, or
LITELLM_DEFAULT_EMBEDDING_ENCODING_FORMAT, leave the field out of the
upstream request instead of defaulting to float, and bypass the OpenAI
SDK's own base64 default so nothing re-adds it on the wire. Downstreams
that reject encoding_format, such as a second LiteLLM proxy fronting
Bedrock Titan embeddings, now work when the client omits the field.

Fixes #38661
2026-08-29 11:06:29 -07:00
ryan-crabbe-berri
7745fe887f Merge branch 'litellm_window_spend_writer' into litellm_window_spend_reader 2026-08-29 11:06:05 -07:00
ryan-crabbe-berri
041cae8280 fix(proxy): bound the window spend seed exclusion to the batch's own start time
request_id can be chosen by the client through x-litellm-call-id, so an
unbounded NOT (request_id = ANY(batch)) let a replayed old id drop that id's
historical LiteLLM_SpendLogs row from the one-time seed while its increment
still landed. The increment now carries the request start, the batch keeps
the earliest one, and the seed only excludes ids whose startTime is at or
after it.
2026-08-29 11:05:57 -07:00
Yuneng Jiang
fcd6ea46ce
refactor(ui): keep the log drawer's trace sidebar in flow as a collapsed rail
The toggle was absolutely positioned over the drawer's flex row, owned by
neither column. That forced two coupled workarounds: a stacking level so it
could beat whatever it landed on, and pl-12 on the sidebar header to reserve
space for a button that was not its child.

The sidebar column now always renders, at 224px expanded and a 40px rail
collapsed, and the toggle is a normal in-flow child of the column it controls.
No absolute, no z-index, no reserved padding, and nothing that can paint over
the button. It also stops the toggle from clipping the provider logo, which it
did in the collapsed state even before the z-index scale landed.

Costs 40px of drawer width while collapsed.
2026-08-29 11:00:47 -07:00
tin-berri
2a5d09ee87
fix(policy): let the AI policy suggester drop sampling params its model refuses (#38594)
The suggester pins temperature=0.2 for tool-selection determinism and passed no
drop_params, so an operator-supplied reasoning model whose only accepted temperature is 1
made litellm raise UnsupportedParamsError and the whole suggestion fail. The default
gpt-4o-mini is unaffected; the failure needs the caller to name a model.

Every other internal LLM call the proxy makes on a user's behalf already opts in through
judge_acompletion, which sets drop_params=True on both dispatch paths. This was the one
caller outside that contract, so the sampling preference is now advisory here too and the
call degrades instead of dying.

Resolves LIT-6352
2026-08-29 10:58:27 -07:00
ryan-crabbe-berri
80e0bc2dc6 docs(claude.md): require tests to check behavior, not code structure
Claude-Session: https://claude.ai/code/session_017nZW6omb93ZuAfqqCzKSU5
2026-08-29 10:58:10 -07:00
Mateo Wang
c3edb95e8d Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_managed_batches_observability
# Conflicts:
#	tests/test_litellm/proxy/spend_tracking/test_spend_management_endpoints.py
2026-08-29 10:56:02 -07:00
Yuneng Jiang
4c0bb1226c
fix(ui): make the logs JSON viewer follow the theme in dark mode
The request and response tree passed the library's light palette in every
theme, so in dark mode the string values rendered dark green and the
punctuation rendered black on a near black surface. Pick the palette from
the resolved theme instead, and let the tree inherit the themed surface
rather than painting the library's own background.
2026-08-29 10:54:48 -07:00
mateo-berri
acb621c35b Merge remote-tracking branch 'origin/litellm_internal_staging' into devin_ai_fix_model_new_read_replica_lag_38556 2026-08-29 10:54:46 -07:00
mateo-berri
2457e60cfc Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_lit_4913_headroom_streaming_ccr 2026-08-29 10:54:24 -07:00
ryan-crabbe-berri
08118e6246 fix(proxy): let the exact model= filter match team BYOK public names
Team-scoped deployments keep the internal model_name_{team_id}_{uuid} routing key and expose the public name in model_info.team_public_model_name. The dashboard links team model chips with the public name, so the exact filter now matches either name via the existing helper.
2026-08-29 10:52:15 -07:00
ryan-crabbe-berri
e7d99079de Merge branch 'litellm_window_spend_writer' into litellm_window_spend_reader 2026-08-29 10:51:50 -07:00
ryan-crabbe-berri
83cedfd2b2 refactor(proxy): type the budget window reset_at passed to the window spend enqueue 2026-08-29 10:51:38 -07:00
Yuneng Jiang
8278a92a06
fix(ui): let the log drawer's trace sidebar expand again once collapsed
The sidebar toggle sits absolutely positioned over the drawer's flex row. With
the sidebar expanded it lands on the sidebar header, but once collapsed it lands
on the drawer header, which is sticky at z-chrome (10). The named-z-scale
refactor moved the toggle from z-20 to z-raised (1), so from then on the header
painted over it and swallowed the click: collapse the trace list and there was
no way to bring it back.

Moves the toggle to z-floating (30) and folds the two mirrored buttons into one,
since they only ever differed by icon, label and handler.

Covered by a Playwright spec, which is the tier that can see the layering: the
button stays visible and enabled either way, so the pre-fix failure is a click
interception that jsdom cannot reproduce.
2026-08-29 10:42:48 -07:00
yujonglee
352789257d
perf(rust): use pythonize for bridge serialization (#38764)
* perf(rust): use pythonize for bridge serialization

* perf(rust): benchmark serialization payload sizes

* refactor(rust): remove mutable benchmark state
2026-08-29 10:41:10 -07:00
ryan-crabbe-berri
9beb5ead4d fix(proxy): keep the exact model= DB predicate within the type-discipline budget
The where clause now uses the exact name string directly and skips the DB query when the typed search cannot occur in that name, so no new mutable literals are added (LIT002 gate).
2026-08-29 10:40:39 -07:00
ryan-crabbe-berri
e585aab3ea
Merge pull request #38771 from BerriAI/litellm_/api-reference-dark-mode-4ec67f
fix(ui): make code blocks follow the theme in dark mode
2026-08-29 10:39:25 -07:00
ryan-crabbe-berri
3e99ee8d0e fix(proxy): scope the DB-side model search by the exact model= filter
With model=<group>&search=<term>, the router list was narrowed to the group but the DB query only matched the substring, so other groups' rows leaked into the page and total_count.
2026-08-29 10:32:29 -07:00
ryan-crabbe-berri
4411562a0f fix(ui): satisfy lint and the modelInfoCall arity in the models hook tests 2026-08-29 10:32:29 -07:00
Yuneng Jiang
8f07a12726
Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_/api-reference-dark-mode-4ec67f 2026-08-29 10:25:55 -07:00
devin-ai-integration[bot]
30efcfd684
feat(mcp): support asymmetric (RS256) signing for MCP gateway session tokens (#38728)
* feat(mcp): support RS256 signing for MCP gateway session tokens

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* style: ruff format session token modules

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(mcp): enforce key strength on rotated public keys and unique kids

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-29 10:18:59 -07:00
ryan-crabbe-berri
fe63ebdb19 fix(ui): filter the models page by exact model group instead of substring search
Pass the selected group as the exact model= param on /v2/model/info rather than as the substring search, so a group like gpt-4 no longer pulls gpt-4o rows into the page and count. Drop two comments that restated helper behavior.
2026-08-29 10:18:06 -07:00
devin-ai-integration[bot]
0de1825450
fix(health): honor allow_requests_on_db_unavailable in readiness probe (#37640)
* fix(health): honor allow_requests_on_db_unavailable in readiness probe

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(health): bound readiness DB check and pass reconnect timeout

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(health): bound whole readiness DB check with one deadline

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(health): keep readiness deadline fallback within lint budgets

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(health): suppress TQ008 for proxy-global readiness patches

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(db): release reconnect lock when a waiting reconnect is cancelled

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: milan <milan@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: yassin <yassin@berri.ai>
2026-08-29 10:17:12 -07:00
Yuneng Jiang
03f64200c2
fix(ui): stop CodeBlock painting a hardcoded light background in dark mode
The customStyle background only reaches the <pre>, so the prism theme's own
background stayed on the inner <code>. In dark mode that left a white card
with a dark box hugging every line. Let the block inherit the themed surface
from its wrapper instead.
2026-08-29 10:15:05 -07:00
Mateo Wang
cb7d41a5c6
Merge pull request #38739 from BerriAI/litellm_fix_tag_routing_reads_merged_metadata_tags
fix(proxy): tag routing misses proxy-merged tags when chat requests carry litellm_metadata
2026-08-29 10:14:05 -07:00
Mateo Wang
ed2f1836de
Merge pull request #38738 from BerriAI/litellm_fix_batch_list_page_past_unparseable_rows
fix(batches): fill a managed batch page past rows that will not parse
2026-08-29 10:13:29 -07:00