Commit graph

42990 commits

Author SHA1 Message Date
yuneng-jiang
9f3b1dfec5
Merge pull request #36787 from BerriAI/litellm_osv_nanoid_3_3_18
build(deps): bump nanoid to 3.3.18 to clear osv-scan
2026-08-13 11:09:35 -07:00
Mateo Wang
0a73e0f8d9
Merge pull request #36792 from BerriAI/litellm_gemini_3_7_flash
feat(gemini): day-0 pricing for gemini-3.7-flash
2026-08-13 10:28:16 -07:00
mateo-berri
d9020795dd test: use _local_model_cost_map fixture in gemini 3.7 flash pricing tests 2026-08-13 10:13:40 -07:00
Yuneng Jiang
bc9005c149
refactor(ui): migrate SectionHeader and ToolsSection to shadcn 2026-08-13 10:04:38 -07:00
mateo-berri
d3d259b211 feat(gemini): day-0 pricing for gemini-3.7-flash 2026-08-13 09:49:55 -07:00
Devin AI
630a1d0b69 build(deps): bump nanoid to 3.3.18 to clear osv-scan
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-13 16:38:24 +00:00
Yuneng Jiang
eb23dc2e81
test(ui): pin SectionHeader and ToolsSection behaviour before migration 2026-08-13 09:37:43 -07:00
yuneng-jiang
9d069f21dc
refactor(ui): migrate guardrails content tables to shared DataTable (#36708)
* test(ui): characterize guardrails content tables

* refactor(ui): migrate guardrails content tables to shared DataTable

* test(ui): verify shared guardrails table integration
2026-08-13 09:21:00 -07:00
yuneng-jiang
b8241d6664
refactor(ui): migrate guardrails monitor table to shared DataTable (#36709)
* test(ui): characterize guardrails monitor table

* refactor(ui): migrate guardrails monitor table to shared DataTable
2026-08-13 09:20:48 -07:00
yuneng-jiang
bfa34e2b10
refactor(ui): migrate usage tables to shared DataTable (#36707)
* test(ui): decouple usage table test from antd

* refactor(ui): migrate usage tables to shared DataTable

* test(ui): preserve data utility exports in usage tests
2026-08-13 09:20:45 -07:00
mateo-berri
08ae8648e0 fix: restore tolerant auth attribute access in compact summary checks
The summary model access and budget checks read team_id, user_id,
project_id, and end_user_id via getattr with a None default so duck-typed
auth objects without those attributes keep working
2026-08-13 08:24:23 -07:00
yassin
6346497498 fix(ui): add nvidia riva to the model provider list
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-13 13:01:40 +00:00
mateo-berri
6b5249bcce refactor: replace Any with precise types across responses, proxy, and llms modules
Types 28 files with Protocols, TypedDicts, and Pydantic validation in place
of Any, cutting basedpyright reportAny by 974 and reportExplicitAny by 261
(1411 errors total across 48 rules), and ratchets the basedpyright, ruff
strict, and type discipline budgets down to match
2026-08-13 03:59:26 -07:00
yucheng-berri
09889e1986
fix(langfuse): coerce header-sourced mask and trace-update steering values (#36740)
langfuse_* request headers land in metadata as strings, but the trace path reads
mask_input/mask_output with a bare truthiness check and iterates update_trace_keys
directly. A header saying mask_input: false redacted the payload it was asked to
keep, and update_trace_keys was walked one character at a time so every requested
key silently failed to match
2026-08-13 00:44:26 -07:00
yuneng-jiang
a7397b2459
fix(helm): render nodeSelector on the migrations job (#36747)
The template rendered affinity and tolerations but never nodeSelector, so a
values file that pinned the chart to a node pool got the gateway and every
subchart placed correctly while the migration Job silently fell through to
whatever the cluster's default pool was.

That is worse than an outright failure. On EKS Auto Mode the default pool hands
out 3 GiB nodes and the migration container needs roughly 3.6 GB, so the Job
was OOM-killed on a pool it was never meant to run on, while the values file
that would have placed it on a large enough node looked correct.

The new test fails against the old template with "unknown path
spec.template.spec.nodeSelector".
2026-08-13 00:41:36 -07:00
yucheng-berri
6f84c468d4
fix(guardrails): scan and re-emit raw Anthropic SSE streams in the bedrock post-call hook (#36598)
* fix(guardrails): scan and re-emit raw Anthropic SSE streams in the bedrock post-call hook

* fix(guardrails): keep upstream id and model on a blocked Anthropic stream

* fix(guardrails): deliver a blocked Anthropic stream as an error frame

* fix(guardrails): deliver an unscannable Anthropic stream as an error frame

* fix(guardrails): emit the guardrail block detail as JSON in the stream error frame

* fix(guardrails): deliver an Anthropic block through the shared block-SSE builder

* fix(guardrails): keep the shared SSE assembler behavior-identical for existing callers

* fix(guardrails): keep the stream error message a string and drop an unreachable branch

* chore(guardrails): drop a comment that repeated its own docstring

* fix(guardrails): let bedrock service failures keep their status instead of framing them as blocks

* fix(guardrails): key the streamed block decision on status, not detail shape

InvokeGuardrailChecks details a Mapping on its 500 for an unparseable response,
so a detail-shape test read that outage as a policy block and framed it as a 200
guardrail_error. Both block sites raise 400, so gate on the status too.

* refactor(guardrails): narrow the SSE error-frame helper to the input it actually takes

Both callers pass a string, so the Mapping overload and its json.dumps branch
were unreachable. Folds the block branch's narrative comment into the rebind
suppressions that already carry a reason.
2026-08-13 07:23:49 +00:00
yuneng-jiang
3d76dfc72e
refactor(ui): migrate search and user controls to shadcn (#36694)
* test(ui): characterize shared migration surfaces

* refactor(ui): migrate search and user controls

* fix(ui): restore search tool clear action
2026-08-13 06:43:30 +00:00
yuneng-jiang
fd00b98f64
refactor(ui): migrate guardrails-monitor, projects, logs to shadcn (#34606)
* test(ui): pin behaviour of guardrails-monitor, projects and logs components before migration

Adds role- and text-based characterisation tests for EvaluationSettingsModal,
GuardrailDetail and AuditLogDrawer, which had none, and moves the remaining
antd-specific assertions (.ant-spin, the icon role of an antd Spin indicator)
onto library-neutral ARIA queries. Also covers the enterprise banner on the
deleted keys and deleted teams pages, which no test reached.

All of these pass against the current antd and Tremor components.

* refactor(ui): migrate guardrails-monitor, projects and logs to shadcn

Replaces antd and Tremor with installed shadcn primitives across the files
these three routes exclusively own. Markup only, except where noted below.

Deletes AntDLoadingSpinner, an antd-only primitive living in the shadcn ui/
folder, and moves its single call site onto ui/ui-loading-spinner.

Two behaviour notes. The logs tab handler previously mapped every tab past
the first to "audit logs", so the audit panel kept polling while Deleted
Keys or Deleted Teams was on screen; each tab now reports its own value and
panels stay mounted via keepMounted. The evaluation settings dialog is
bounded to the viewport and scrolls internally, which the antd Modal got
from being top-anchored on a scrolling page.

The tests added in the previous commit pass unedited against these
components.
2026-08-13 06:43:12 +00:00
yuneng-jiang
4df421e058
refactor(ui): migrate guardrail and duration controls to shadcn (#36693)
* test(ui): characterize shared migration surfaces

* refactor(ui): migrate guardrail and duration controls

* fix(ui): preserve duration select callback shape

* fix(ui): narrow duration selection value
2026-08-13 06:42:42 +00:00
yuneng-jiang
e619106306
refactor(ui): migrate team detail controls to shadcn (#36695)
* test(ui): characterize shared migration surfaces

* refactor(ui): migrate team detail controls
2026-08-13 06:41:28 +00:00
Mateo Wang
73e555a3e3
test(interactions): follow Google spec drift replacing Turn with typed steps (#36730)
* test(interactions): follow Google spec drift replacing Turn with typed steps

* test(interactions): send step and content-list input to the live Gemini API
2026-08-12 23:32:57 -07:00
Yuneng Jiang
266a35b88c
test(ui): cover keyboard activation of the log drawer collapse rows
The migration turned each collapse row into a real button, but the existing
tests only click, so a regression in Enter or Space activation would still
pass. Add one test per component that tabs to the row, expands with Enter and
collapses with Space, asserting visibility rather than markup.

Both fail against the antd version and pass against the migrated one.
2026-08-12 22:39:29 -07:00
Yuneng Jiang
f17143086a
refactor(ui): migrate TruncatedValue and OutputCard to shadcn
TruncatedValue swaps antd Tooltip and Typography for the shadcn Tooltip
and the shared CopyButton, so the full value now lives in the page and is
clipped with CSS rather than shortened in the text node.

OutputCard drops antd Typography for token-styled markup and folds its two
duplicated branches into one. Its border and the drawer's mono font stay on
the module's existing constants so the card still matches InputCard, which
is out of scope here.

Both files retire their no-restricted-imports suppression.
2026-08-12 22:29:57 -07:00
Yuneng Jiang
9355953414
refactor(ui): migrate SimpleMessageBlock and SimpleToolCallBlock to shadcn
Replace antd Typography and inline hex styling in the log drawer's message
and tool-call blocks with plain elements and design tokens. Both components'
existing tests pass unedited before and after.

Retires their no-restricted-imports entries from the eslint suppressions
baseline.
2026-08-12 22:27:20 -07:00
Yuneng Jiang
c25f927f03
test(ui): make the TokenFlow grouping assertion locale-independent
The assertion hard-coded en-US separators while toLocaleString follows the
host locale, so it failed under de_DE. Building the expected string the same
way keeps it deterministic everywhere and still catches a dropped
toLocaleString wherever the locale groups at all.
2026-08-12 22:23:06 -07:00
Yuneng Jiang
214a476c3e
refactor(ui): migrate TokenFlow and JsonViewer to shadcn
Drops the antd Typography import from both. TokenFlow renders inside an antd
Descriptions.Item that already sets the colour, font size, line height and
wrapping the Text wrapper restated, so a bare span is pixel-identical there.
JsonViewer's placeholder moves onto the muted-foreground token.

The characterisation tests from the previous commit are unchanged and stay
green, which is what shows the markup swap did not move behaviour.
2026-08-12 22:10:39 -07:00
mateo-berri
2a3b54394f refactor(interactions): drop Any and extra casts from bridge input helpers 2026-08-12 22:03:33 -07:00
Yuneng Jiang
49c697ee89
refactor(ui): migrate HistoryTree and CollapsibleMessage to shadcn
Both log-drawer collapse rows hand-rolled the same pattern: a click
handler on a plain div, hover tracked in React state, and a max-height
tween on an always-mounted panel. Move both onto the ui/collapsible
primitive with lucide chevrons, so the row is a real button that keyboard
users can reach and the open state lives in the primitive.

CollapsibleContent keeps keepMounted, which preserves the existing
contract that panel content stays in the DOM while collapsed.

Neither test file is touched: both were already role and text based, and
they pass unedited against the new markup.
2026-08-12 22:03:20 -07:00
Yuneng Jiang
89d5037780
test(ui): characterize TokenFlow and JsonViewer
Both components are shared by the logs, guardrails-monitor and tool-policies
routes and had no test. These assert on rendered text and roles only, so they
hold against antd Typography and against its replacement.
2026-08-12 21:48:51 -07:00
mateo-berri
432ea8644b test(interactions): send step and content-list input to the live Gemini API 2026-08-12 21:47:33 -07:00
mateo-berri
0aeed16125 test(interactions): follow Google spec drift replacing Turn with typed steps 2026-08-12 21:47:33 -07:00
mateo-berri
3e94f7d71e fix(interactions): map step and turn input to Responses API roles and content types 2026-08-12 21:47:00 -07:00
mateo
8947008fd2 fix(batches): only retire on a 404 that names the batch
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-13 04:22:43 +00:00
mateo-berri
da84142288 fix(batches): only trust a 404 from the batch's own deployment 2026-08-12 21:19:37 -07:00
yucheng-berri
d86336a7c6
fix(langfuse): emit otel trace version and release on the keys langfuse v4 reads (#36702)
* fix(langfuse): emit otel trace version and release on the keys langfuse v4 reads

The langfuse_otel exporter wrote version to langfuse.generation.version and
langfuse.trace.version, and release to langfuse.trace.release. Langfuse v4
recognizes neither, so both landed in the generic span attribute bag and every
trace reported version and release as null. v4 has a single langfuse.version
key, lifted to the trace when it sits on the root span, plus langfuse.release.

Also routes the otel v2 preset's per-request headers through the shared builder
so key-scoped and team-scoped exports carry x-langfuse-ingestion-version like
the other three exporter paths already do.

* fix(langfuse): give trace_version precedence over version on the shared v4 key

Matches the documented contract in docs/observability/langfuse_integration.md
and the legacy langfuse SDK callback, which both treat trace_version as the
authoritative trace version with version as its fallback.
2026-08-12 20:39:58 -07:00
Mateo Wang
c7f5527870
Merge pull request #36726 from BerriAI/litellm_claude_md_remove_at_prefix
docs(claude): drop the @ prefix from the PR template path
2026-08-12 20:31:59 -07:00
mateo
de4765ae88 docs(claude): drop the @ prefix from the PR template path
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-13 03:20:15 +00:00
devin-ai-integration[bot]
3864e12415
fix(spend): stop losing spend log rows when a flush is cancelled (#34826) 2026-08-12 20:10:50 -07:00
Mateo Wang
fdd72b5b23
Merge pull request #36627 from BerriAI/litellm_fix_autorouter_untagged_hijack
Some checks failed
Unit Tests: MCP, Secrets, Containers & Misc / misc (push) Waiting to run
Unit Tests: Proxy Auth & Key Management / proxy-auth (push) Waiting to run
Unit Tests: Proxy DB Operations / assert-shard-coverage (push) Waiting to run
Unit Tests: Proxy DB Operations / auth-checks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / budgets (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / custom-logging (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / db-and-spend (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / endpoints-and-responses (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / guardrails-hooks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / jwt-and-keys (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / key-generation (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / logging-misc (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-runtime (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-server-core (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-utils (push) Blocked by required conditions
Unit Tests: Proxy API Endpoints / proxy-endpoints (push) Waiting to run
Unit Tests: Proxy API Endpoints / proxy-server (push) Waiting to run
Unit Tests: Proxy Infrastructure / proxy-infra (push) Waiting to run
Unit Tests: Proxy Legacy Tests / auth-and-jwt (push) Waiting to run
Unit Tests: Proxy Legacy Tests / key-generation (push) Waiting to run
Unit Tests: Proxy Legacy Tests / proxy-config (push) Waiting to run
Unit Tests: Proxy Legacy Tests / proxy-response-and-misc (push) Waiting to run
GitHub Actions Security Analysis / zizmor (push) Waiting to run
Unit Tests: Proxy Legacy Tests / proxy-server (push) Waiting to run
Unit Tests: Proxy Legacy Tests / proxy-server-extras (push) Waiting to run
Unit Tests: Proxy Legacy Tests / proxy-token-counter (push) Waiting to run
Unit Tests: Proxy Legacy Tests / proxy-user-auth-and-spend (push) Waiting to run
Unit Tests: Proxy Legacy Tests / proxy-utils (push) Waiting to run
Unit Tests: Responses, Caching & Types / responses-caching-types (push) Waiting to run
Terraform Modules / fmt, validate, test (aws) (push) Has been cancelled
fix(router): let untagged requests bypass a tagged pre-routing strategy on shared model names
2026-08-12 19:48:00 -07:00
mateo-berri
39a5ede2d9 fix(model_prices): bill muse spark web search grounding per query 2026-08-12 19:44:20 -07:00
Mateo Wang
6038dc6891
Merge pull request #36664 from BerriAI/litellm_groq_model_registry_audit
fix(model_prices): sync the Groq registry with Groq's docs
2026-08-12 19:37:39 -07:00
mateo-berri
1369b3e6f3 test(model_prices): pin muse spark cost assertions to the bundled map
cost_per_token reads litellm.model_cost, which CI loads from the
model_prices_and_context_window.json on main, so the new entries were
missing until merge. Use the same local_model_cost_map fixture the
sonnet 5 metadata test uses to force the branch's own backup map.
2026-08-12 19:14:19 -07:00
Mateo Wang
6209b8928d
Merge pull request #36698 from BerriAI/litellm_bedrock_gpt56_long_context
fix(bedrock_mantle): 1M context window and long-context pricing for GPT-5.6 Sol/Terra/Luna
2026-08-12 18:49:45 -07:00
mateo-berri
ad36c603b6 feat(model_prices): add meta/muse-spark-1.2 and its contributor tier 2026-08-12 17:51:19 -07:00
mateo-berri
feff5ae34c Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_fix_autorouter_untagged_hijack
# Conflicts:
#	litellm/router.py
2026-08-13 00:47:52 +00:00
mateo-berri
ce3d30a434 fix(router): read no tags, instead of raising, from a non-dict metadata bucket
Pre-routing now reads the request's tags on every request with a registered
strategy, including the single-strategy case that used to short-circuit before
looking at tags. Metadata is request-controlled, so a caller that sends
`litellm_metadata` (or `tags`) as a string or any other non-dict shape crashed
tag lookup with an AttributeError instead of routing untagged.
2026-08-13 00:45:49 +00:00
Mateo Wang
3b5317c40d
Merge pull request #36626 from BerriAI/litellm_fix_autorouter_alias_forwarding
fix(router): forward auto-router alias params from the marker entry, not the first same-name deployment
2026-08-12 16:55:12 -07:00
Yassin Kortam
50ff555d79
fix(ui): show and edit key-level router settings on a virtual key (#36674)
The virtual-key edit view never rendered the router settings stored on a
key, so fallbacks configured at creation could not be verified or changed
afterwards. The key info panel now summarises them and the edit view
embeds the router settings accordion.

The accordion is a fixed-field editor, so its value is merged over the
stored object instead of replacing it. Routing fields it cannot render,
such as tag_routing_prefix or model_group_retry_policy, survive an
unrelated edit, while a field it does own that the admin emptied still
goes out as null so the clear reaches the server. Emptying every field
sends {}, which the proxy reads as no key-level override, so the key
falls back to its team and global routing rather than being pinned to a
blob of nulls.
2026-08-12 16:47:45 -07:00
mateo
c11ebbed27 fix(batches): stop uncostable batches from starving the cost poll page
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-12 23:45:36 +00:00
mateo
9f1129ae19 Merge branch 'litellm_internal_staging' into litellm_fix_autorouter_untagged_hijack
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-12 22:57:01 +00:00