Commit graph

43192 commits

Author SHA1 Message Date
mateo
b5bcdbccd4 refactor(proxy): type gateway protocol payloads with pydantic models
Some checks failed
LiteLLM Rust / rustfmt, clippy, test (push) Has been cancelled
Terraform Modules / fmt, validate, test (aws) (push) Has been cancelled
Terraform Provider / gofmt, vet, build, test (push) Has been cancelled
Terraform Provider / Provider endpoints vs proxy OpenAPI schema (push) Has been cancelled
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-15 18:33:09 +00:00
mateo
6230a379b9 fix(proxy): use builtin dict annotation for gateway managed settings config
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-15 17:58:43 +00:00
mateo
a00fdd1d1f Merge branch 'litellm_internal_staging' into litellm_claude_code_gateway_protocol
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-15 17:57:43 +00:00
yuneng-jiang
487da4d1e4
Merge pull request #37021 from BerriAI/litellm_models_source_column
fix(ui): rename models table Status column to Source
2026-08-15 10:55:06 -07:00
Mateo Wang
87abb8781e
Merge pull request #36925 from BerriAI/litellm_model_registry_lifecycle_audit_20260814 2026-08-15 09:58:31 -07:00
yuneng-jiang
f1acd6d375
Merge pull request #37022 from BerriAI/litellm_user_teams_walkable
feat(ui): link user detail team names to team pages
2026-08-15 09:44:24 -07:00
yuneng-jiang
d4a52f9fa5
Merge pull request #36991 from BerriAI/litellm_/blocked-badge-styling-c03b31
refactor(ui): re-sync badge and skeleton onto the base-vega shadcn style
2026-08-15 09:24:26 -07:00
Yuneng Jiang
09741740d4
test(ui): follow the models table Status to Source column rename 2026-08-15 09:20:08 -07:00
yuneng-jiang
759cb4f24b
Merge pull request #37033 from BerriAI/litellm_access_group_create_rhf
refactor(ui): migrate access group create modal to RHF + zod + shadcn
2026-08-15 09:15:23 -07:00
ryan-crabbe-berri
8c991076e3 refactor(ui): migrate access group create modal to RHF + zod + shadcn 2026-08-15 08:13:16 -07:00
ryan-crabbe-berri
1b2f05d6b0 feat(ui): link user detail team names to team pages 2026-08-15 03:18:23 -07:00
ryan-crabbe-berri
aacab940f4 fix(ui): rename models table Status column to Source 2026-08-15 03:15:53 -07:00
Devin AI
30b14597af fix(model_prices): revert unverified Gemini deprecation dates
Google's Gemini deprecations page lists no shutdown date for the 2.5 GA models and May 14, 2028 for gemini-embedding-001; keep only the DeepSeek V4 max output correction.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-15 08:35:58 +00:00
Mateo Wang
7a5b98e6a9
Merge pull request #37005 from BerriAI/litellm_claudemd_lit012
docs(claude): require ReadOnly on every TypedDict field (LIT012)
2026-08-15 01:27:50 -07:00
Mateo Wang
3798df392a
chore: make it more concise 2026-08-15 01:13:16 -07:00
Mateo Wang
d70cc14981
Merge pull request #35092 from BerriAI/litellm_vertex_batch_embeddings_translation
fix(vertex_ai): translate /v1/embeddings batch rows to the Gemini embedding shape
2026-08-14 21:52:32 -07:00
Mateo Wang
89f233a15d
Merge pull request #36720 from BerriAI/litellm_tiered_pricing_cache_creation
fix(cost): tiered pricing supports cache creation cost and is all-or-nothing
2026-08-14 21:51:54 -07:00
mateo
1abde19280 docs(claude): require ReadOnly on every TypedDict field (LIT012)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-15 04:24:18 +00:00
mateo-berri
18752c860c fix(cost): honor explicit zero tier rates and skip synthesized backend output rates 2026-08-14 21:05:22 -07:00
devin-ai-integration[bot]
6e7984e537
fix(proxy): requeue spend logs when the DB write fails with a transport error (#36716)
* fix(proxy): requeue spend logs when the DB write fails with a transport error

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): hardcode the spend log queue cap and drop the stale re-export

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(proxy): keep the spend log requeue within the type discipline budget

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): apply the spend log queue cap to producer appends too

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): lower the spend log queue cap to 1k and make it env configurable

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): bound the spend log queue by bytes instead of row count

A row cap cannot bound memory: a row carries the whole prompt under store_prompts_in_spend_logs, so a cap that rides out an outage of counter-only rows is an OOM once prompts are stored. Every enqueue and dequeue now goes through one pair that tracks what the queue costs and drops the oldest rows past a 64 MB budget.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): make the spend log queue byte budget env configurable

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): use a string default for the spend log queue byte budget env read

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): make the spend log queue byte total a public attribute

The queue it accounts for is already public, and a private name only bought reportPrivateUsage errors at every call site.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: shivam <shivam@berri.ai>
2026-08-14 20:49:45 -07:00
mateo
e46f2ca62f fix(dashscope): honor the model reasoning rate when a tier omits output rates
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-15 03:42:23 +00:00
mateo-berri
34918d34f9 fix(cost): inherit the backend output rate when a deployment's tiers omit one 2026-08-14 20:37:18 -07:00
Mateo Wang
6c2dcb801b
Merge pull request #36907 from guptaishaan/fix/issue-36880-8933
fix: report real token usage on guardrail-blocked /v1/responses replies
2026-08-14 20:16:25 -07:00
mateo
c3e38a0b52 fix(cost): fall back to the model output rate when a tier omits one
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-15 03:05:13 +00:00
Mateo Wang
1a183efaa1
Merge pull request #36159 from daleselaji-dev/codex/bedrock-alias-36156
Some checks are pending
CodSpeed Benchmarks / benchmarks (push) Waiting to run
Publish basedpyright base counts / publish (push) Waiting to run
Code Quality Checks / code-quality (push) Waiting to run
UI Unit Tests / ui-unit-tests (push) Waiting to run
Unit Tests: Core Utilities / core-utils (push) Waiting to run
Unit Tests: Documentation Validation / documentation (push) Waiting to run
Unit Tests: Enterprise, Google GenAI & Routing / enterprise-routing (push) Waiting to run
Unit Tests: Integrations (Callbacks & Logging) / integrations (push) Waiting to run
Unit Tests: LLM Provider Transformations / Vertex AI (push) Waiting to run
Unit Tests: LLM Provider Transformations / All Other Providers (push) Waiting to run
Unit Tests: MCP, Secrets, Containers & Misc / misc (push) Waiting to run
Unit Tests: Proxy Auth & Key Management / proxy-auth (push) Waiting to run
Unit Tests: Proxy DB Operations / assert-shard-coverage (push) Waiting to run
Unit Tests: Proxy DB Operations / auth-checks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / budgets (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / custom-logging (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / db-and-spend (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / endpoints-and-responses (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / guardrails-hooks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-utils (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / jwt-and-keys (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / key-generation (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / logging-misc (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-runtime (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-server-core (push) Blocked by required conditions
Unit Tests: Proxy API Endpoints / proxy-endpoints (push) Waiting to run
Unit Tests: Proxy API Endpoints / proxy-server (push) Waiting to run
Unit Tests: Proxy Infrastructure / proxy-infra (push) Waiting to run
Unit Tests: Responses, Caching & Types / responses-caching-types (push) Waiting to run
GitHub Actions Security Analysis / zizmor (push) Waiting to run
fix(bedrock): resolve aliases in batch file records
2026-08-14 18:20:21 -07:00
Mateo Wang
70d51a196f
Merge pull request #35286 from BerriAI/devin_ai_lit_5013_web_search_cost
fix(cost): track OpenAI/Azure web search tool cost per call
2026-08-14 18:16:59 -07:00
Mateo Wang
cba2beaf42
Merge pull request #36965 from erensh27/feat/per-component-cost-headers
feat(proxy): per-component response cost headers
2026-08-14 18:15:23 -07:00
mateo
3f64cbe41b fix(ptu): empty a PTU deployment's tiered_pricing instead of dropping it
Dropping it falls back to the public cost map's tier table, whose rates outrank the
zeros written beside them, so a PTU deployment on a tiered model keeps billing its
traffic per token. Stored empty, the tiers no longer apply and the zeros win
2026-08-15 01:14:14 +00:00
yuneng-jiang
a4aab2e9f3
Merge pull request #36995 from BerriAI/litellm_/playground-config-select-25ad99
fix(ui): anchor chips-combobox popups to the field instead of the inner input
2026-08-14 18:11:38 -07:00
mateo
9079844fa2 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_tiered_pricing_cache_creation 2026-08-15 01:05:01 +00:00
yuneng-jiang
491e2ffb1b
Merge pull request #36993 from BerriAI/litellm_/scrollable-tabs-layout-6aa9f7
fix(ui): stop the models tab strip from scrolling vertically
2026-08-14 18:04:42 -07:00
mateo-berri
5970754a85 fix(ptu): clear a PTU deployment's tiered_pricing instead of zeroing it
tiered_pricing is a list, so the 0.0 the flat-rate zeroing stores does not
even validate. Supplying tiers alongside PTU config gets the same 400 as a
flat rate; tiers already stored are dropped from both blobs
2026-08-14 18:04:01 -07:00
mateo-berri
0176e4b3f6 refactor(cost): make the shared token-details parsers public
parse_prompt_tokens_details and parse_completion_tokens_details are imported
by four modules, so the leading underscore made every import a
reportPrivateUsage violation
2026-08-14 18:04:01 -07:00
Yuneng Jiang
c7084c04c0
test(ui): assert which element the chips-combobox popup anchors to
The previous assertion read data-chips, which is derived from the anchor prop
being truthy, so it stayed true even when the ref never reached the DOM and the
popup was still anchored to the inner input.

Stub distinct widths on the chips container and the input, then read the width
the positioner resolved. Reverting the anchor wiring now reports the input's
width instead of the field's, which is the actual bug.
2026-08-14 18:03:34 -07:00
Yuneng Jiang
6e38e9490d
Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_/playground-config-select-25ad99 2026-08-14 17:57:27 -07:00
Yuneng Jiang
654fab224a
Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_/scrollable-tabs-layout-6aa9f7 2026-08-14 17:55:49 -07:00
Yuneng Jiang
a0e5c7e818
fix(ui): anchor chips-combobox popups to the field instead of the inner input
Base UI positions a combobox popup against the Combobox.Input by default. In
chips mode the visible field is the ComboboxChips wrapper and the input is a
smaller box nested inside it, so every chips-combobox in the dashboard opened
its popup 11px right of the field and 17px past its right edge.

shadcn ships the wiring for this and their combobox-multiple example uses it:
useComboboxAnchor on the chips container, passed to ComboboxContent as anchor.
The anchor prop also drives data-chips, which cancels the extra min-width an
ordinary combobox wants. Every chips site in the dashboard omitted it.

The anchor is attached through Base UI's render prop rather than a plain ref,
because React 18 drops refs on function components and ComboboxChips is one.

Adds MultiSelect's first test, covering the anchor wiring plus selection,
chip rendering and custom values.
2026-08-14 17:51:54 -07:00
Mateo Wang
40a418440c
Merge pull request #35969 from milesadkins/fireworks_nim_vllm_compat
feat(fireworks_ai): translate NIM/vLLM extra params to Fireworks-native args
2026-08-14 17:47:47 -07:00
Ahmed N
691c7fd4d6
fix(anthropic_messages): make tool_result images visible to OpenAI-compatible providers (#34462)
Images nested inside an Anthropic `tool_result` block were dropped when the
request was adapted for an OpenAI-compatible provider, because the OpenAI tool
message shape only carried text. Hoist those images out of the tool result and
into a following user message so the model can still see them, and widen the
tool message content type to accept image parts.
2026-08-14 17:47:38 -07:00
mateo-berri
2c6409c7e6 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_tiered_pricing_cache_creation 2026-08-14 17:44:39 -07:00
Yuneng Jiang
5219658b8f
fix(ui): stop the models tab strip from scrolling vertically
The tab strip carried overflow-x-auto directly on the TabsList. CSS forces
overflow-y from visible to auto once overflow-x is not visible, and the line
variant's active-tab underline is an absolutely positioned ::after that hangs
5px below its trigger, so the strip picked up a pixel of vertical scroll on top
of the horizontal scroll it actually wants.

The scroll container now lives on a wrapper whose bottom padding leaves room for
the underline, offset by a matching negative margin so the row keeps its exact
geometry.
2026-08-14 17:44:00 -07:00
Yuneng Jiang
b1696b3edf
test(ui): assert cache and retry tags by text instead of class name
Three assertions in LogDetailContent.test.tsx matched a regex against the
rendered class string to prove a tag was green or was not red. That pins styling
rather than behavior, and jsdom does not resolve the utilities anyway, so the
checks only ever proved that a substring survived into the class attribute

The badge re-sync exposed it: base-vega's base string carries aria-invalid
variants of the destructive token, so a "not destructive" regex started matching
every badge regardless of variant

Each one now asserts the tag's text is present, which is what the surrounding
cases already do and what the user actually observes
2026-08-14 17:38:47 -07:00
mateo-berri
de77711cf9 test(vertex_ai): cover duplicated fan-out rows in embeddings batch reassembly
Some checks failed
Terraform Modules / fmt, validate, test (aws) (push) Has been cancelled
Terraform Provider / gofmt, vet, build, test (push) Has been cancelled
Terraform Provider / Provider endpoints vs proxy OpenAPI schema (push) Has been cancelled
Also ruff-formats the batch transformation test file, which the formatter
gate flags once the file is touched.
2026-08-14 17:37:07 -07:00
Mateo Wang
118523ede6
Merge pull request #34581 from BerriAI/litellm_anthropic_messages_response_cache
fix(caching): cache anthropic /v1/messages responses, including streaming
2026-08-14 17:34:08 -07:00
Mateo Wang
dc92749c07
Merge pull request #35360 from BerriAI/devin_ai_fix_batch_cost_completed_no_output
fix(batches): mark terminal batch with no output file as processed in CheckBatchCost
2026-08-14 17:33:07 -07:00
Yuneng Jiang
0da672c8e2
Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_/blocked-badge-styling-c03b31 2026-08-14 17:31:48 -07:00
Yuneng Jiang
2de3195757
refactor(ui): re-sync badge and skeleton onto the base-vega shadcn style
components.json has declared "style": "base-vega" since cfe9e39e55, but badge
and skeleton were added a few days earlier under new-york and never re-synced,
so both still carried the previous style's classes. Badge's destructive variant
rendered as solid red with white text instead of the tinted wash the rest of the
dashboard uses, which is already the convention for Button

Re-runs npx shadcn add for both and keeps the two local deltas the registry
cannot supply: cva comes from @/lib/cva.config, since class-variance-authority
is not a dependency here, and both stay wrapped in React.forwardRef, which the
tripwire in tests/setupTests.ts requires until the React 19 upgrade

Adds Badge to ref-forwarding.test.tsx. Nothing covered it before, even though
two TooltipTrigger sites compose over it, so the wrapper could have been dropped
by the next re-sync without a single test going red

Retargets one assertion in LogDetailContent.test.tsx. It regex-matched the whole
class string for "destructive" to prove a tag was not alarming red, which the
restored aria-invalid classes now satisfy for every variant; it checks the
variant attribute and red utility classes instead
2026-08-14 17:31:44 -07:00
mateo-berri
05beb7abb5 fix(proxy): emit uncached input cost so component headers sum to the total 2026-08-14 17:30:50 -07:00
mateo-berri
7e539405ed fix(cost-tracking): price web search on dated search-preview map entries 2026-08-14 17:29:56 -07:00
milan
0d0c712df7 fix(vertex_ai): fail an embeddings batch entry whose fan-out came back incomplete
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-15 00:29:35 +00:00