Commit graph

45043 commits

Author SHA1 Message Date
mateo
816fa50394 refactor(proxy): make the deprecation loop entrypoint public and drop a dead None check
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-15 08:35:12 +00:00
mateo-berri
16e6aa2e9f fix(model_prices): correct deprecation dates flagged in review
Revert gemini-embedding-001 to its published 2028-05-14 shutdown, move
gpt-4-turbo-preview to the 2026-03-26 shutdown of the gpt-4-0125-preview
snapshot it aliases, and drop the unannounced Gemini 2.5 shutdown dates
2026-08-15 01:33:01 -07:00
Mateo Wang
7a5b98e6a9
Merge pull request #37005 from BerriAI/litellm_claudemd_lit012
docs(claude): require ReadOnly on every TypedDict field (LIT012)
2026-08-15 01:27:50 -07:00
Mateo Wang
3798df392a
chore: make it more concise 2026-08-15 01:13:16 -07:00
Yuneng Jiang
926e4eb985
Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_shadcn_next_0814 2026-08-15 00:33:27 -07:00
Yuneng Jiang
de25e199d4
fix(ui): de-duplicate the reset budget option and polish shadcn surfaces
Create Key offered two options labelled "Never resets" in the Reset Budget
dropdown. BudgetDurationDropdown renders its unset item using the caller's
placeholder, and create_key_button passed placeholder="Never resets" alongside
showNeverResets, so the omit option and the explicit-null option looked
identical while behaving differently. An omitted budget_duration picks up
default_key_generate_params and the linked budget tier's schedule, whereas the
"none" sentinel is converted to an explicit null and truly never resets. The
unset item now reads "Not set", matching getBudgetDurationLabel, and the
create-key test mock passes the placeholder through so a future collision fails
the suite

The rest is migration cleanup found during manual QA. The models and endpoints
tab strip hides its scrollbar and fades at the right edge using a vendored copy
of the shadcn scroll-fade utility, keeping the CLI package out of the build.
globals.css neutralises the @tailwindcss/forms resting-state rules for
combobox-chip-input, which lets twelve call sites drop the same copy-pasted
className workaround. Guardrails moves to the line tab variant and stops
clipping its textarea focus ring, the log drawer JSON tree takes the app
background, the audit log empty state centres, the caching page selects no
longer stretch to the row height, the usage page team filter shares its row
with the Export button through a new filterSlot prop, and the cost optimization
and vector store tab strips drop their leftover full-width divider
2026-08-15 00:33:20 -07:00
Mateo Wang
d70cc14981
Merge pull request #35092 from BerriAI/litellm_vertex_batch_embeddings_translation
fix(vertex_ai): translate /v1/embeddings batch rows to the Gemini embedding shape
2026-08-14 21:52:32 -07:00
Mateo Wang
89f233a15d
Merge pull request #36720 from BerriAI/litellm_tiered_pricing_cache_creation
fix(cost): tiered pricing supports cache creation cost and is all-or-nothing
2026-08-14 21:51:54 -07:00
mateo
1abde19280 docs(claude): require ReadOnly on every TypedDict field (LIT012)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-15 04:24:18 +00:00
mateo-berri
17f5c909f0 fix(make): acquire the gate slot before lint setup deps 2026-08-14 21:17:14 -07:00
mateo-berri
18752c860c fix(cost): honor explicit zero tier rates and skip synthesized backend output rates 2026-08-14 21:05:22 -07:00
devin-ai-integration[bot]
6e7984e537
fix(proxy): requeue spend logs when the DB write fails with a transport error (#36716)
* fix(proxy): requeue spend logs when the DB write fails with a transport error

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): hardcode the spend log queue cap and drop the stale re-export

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(proxy): keep the spend log requeue within the type discipline budget

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): apply the spend log queue cap to producer appends too

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): lower the spend log queue cap to 1k and make it env configurable

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): bound the spend log queue by bytes instead of row count

A row cap cannot bound memory: a row carries the whole prompt under store_prompts_in_spend_logs, so a cap that rides out an outage of counter-only rows is an OOM once prompts are stored. Every enqueue and dequeue now goes through one pair that tracks what the queue costs and drops the oldest rows past a 64 MB budget.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): make the spend log queue byte budget env configurable

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): use a string default for the spend log queue byte budget env read

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): make the spend log queue byte total a public attribute

The queue it accounts for is already public, and a private name only bought reportPrivateUsage errors at every call site.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: shivam <shivam@berri.ai>
2026-08-14 20:49:45 -07:00
mateo
e46f2ca62f fix(dashscope): honor the model reasoning rate when a tier omits output rates
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-15 03:42:23 +00:00
mateo-berri
34918d34f9 fix(cost): inherit the backend output rate when a deployment's tiers omit one 2026-08-14 20:37:18 -07:00
Mateo Wang
6c2dcb801b
Merge pull request #36907 from guptaishaan/fix/issue-36880-8933
fix: report real token usage on guardrail-blocked /v1/responses replies
2026-08-14 20:16:25 -07:00
Devin AI
726db1a4c1 test(bedrock): cover async header forwarding for converse and invoke
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-15 03:07:14 +00:00
mateo
c3e38a0b52 fix(cost): fall back to the model output rate when a tier omits one
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-15 03:05:13 +00:00
Devin AI
9c922f4aa4 fix(bedrock): forward provider response headers on chat completions
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-15 02:49:10 +00:00
Mateo Wang
1a183efaa1
Merge pull request #36159 from daleselaji-dev/codex/bedrock-alias-36156
Some checks are pending
CodSpeed Benchmarks / benchmarks (push) Waiting to run
Publish basedpyright base counts / publish (push) Waiting to run
Code Quality Checks / code-quality (push) Waiting to run
UI Unit Tests / ui-unit-tests (push) Waiting to run
Unit Tests: Core Utilities / core-utils (push) Waiting to run
Unit Tests: Documentation Validation / documentation (push) Waiting to run
Unit Tests: Enterprise, Google GenAI & Routing / enterprise-routing (push) Waiting to run
Unit Tests: Integrations (Callbacks & Logging) / integrations (push) Waiting to run
Unit Tests: LLM Provider Transformations / Vertex AI (push) Waiting to run
Unit Tests: LLM Provider Transformations / All Other Providers (push) Waiting to run
Unit Tests: MCP, Secrets, Containers & Misc / misc (push) Waiting to run
Unit Tests: Proxy Auth & Key Management / proxy-auth (push) Waiting to run
Unit Tests: Proxy DB Operations / assert-shard-coverage (push) Waiting to run
Unit Tests: Proxy DB Operations / auth-checks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / budgets (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / custom-logging (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / db-and-spend (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / endpoints-and-responses (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / guardrails-hooks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-utils (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / jwt-and-keys (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / key-generation (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / logging-misc (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-runtime (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-server-core (push) Blocked by required conditions
Unit Tests: Proxy API Endpoints / proxy-endpoints (push) Waiting to run
Unit Tests: Proxy API Endpoints / proxy-server (push) Waiting to run
Unit Tests: Proxy Infrastructure / proxy-infra (push) Waiting to run
Unit Tests: Responses, Caching & Types / responses-caching-types (push) Waiting to run
GitHub Actions Security Analysis / zizmor (push) Waiting to run
fix(bedrock): resolve aliases in batch file records
2026-08-14 18:20:21 -07:00
Mateo Wang
70d51a196f
Merge pull request #35286 from BerriAI/devin_ai_lit_5013_web_search_cost
fix(cost): track OpenAI/Azure web search tool cost per call
2026-08-14 18:16:59 -07:00
Mateo Wang
cba2beaf42
Merge pull request #36965 from erensh27/feat/per-component-cost-headers
feat(proxy): per-component response cost headers
2026-08-14 18:15:23 -07:00
mateo
3f64cbe41b fix(ptu): empty a PTU deployment's tiered_pricing instead of dropping it
Dropping it falls back to the public cost map's tier table, whose rates outrank the
zeros written beside them, so a PTU deployment on a tiered model keeps billing its
traffic per token. Stored empty, the tiers no longer apply and the zeros win
2026-08-15 01:14:14 +00:00
yuneng-jiang
a4aab2e9f3
Merge pull request #36995 from BerriAI/litellm_/playground-config-select-25ad99
fix(ui): anchor chips-combobox popups to the field instead of the inner input
2026-08-14 18:11:38 -07:00
mateo
9079844fa2 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_tiered_pricing_cache_creation 2026-08-15 01:05:01 +00:00
yuneng-jiang
491e2ffb1b
Merge pull request #36993 from BerriAI/litellm_/scrollable-tabs-layout-6aa9f7
fix(ui): stop the models tab strip from scrolling vertically
2026-08-14 18:04:42 -07:00
mateo-berri
5970754a85 fix(ptu): clear a PTU deployment's tiered_pricing instead of zeroing it
tiered_pricing is a list, so the 0.0 the flat-rate zeroing stores does not
even validate. Supplying tiers alongside PTU config gets the same 400 as a
flat rate; tiers already stored are dropped from both blobs
2026-08-14 18:04:01 -07:00
mateo-berri
0176e4b3f6 refactor(cost): make the shared token-details parsers public
parse_prompt_tokens_details and parse_completion_tokens_details are imported
by four modules, so the leading underscore made every import a
reportPrivateUsage violation
2026-08-14 18:04:01 -07:00
Yuneng Jiang
c7084c04c0
test(ui): assert which element the chips-combobox popup anchors to
The previous assertion read data-chips, which is derived from the anchor prop
being truthy, so it stayed true even when the ref never reached the DOM and the
popup was still anchored to the inner input.

Stub distinct widths on the chips container and the input, then read the width
the positioner resolved. Reverting the anchor wiring now reports the input's
width instead of the field's, which is the actual bug.
2026-08-14 18:03:34 -07:00
Yuneng Jiang
6e38e9490d
Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_/playground-config-select-25ad99 2026-08-14 17:57:27 -07:00
Yuneng Jiang
654fab224a
Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_/scrollable-tabs-layout-6aa9f7 2026-08-14 17:55:49 -07:00
Yuneng Jiang
a0e5c7e818
fix(ui): anchor chips-combobox popups to the field instead of the inner input
Base UI positions a combobox popup against the Combobox.Input by default. In
chips mode the visible field is the ComboboxChips wrapper and the input is a
smaller box nested inside it, so every chips-combobox in the dashboard opened
its popup 11px right of the field and 17px past its right edge.

shadcn ships the wiring for this and their combobox-multiple example uses it:
useComboboxAnchor on the chips container, passed to ComboboxContent as anchor.
The anchor prop also drives data-chips, which cancels the extra min-width an
ordinary combobox wants. Every chips site in the dashboard omitted it.

The anchor is attached through Base UI's render prop rather than a plain ref,
because React 18 drops refs on function components and ComboboxChips is one.

Adds MultiSelect's first test, covering the anchor wiring plus selection,
chip rendering and custom values.
2026-08-14 17:51:54 -07:00
Mateo Wang
40a418440c
Merge pull request #35969 from milesadkins/fireworks_nim_vllm_compat
feat(fireworks_ai): translate NIM/vLLM extra params to Fireworks-native args
2026-08-14 17:47:47 -07:00
Ahmed N
691c7fd4d6
fix(anthropic_messages): make tool_result images visible to OpenAI-compatible providers (#34462)
Images nested inside an Anthropic `tool_result` block were dropped when the
request was adapted for an OpenAI-compatible provider, because the OpenAI tool
message shape only carried text. Hoist those images out of the tool result and
into a following user message so the model can still see them, and widen the
tool message content type to accept image parts.
2026-08-14 17:47:38 -07:00
mateo-berri
2c6409c7e6 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_tiered_pricing_cache_creation 2026-08-14 17:44:39 -07:00
Yuneng Jiang
5219658b8f
fix(ui): stop the models tab strip from scrolling vertically
The tab strip carried overflow-x-auto directly on the TabsList. CSS forces
overflow-y from visible to auto once overflow-x is not visible, and the line
variant's active-tab underline is an absolutely positioned ::after that hangs
5px below its trigger, so the strip picked up a pixel of vertical scroll on top
of the horizontal scroll it actually wants.

The scroll container now lives on a wrapper whose bottom padding leaves room for
the underline, offset by a matching negative margin so the row keeps its exact
geometry.
2026-08-14 17:44:00 -07:00
Yuneng Jiang
b1696b3edf
test(ui): assert cache and retry tags by text instead of class name
Three assertions in LogDetailContent.test.tsx matched a regex against the
rendered class string to prove a tag was green or was not red. That pins styling
rather than behavior, and jsdom does not resolve the utilities anyway, so the
checks only ever proved that a substring survived into the class attribute

The badge re-sync exposed it: base-vega's base string carries aria-invalid
variants of the destructive token, so a "not destructive" regex started matching
every badge regardless of variant

Each one now asserts the tag's text is present, which is what the surrounding
cases already do and what the user actually observes
2026-08-14 17:38:47 -07:00
mateo-berri
de77711cf9 test(vertex_ai): cover duplicated fan-out rows in embeddings batch reassembly
Some checks failed
Terraform Modules / fmt, validate, test (aws) (push) Has been cancelled
Terraform Provider / gofmt, vet, build, test (push) Has been cancelled
Terraform Provider / Provider endpoints vs proxy OpenAPI schema (push) Has been cancelled
Also ruff-formats the batch transformation test file, which the formatter
gate flags once the file is touched.
2026-08-14 17:37:07 -07:00
Mateo Wang
118523ede6
Merge pull request #34581 from BerriAI/litellm_anthropic_messages_response_cache
fix(caching): cache anthropic /v1/messages responses, including streaming
2026-08-14 17:34:08 -07:00
Mateo Wang
dc92749c07
Merge pull request #35360 from BerriAI/devin_ai_fix_batch_cost_completed_no_output
fix(batches): mark terminal batch with no output file as processed in CheckBatchCost
2026-08-14 17:33:07 -07:00
Yuneng Jiang
0da672c8e2
Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_/blocked-badge-styling-c03b31 2026-08-14 17:31:48 -07:00
Yuneng Jiang
2de3195757
refactor(ui): re-sync badge and skeleton onto the base-vega shadcn style
components.json has declared "style": "base-vega" since cfe9e39e55, but badge
and skeleton were added a few days earlier under new-york and never re-synced,
so both still carried the previous style's classes. Badge's destructive variant
rendered as solid red with white text instead of the tinted wash the rest of the
dashboard uses, which is already the convention for Button

Re-runs npx shadcn add for both and keeps the two local deltas the registry
cannot supply: cva comes from @/lib/cva.config, since class-variance-authority
is not a dependency here, and both stay wrapped in React.forwardRef, which the
tripwire in tests/setupTests.ts requires until the React 19 upgrade

Adds Badge to ref-forwarding.test.tsx. Nothing covered it before, even though
two TooltipTrigger sites compose over it, so the wrapper could have been dropped
by the next re-sync without a single test going red

Retargets one assertion in LogDetailContent.test.tsx. It regex-matched the whole
class string for "destructive" to prove a tag was not alarming red, which the
restored aria-invalid classes now satisfy for every variant; it checks the
variant attribute and red utility classes instead
2026-08-14 17:31:44 -07:00
mateo-berri
05beb7abb5 fix(proxy): emit uncached input cost so component headers sum to the total 2026-08-14 17:30:50 -07:00
mateo-berri
7e539405ed fix(cost-tracking): price web search on dated search-preview map entries 2026-08-14 17:29:56 -07:00
Devin AI
87765fcc76 fix(router): merge model_info without new mutable constructions
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-15 00:29:50 +00:00
milan
0d0c712df7 fix(vertex_ai): fail an embeddings batch entry whose fan-out came back incomplete
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-15 00:29:35 +00:00
yuneng-jiang
0f2566e823
Merge pull request #36984 from BerriAI/litellm_/key-info-badge-styling-b293cc
fix(ui): match the MCP servers count badge to its sibling permission badges
2026-08-14 17:26:31 -07:00
mateo-berri
688575e5bf fix(cost): bill reasoning tokens at the selected tier's reasoning rate 2026-08-14 17:26:29 -07:00
Mateo Wang
a5038661c9
Merge pull request #34850 from BerriAI/litellm_lit_4866_anthropic_geo_cache_uplift
fix(anthropic cost): apply regional geo uplift to cached tokens
2026-08-14 17:26:17 -07:00
Mateo Wang
b9d70b5ef5
Merge pull request #34860 from BerriAI/litellm_lit_4868_cache_write_split
fix(anthropic): aggregate 5m/1h cache-write split across iterations path
2026-08-14 17:25:20 -07:00
mateo-berri
7562445273 test(proxy): assert production nesting semantics for component cost headers 2026-08-14 17:24:10 -07:00