Commit graph

20246 commits

Author SHA1 Message Date
Mrinal Chanshetty
68fe044b32 test: add the connected_proxy_prisma integration helper
Prerequisite for the #47190 integration test on stable/1.106.x. Copied unchanged from tests/integration/_support/prisma_clients.py on main, added there by #45939.
2026-10-20 17:24:32 -07:00
joshua-berri
8016cfa249 fix(batch): preserve recoverable costs and prevent poll starvation (#47190)
Backport of #47190 to stable/1.106.x.
Cherry-picked from 0ede0e10e7 (main).
Hand-ported onto the line's poller. ManagedObjectRow lives in check_batch_cost.py on this line, so created_at and updated_at are added to its local _ManagedObjectRow. _finalize_unbilled_terminal_job keeps the line's None return and skips the log when the snapshot no longer matches. Test hunks for TestBatchLineItemsFromThePoller are dropped, and the appended tests do not assert the bool return.
2026-10-20 16:55:54 -07:00
Kerry Lu
568ccb1830 fix(logging): report the cost-map entry billing used in model_map_information (#47182)
Backport of #47182 to stable/1.106.x.
Cherry-picked from 1212260ab1 (main).
Test conflict resolved by keeping the line's file plus this PR's own tests.
2026-10-20 16:55:54 -07:00
joshua-berri
c0009b0e8b fix(batches): preserve spend across malformed output and metadata failures (#47184)
Backport of #47184 to stable/1.106.x.
Cherry-picked from 3fb95c8fe3 (main).
Hand-ported onto the line's poller: the claim now writes status and file_object with batch_processed and the release restores them, and _ManagedObjectRow gains status. The line has no postgres_storable_json, so the NUL-stripping sanitizer (_storable_text, _storable_json_value, postgres_storable_json, from #45703 on main) is added to managed_files.py and the claim stores postgres_storable_json(response). Test hunks for TestBatchLineItemsFromThePoller (from #41691, not on the line) are dropped.
2026-10-20 16:55:53 -07:00
joshua-berri
99341ad05d fix(batches): handle empty models when logging batch cost (#47155)
Backport of #47155 to stable/1.106.x.
Cherry-picked from 4143e0869e (main).
2026-10-20 16:04:53 -07:00
devin-ai-integration[bot]
465da549ef fix(anthropic): add the dangerous-tool-use beta when a /v1/messages request carries safeguards (#43955)
Backport of #43955 to stable/1.106.x.
Cherry-picked from eb5295b20c (main).
Adapted to the line: the existing CLI-beta wire assertion change is dropped since that test is not on the line.
2026-10-20 16:04:53 -07:00
mrinal-berri
328bad6bc8 fix(databricks): send a single plain text block as a string for non-Claude models (#45686)
Backport of #45686 to stable/1.106.x.
Cherry-picked from 1aee1a21ed (main).
2026-10-20 15:50:22 -07:00
mrinal-berri
195c2b8e15 fix(databricks): keep #/$defs refs in json_schema response_format (#45659)
Backport of #45659 to stable/1.106.x.
Cherry-picked from fa7ad80a62 (main).
2026-10-20 15:50:22 -07:00
Yuneng Jiang
1e23ad4791
fix(anthropic): forward thinking.block_binding only from the exact Anthropic, Azure AI and Bedrock Invoke configs
e46904b546 decided by resolved provider name, so the OpenAI-compatible
/v1/messages passthrough and Bedrock Claude Platform /v1/messages, which
inherit the "anthropic" provider name, and the Bedrock Mantle chat and
/v1/messages configs, which inherit the Bedrock Invoke override, still sent
block_binding where 1.106.0 sent {"type": "adaptive"}. Each in-scope config
now opts in for its own class only, and the Bedrock Invoke display: updates
and clear_thinking rebuilds keep block_binding only when the config carries it
2026-10-20 02:42:05 -07:00
Yuneng Jiang
e46904b546
fix(anthropic): carry only thinking.block_binding through the legacy thinking rebuild
The #46342 pick spread every caller key from a legacy thinking.type=enabled
request into the rebuilt adaptive param, so block_binding and any unknown key
newly reached Bedrock Converse, Vertex AI and Databricks, where 1.106.0 sent
only {"type": "adaptive"}. The rebuild now carries block_binding alone, and
only for Anthropic, Azure AI and Bedrock Invoke. The Bedrock Invoke
display: updates and clear_thinking rebuilds keep block_binding and drop the
other caller keys, as 1.106.0 did
2026-10-20 01:20:54 -07:00
joshua-berri
699e79c04e
Merge pull request #47454 from BerriAI/litellm_restore_token_count_input_precedence
fix(proxy): restore legacy token count input precedence

(cherry picked from commit 5346a40c37)
2026-10-19 23:07:36 -07:00
Mateo Wang
98d702fba9
Merge pull request #47446 from BerriAI/litellm_backport_google_count_shapes_stable_1_104_x
fix(count_tokens): restore Google request compatibility on stable/1.104.x

(cherry picked from commit a63df167da)
2026-10-19 23:07:00 -07:00
yuneng-jiang
978176398a
Merge pull request #47389 from BerriAI/litellm_backport_count_tokens_stable_1_104_x
fix(count_tokens): backport October 19 fixes to stable/1.104.x

(cherry picked from commit 5e4d160423)
2026-10-19 23:06:47 -07:00
Mrunmayee Daware
07f72a67b1
fix(proxy): authorize the public name of a team-scoped model on batch retrieve/cancel (#47067)
* fix(proxy): authorize the public name of a team-scoped model on batch retrieve/cancel

A unified batch id resolves to the deployment's internal model_name
(model_name_{team_id}_{uuid}), which is in no key or team allowlist, so
authorize_model_for_key always denied and team keys got 403 retrieving or
cancelling their own batches. Authorize against model_info
team_public_model_name when the resolved deployment is team-scoped, falling
back to the resolved name otherwise.

Fixes #46958

* fix(proxy): authorize team-scoped batch models by their public name on both auth paths

* fix(proxy): accept the owning team's public name for its own team deployment on every auth path

Managed batch, file, fine-tuning, vector store, character and video ids resolve to the
team deployment's internal name again, so teams that granted the internal name keep access.
The model access check now also accepts the owning team's public name and its legacy team
aliases for that internal name, and only when the caller's team owns the deployment.

* fix(proxy): match a team deployment's public and alias names against the caller's own grants

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
(cherry picked from commit e979bd0ecf)
2026-10-19 22:45:17 -07:00
yuneng-jiang
da5b2b556b
fix(vector-stores): keep proxy context out of provider vector store metadata (#47021)
* fix(vector-stores): keep proxy context out of provider vector store metadata

The proxy merged its request context (user_api_key_* fields, endpoint,
user_agent, requester_ip_address, litellm_api_version) and the router's
deployment info into the vector store request's metadata. For vector
stores that field is also OpenAI's own metadata, which is capped at 16
keys, so a store created with a team key was full on creation and every
later metadata update returned 400.

Vector store routes now keep that context in litellm_metadata, like
batches, files and responses, and the router does the same for
avector_store_create. Provider-side metadata holds only what the caller
sent.

Adds db_scripts/strip_vector_store_internal_metadata.py, an opt-in
script that removes the proxy-written keys from stores created before
this change. It is a dry run unless --apply is passed.

* fix(vector-stores): page the metadata cleanup script without recursion and tolerate non-string values

Listing every store recursed once per page, which overflows the interpreter stack on large projects. Pages are now walked iteratively. Metadata values are no longer required to be strings, and a malformed response exits 2 with a fixed message instead of a traceback that could echo metadata values.

* fix(vector-stores): read proxy agent id, tags and spend_logs_metadata from litellm_metadata

Vector store routes now keep proxy context in litellm_metadata, but the v3
limiter only read agent_id from metadata, and logging let provider metadata
tags or spend_logs_metadata replace the proxy's values. A string tags value
also crashed get_request_tags.

The limiter now prefers the proxy-resolved agent_id in litellm_metadata.
Logging combines list tags from both buckets (metadata first, deduplicated)
and merges dict spend_logs_metadata with litellm_metadata winning, ignoring
values of the wrong type.

* fix(vector-stores): narrow the logging fallback to mistyped caller values

The tag union and spend_logs_metadata merge changed logged tags and
spend_logs_metadata on every route that has both metadata buckets, deduped
tags quadratically, and stopped background interaction polling from keeping
string tags as sent. Caller values now win exactly as before, and the
proxy's litellm_metadata tags or spend_logs_metadata only replace a caller
value of the wrong type.

The limiter reads agent ids through a pydantic model instead of a cast and
ignores non-string ids.

* fix(vector-stores): log the proxy's full tag list on litellm_metadata routes

On routes that keep proxy context in litellm_metadata, the proxy builds
litellm_metadata.tags from the caller's tags plus key, team and header
tags, but get_request_tags preferred the caller's metadata.tags. A caller
sending metadata.tags on a vector store call dropped the key's tags from
request_tags, which tag spend tracking reads. get_request_tags now prefers
a non-empty litellm_metadata tag list, and falls back to metadata.tags as
before.

* fix(vector-stores): strip LiteLLM-written keys at the provider boundary instead of moving metadata buckets

Moving vector store proxy context into litellm_metadata broke shared readers
(agent limits, request tags, merged logging metadata), and fixing them
widened the change to every litellm_metadata route. This reverts the bucket
move, the router change and the reader fixes, so proxy context stays in
metadata exactly as on main.

Vector store create and update now drop the keys LiteLLM itself writes
(user_api_key*, requester and deployment keys, agent ids) from the metadata
sent to the provider, in the shared optional-param helper both paths use.
The cleanup script reuses the same key list.

* fix(vector-stores): pass non-string metadata keys through the provider metadata filter

* fix(vector-stores): strip LiteLLM-written metadata keys only on proxy and router calls

* fix(vector-stores): send the provider exactly the metadata the caller sent through the proxy

* fix(vector-stores): snapshot caller metadata from the raw request body and keep it out of logged request bodies

* fix(vector-stores): copy the parsed-body metadata when the raw request body cannot be reparsed

(cherry picked from commit 4846cfabc7)
2026-10-19 22:43:02 -07:00
ryan-crabbe-berri
4f538f48c8
fix(anthropic): keep signed empty thinking blocks and honor thinking.block_binding (#46342)
* fix(anthropic): treat adaptive thinking as enabled when mapping response_format

is_thinking_enabled only recognised thinking.type=enabled, so a request that
paired thinking.type=adaptive with response_format took the synthetic JSON
tool path with a forced tool_choice, which Anthropic rejects alongside
thinking. The same check also gates the Bedrock Converse and Databricks
forced tool_choice downgrades, so they now see adaptive thinking too.

* fix(anthropic): keep signed thinking blocks with empty text on chat completions

Models that default to display: omitted return signed thinking blocks whose
text is empty, and Anthropic rejects the next turn when they are not replayed
unchanged. is_unsignable_thinking_block dropped them along with unsigned
blocks, and the inline content path required non-empty text, so a tool-use
turn on Opus 4.7+ or the 5 families lost its thinking blocks and 400ed on the
tool-result turn. Only a missing signature now drops a block, and the
mid-conversation system placement mirrors that rule. The /v1/messages strip
helper is left to #41362.

* fix(anthropic): skip the thinking-strip retry when block_binding asks for an error

The /v1/messages retry loop strips thinking blocks and resends on any
invalid-thinking 400, including the binding mismatch a caller opted into
with thinking.block_binding.prefix_mismatch_behavior: error. The retry
decision now sees the request body and surfaces that 400 instead of
overriding the caller's choice. #42888 covers the litellm_params side.

* fix(bedrock): carry thinking.block_binding through the Invoke thinking rebuilds

Bedrock Invoke rebuilt the thinking param from scratch when display: updates
was requested, on both the /v1/messages and chat completions paths, and the
legacy enabled-to-adaptive translation plus the clear_thinking injection did
the same, so thinking.block_binding never reached Bedrock. Every rebuild now
spreads the caller's thinking and only overrides the keys it owns, and
AnthropicThinkingParam types block_binding. The thinking-binding-controls
beta was already on the Bedrock allowlist.

* fix(anthropic): keep forcing the response_format tool under adaptive thinking

This reverts ad9e861ae7. Anthropic, Bedrock Invoke and Vertex all accept a
forced tool_choice next to thinking.type=adaptive and only reject it with
type=enabled, so counting adaptive as enabled fixed no live failure. It did
drop the forced JSON tool, so Claude answered response_format requests in
prose, and it downgraded a required tool_choice to auto on Bedrock Converse

* test(e2e): drop the adaptive thinking tool round trip case

With adaptive thinking the model often skips thinking on the weather prompt, so
turn 1 carries no signed empty block and the case fails before the replay it is
meant to cover. Anthropic also answers the replay with a 200 whether or not the
block is forwarded, so the case could not tell the fix from main. The factory
unit tests pin the outgoing shape instead

* fix(anthropic): only skip the thinking-strip retry for a binding mismatch 400

prefix_mismatch_behavior: error asks Anthropic to reject a block bound to
another conversation. It says nothing about unrelated bad blocks such as a
null signature, so those still get the strip-and-retry self-heal. Also trims
the empty-thinking docstrings to what the code guarantees and types the new
test parameters

* refactor(anthropic): narrow thinking fields with the mapping validator instead of typing.cast

typing.cast is banned by the strict ruff gate (TID251), so the three new casts pushed that rule one over its merge-base count. The existing pydantic mapping validator gives the same typed view without an unchecked cast, and it now has a generic name since it is no longer only used for Claude Code payloads

* refactor(anthropic): read thinking settings through the validated mapping so basedpyright stays at its merge-base count

The Bedrock clear_thinking rewrite still narrowed with a cast and an untyped dict fallback, which added two reportUnknownVariableType errors. It now uses the same pydantic mapping validator, made public for that reuse, and the retry hook validates the request body itself so the handler passes a typed dict instead of an untyped one

---------

Co-authored-by: yuneng-jiang <yuneng@berri.ai>
(cherry picked from commit f470e418ea)
2026-10-19 22:42:04 -07:00
Chloe Lu
cbe5c288b8
fix(anthropic): keep signed empty thinking blocks on /v1/messages requests (#41362)
The request-side sanitizer on /v1/messages dropped every thinking block whose
text is empty, even with a valid signature. Claude Opus 4.8 / Fable 5 / Fable 5.1
return exactly that shape by default (thinking.display omitted): the signature
is the preserved reasoning. Replaying the assistant turn as returned therefore
lost the model's reasoning on every multi-turn request, silently (the provider
answers 200 without it), and left preserved-thinking enforcement
(thinking.block_binding) nothing to bind.

Anthropic accepts an empty thinking block that carries a signature and binds it
to its prefix; only an empty block with no signature (the LIT-6357 case, a turn
a non-Anthropic model served through the bridge) is rejected with "each
thinking block must contain thinking". Use the signature-aware predicate
is_empty_unsigned_thinking_block, which #38809 already applied on the emit
side, for the request side too. The unsigned case is still dropped.

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit 897bfea38d)
2026-10-19 22:29:53 -07:00
Yassin Kortam
1c6e82a60e
fix(mcp): resolve a bare tool name on a single-server scope against that server (#46225)
A JSON-RPC tools/call on /{server_name}/mcp (or /mcp with x-mcp-servers naming one server) that uses the upstream's bare tool name answered "Tool not found" on every proxy process that had not yet listed that server, because the per-process tool-name map stays empty for a server whose upstream only lists tools with the caller's own credentials, and mcp SDK 2 no longer lists before an uncached call. call_mcp_tool now hands the single scoped server to execute_mcp_tool as requested_server_id, so the existing REST contract lists the server with the caller's headers and routes the bare name to it, and a bare name shared by two servers goes to the one the path names

Co-authored-by: aayushbaluni <73417844+aayushbaluni@users.noreply.github.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit ec28acf322)
2026-10-19 22:29:02 -07:00
devin-ai-integration[bot]
35f52bf3e8
fix(responses): fold bridged Responses output into one chat choice (#46099)
* fix(responses): fold bridged Responses output into one chat choice

A Chat Completions request bridged to /responses got one choice per
Responses message item, so a reply shaped [message, message,
function_call] put the tool call on choices[1] and clients reading
only choices[0] never saw it. Fold every message text, annotation,
reasoning item and tool call into choices[0]

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(responses): drop mutable annotations and the extra cast in the choice fold

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(responses): remove a cast-ok marker that suppressed nothing

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(responses): join folded message texts with a newline

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* revert(responses): join folded message texts with no separator so stream and non-stream match

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit 4b7b8efd4c)
2026-10-19 22:27:55 -07:00
yuneng-jiang
955bdaafa8
test(e2e): backport three stale e2e test fixes from main to rc/1.106.0 (#47229)
Takes the main version of each file, which is the rc copy plus one main commit:
- test_image_generation_e2e.py from #46998 (Nova Canvas reached end of life)
- test_bedrock_converse.py from #45530 (ask an uncached model once more)
- migratedPages.spec.ts from #45863 (/ui/ lands on Home since #45858)
2026-10-17 20:22:41 -07:00
yuneng-jiang
d50932dea7
fix(caching): backport #45784 and #46488 to rc/1.106.0 (#47006)
* fix(caching): give RedisClusterCache one async client per event loop (#45784)

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit cb73bad48a)

* fix(caching): make Redis Cluster commands wait for a pool slot instead of tripping the circuit breaker (#46488)

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit ca2f99edac)

---------

Co-authored-by: Yassin Kortam <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-16 15:34:17 -07:00
Mateo Wang
574ad49442 fix(proxy): check previous_response_id ownership on /v1/responses/compact (#46408)
* fix(proxy): check previous_response_id ownership on /v1/responses/compact

* test(proxy): cover the unmanaged response id escape hatches on every response id route

* test(proxy): audit compact previous_response_id ownership

Add integration cells for the native /v1/responses/compact routes covering owner,
admin, cross-tenant, unmanaged, malformed, duplicated and retained ids, provider
errors, the response cache, id-security settings, an upstream outage, a held
upstream, a worker kill and a proxy restart. Unskip the provider wire cell whose
bug this branch fixes.

* test(proxy): cover the no signing key path on every response id route

Backport of #46408 to rc/1.106.0
Cherry-picked from 389403fede (main)
2026-10-15 20:04:56 -07:00
Mateo Wang
f3904e2e65 feat(typesafe): support Jev 1.14 images, box questions, and long-context headers (#46596)
* feat(typesafe): support Jev 1.14 images, box questions, and long-context headers

* fix(cost): drop the typesafe/jev-1.14.0 entry main already prices

* fix(typesafe): send images only with the images header and box questions only to models that answer them

TypeSafe takes assets and input_image parts only when the request carries
TypeSafe-Danger-Zone-Images: true, so without it they follow drop_params like
any other provider that cannot take images. Box support now comes from a
supports_box_questions cost-map flag set on jev-1.14.0 and jev-preview, so
jev-1.13.0 and jev-latest refuse or drop a box question before the call. A
malformed provider_specific_header answers 400 instead of 500.

* fix(typesafe): allow null box confidence and validate asset references and base64

* fix(typesafe): gate box questions on the images header, drop unreadable assets under drop_params, forward opt-in headers on the pass-through

---------

Backport of #46596 to rc/1.106.0
Cherry-picked from 3fec5075fa (main)

Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-15 06:32:44 +00:00
Mike Hyams
a89c618509 fix(cost): price typesafe/jev-1.14.0 (#46594)
* fix(cost): price typesafe/jev-1.14.0

TypeSafe released jev-1.14.0 on 2026-10-14 at the same price as jev-1.13.0
($0.042 per 1M input tokens, output free; https://docs.typesafe.ai/models).
Without an entry, calls through the TypeSafe pass-through log a cost of 0:
TypeSafePassthroughLoggingHandler prices from litellm.model_cost["typesafe/<model>"],
and a deployment's model_info price is registered under its model id only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(cost): add max_input_tokens to the typesafe jev rows

---------

Backport of #46594 to rc/1.106.0
Cherry-picked from 9d72bae336 (main)

Co-authored-by: Mike Hyams <241361536+hyamie@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-15 06:21:45 +00:00
devin-ai-integration[bot]
0ac0a42e31 feat(decisions)!: send images to Cloudflare Clef through its images field (#45734)
* feat(decisions): send images to Cloudflare Clef

OpenAI-format input_image parts and the System One images field now reach Cloudflare clef and clef-flash in its images field, with remote URLs downloaded and inlined first. Providers that cannot take images drop them under drop_params and answer 400 otherwise.

* fix(decisions): follow drop_params for images and use the call timeout for image downloads

* fix(decisions): drop a malformed images entry on its own under drop_params

* fix(decisions): refuse images next to input whatever drop_params says

* fix(decisions): bound clef image fetches, honor drop_params for images, and log caller URLs

Refuse more than 4 images for Cloudflare clef before any download, keep the
caller's timeout as one deadline across image fetch retries, accept images
next to input under drop_params, validate image strings and objects, and pass
the caller's image URLs (not inlined bytes) to logging callbacks.

* fix(decisions): check input_image URLs and clef image sizes

* fix(decisions): accept data URL media type parameters and check clef image types

* fix(decisions): count clef images after dropping unsupported types and flag clef vision

---------

Backport of #45734 to rc/1.106.0
Cherry-picked from 2f3ccbe7c7 (main)

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-10-15 06:21:45 +00:00
moyai-devin-berriai[bot]
4b94671535 feat(decisions): support Clef-omni audio and video (#45811)
* feat(decisions): support Clef-omni audio and video

* fix(decisions): accept Clef-omni media types with parameters and cover the proxy routes

The clip patterns now mirror Cloudflare's input schema, so a data URL or
content_type carrying parameters like audio/webm;codecs=opus is accepted.
The four proxy routes get media tests of their own, the backup cost map
is synced with the new Clef-omni rows, and the docs page moves to the
docs repo as a follow-up.

* fix(decisions): refuse parameterized media-type data URLs and refresh the OpenAPI snapshot

Cloudflare answers 422 to a data URL whose media type carries parameters even though its schema only requires the data:audio/ prefix, so LiteLLM now refuses those with a 400 before the call. The object form keeps passing parameters through, which Cloudflare accepts. The lazy OpenAPI snapshot and the dashboard API types are regenerated for the new audio and videos fields

* fix(decisions): gate media by cost-map flags, treat null clips as none, drop unsupported clips before validation

* fix(decisions): keep the SDK's positional order and gate media flags on evaluation-mode models

* fix(decisions): gate media on both decisions model modes and move the audit cells onto the harness helpers

* test(decisions): cover the Fireworks System One router in the media refusal cells

---------

Backport of #45811 to rc/1.106.0
Cherry-picked from a86ad9ebf8 (main)

Co-authored-by: moyai-devin-berriai[bot] <336287033+moyai-devin-berriai[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-10-15 06:21:41 +00:00
devin-ai-integration[bot]
9c6b0f1112 feat(decisions): accept decisions as a model mode alongside evaluation (#45474)
* refactor(decisions): rename the evaluation model mode to decisions

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(decisions): canonicalize the health check mode without rebinding

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(decisions): replace canonical_model_mode with is_decisions_model_mode

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(decisions): preserve evaluation mode compatibility

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* chore: drop the rebuilt dashboard bundle from this PR

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* style(dashboard): format decision mode helper

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Backport of #45474 to rc/1.106.0
Cherry-picked from 4c532a5713 (main)

Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-15 06:21:36 +00:00
yucheng-berri
52b9e783a8
fix(prometheus): skip custom metadata labels a metric already carries (#46267) (#46491)
Some checks are pending
LiteLLM Rust / rust-lint (push) Waiting to run
LiteLLM Rust / rust-test (push) Waiting to run
LiteLLM Rust / rust-wheel (push) Waiting to run
* fix(prometheus): skip custom metadata labels a metric already carries

A custom metadata label whose sanitized name matches one of a metric's
built-in labels was appended a second time, so every .labels() call on
that metric raised ValueError. With custom_prometheus_metadata_labels:
[project_alias], the per-project per-model rate limit gauges raised on
every request from a project key with per-model limits, and the rest of
the success event (latency, deployment success, total requests) was
skipped.

* test(prometheus): cover shared custom labels on every endpoint, failure, metadata and outage burst

* test(prometheus): drain the shared upstream at the start of each integration test

* test(prometheus): assert the forwarded route, model and stream per endpoint cell

---------

Backport of #46267 to rc/1.106.0
Cherry-picked from b581ebc405 (main)

Co-authored-by: yucheng-berri <293926326+yucheng-berri@users.noreply.github.com>
2026-10-14 12:10:38 -07:00
moyai-devin-berriai[bot]
29e0219bec fix(guardrails): honor decision model streaming buffering (#46086)
Backport of #46086 to rc/1.106.0
Cherry-picked from 0c04d0bdb7 (main)
Adapted to the line: the list_value test helper comes from main (#43981, not on rc/1.106.0), and the pipeline test keeps only its buffered cases, because the explicitly unbuffered case needs #45516, which is not on this line
2026-10-14 03:29:39 +00:00
Kerry Lu
c57e29a632
feat(ui): source Home What's new from day 0 support blog posts (#46260) (#46309)
Backport of #46260 to rc/1.106.0.
Cherry-picked from d7b58126a7 (main).

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-13 15:44:18 -07:00
mateo
4bfec53865 test(home): pin whats_new_url to the env var and regenerate the dashboard API types
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-11 05:57:04 +00:00
mateo
4dd96800e8 fix(home): serve the bundled whats_new.json by default so a release shows only its own launches
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-11 05:43:27 +00:00
joshua-berri
69738f56db
feat(ui): backport decision auto-router setup to 1.106 (#45916)
* feat(auto-router): backport decision model creation flow to rc 1.106

* fix(ui): preserve deployed decision model editing

* test(ui): use supported decision picker query options

* fix(ui): align decision router backport with release prerequisites

* style(ui): format decision model selection regression

* test(router): verify deployed classifier cost and provider accounting

---------

Co-authored-by: Joshua Valluru <326636767+joshua-berri@users.noreply.github.com>
2026-10-10 22:35:10 -07:00
Mateo Wang
d5a344411f
Merge pull request #45911 from BerriAI/litellm_backport_home_page_rc_1_106_0
feat(ui): backport the Home page dashboard root to rc/1.106.0
2026-10-10 22:24:50 -07:00
Mateo Wang
7b406378bd
Merge pull request #45891 from BerriAI/litellm_backport_decisions_ui_rc_1_106_0
feat(decisions): backport the rest of the decisions and System One launch to rc/1.106.0
2026-10-10 22:15:21 -07:00
mateo
d5b0ad087c test(unit): drop the GLOBAL_LOGGING_WORKER import duplicated by the conftest pick
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-11 04:26:28 +00:00
mateo
cd141c6d36 test(unit): flush queued logging tasks after each test so systemone callback checks stay isolated
Ports the pytest_runtest_call hook from tests/unit/conftest.py on main (#44146). Without it the asystemone call type test picks up adecisions callbacks queued by earlier tests in the same worker

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit 23fec69be2)
2026-10-11 04:15:19 +00:00
devin-ai-integration[bot]
025f451a9a feat(ui): load Home page What's new launches from /public/whats_new at runtime (#45912)
* feat(ui): load the Home page What's new launches from /public/whats_new at runtime

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* feat(ui): show the three newest What's new launches

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* build: ship whats_new.json in the wheel

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit 55c89977d7)
2026-10-11 04:10:00 +00:00
devin-ai-integration[bot]
b0f9d9f785 feat(ui): add a Home page as the dashboard root with usage summary, recent launches, blog updates and a model leaderboard (#45858)
* feat(ui): add a Home page as the dashboard root with usage summary, recent launches, blog updates and a model leaderboard

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(ui): fall back to spend-log request counts on Home when the gateway has recorded none

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(ui): cover the Home page totals, launches, leaderboard and updates feed

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* feat(ui): render Home Updates as rows and return up to 6 blog posts from /public/litellm_blog_posts

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* feat(ui): filter Home Updates by the new-model and api-change blog tags

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(blog): import NotRequired from typing_extensions for py3.10 and test the endpoint passes every fetched post through

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* revert(ui): drop the Updates filter chips, keep Updates as a plain row list

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* chore(ui): restore generated schema.d.ts to main

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* feat(ui): label the Home key button Create New Virtual Key

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* chore(ui): drop the What's new sublabel on Home

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(ui): show an error with retry when Home usage fails to load and move the Home test to the integration tier

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit 9e2aae8490)
2026-10-11 02:12:08 +00:00
joshua-berri
1d18cc43e3
fix(mcp): backport request-local guardrail bearers to RC 106 (#45901)
Some checks are pending
LiteLLM Rust / rust-lint (push) Waiting to run
LiteLLM Rust / rust-test (push) Waiting to run
LiteLLM Rust / rust-wheel (push) Waiting to run
* fix(mcp): preserve caller bearer for guardrail evaluation (#45746)

Co-authored-by: Joshua Valluru <326636767+joshua-berri@users.noreply.github.com>
(cherry picked from commit f57906c124)

* fix(mcp): exclude gateway credentials from guardrail bearers

(cherry picked from commit 77444a7e3b)

* fix(mcp): type legacy JWT claims for RC checks

---------

Co-authored-by: Joshua Valluru <326636767+joshua-berri@users.noreply.github.com>
2026-10-10 18:02:53 -07:00
yuneng-jiang
96b182ff42
fix(credentials): backport credential edit fixes to rc/1.106.0 (#45890)
Backport #45835 from b2cc5f25f6

Keep RC provider support while adapting credential form callbacks and regression tests
2026-10-10 16:39:52 -07:00
devin-ai-integration[bot]
bc924c850c feat(sdk): add litellm.systemone and trim litellm.decisions to the OpenAI shape (#45846)
Backport of #45846 to rc/1.106.0.
Cherry-picked from 25d9b05386 (main).
2026-10-10 16:32:22 -07:00
devin-ai-integration[bot]
0258ae608b test(decisions): bill the OpenAI-format openrouter case from provider-reported cost (#45853)
Backport of #45853 to rc/1.106.0.
Cherry-picked from 5894968c5d (main).
2026-10-10 16:30:26 -07:00
devin-ai-integration[bot]
7318f83976 docs(proxy): describe /v1/decisions and /v1/systemone in Swagger UI (#45838)
Backport of #45838 to rc/1.106.0.
Cherry-picked from bc2cc42340 (main).
2026-10-10 16:30:24 -07:00
devin-ai-integration[bot]
a978acffb3 feat(strands_decider): call a Strands Decider on an AgentCore runtime ARN (#45678)
Backport of #45678 to rc/1.106.0.
Cherry-picked from ed9c31cfa8 (main).
2026-10-10 16:29:33 -07:00
devin-ai-integration[bot]
3880fc285b fix(decisions)!: refuse safety_identifier on providers that cannot take it unless drop_params drops it (#44955)
Backport of #44955 to rc/1.106.0.
Cherry-picked from 546402c98c (main).
2026-10-10 16:29:32 -07:00
Mateo Wang
3c42f96a9d feat(decisions): add Databricks ai_decide as a /v1/decisions provider and auto-router decider (#45200)
Backport of #45200 to rc/1.106.0.
Cherry-picked from 883e3210ee (main).
2026-10-10 16:29:32 -07:00
devin-ai-integration[bot]
ddd4ee61c3 test(decisions): add hosted_vllm /v1/decisions and provider-400 translation cases (#45661)
Backport of #45661 to rc/1.106.0.
Cherry-picked from 097d018dbf (main).
2026-10-10 16:29:18 -07:00
devin-ai-integration[bot]
30ed6e1dd6 fix(openrouter): bill responses, decisions and pass-through from usage.cost and always allow reasoning params (#45640)
Backport of #45640 to rc/1.106.0.
Cherry-picked from 465f20f274 (main).
2026-10-10 16:29:18 -07:00
devin-ai-integration[bot]
6c23c27bca feat(decisions): add hosted_vllm provider (#45501)
Backport of #45501 to rc/1.106.0.
Cherry-picked from 61e5f2dd3c (main).
2026-10-10 16:29:01 -07:00