Commit graph

14665 commits

Author SHA1 Message Date
Mateo Wang
c50d83ece2
Merge pull request #39070 from BerriAI/litellm_bedrock_invoke_native_structured_output
fix(bedrock): forward native structured outputs on Invoke instead of silently inlining the schema
2026-09-01 12:19:15 -07:00
Mateo Wang
435433fa07
Merge pull request #39149 from BerriAI/litellm_qwencloud_provider_aliases
feat(dashscope): add QwenCloud and Qwen AI Platform provider aliases
2026-09-01 12:18:05 -07:00
yuneng-jiang
33004d2f0c
test(e2e/ui): cover the Budgets page create, edit and delete flows (#39052)
* test(e2e/ui): cover the Budgets page create, edit and delete flows

The Budgets page had no browser coverage at all, so an admin creating or
editing a spend cap through the UI was only exercised by hand at RC time.

Each test reads the budget back from /budget/list, a different route from
the one the table renders, so a row that only exists in the table's cache
does not pass. The edit test pins the rate limits an unrelated spend-cap
edit has no business touching.

* test(e2e/ui): trim comments that restate the test steps

Review flagged the explanatory comments as restating ordinary setup rather
than explaining anything. Keeps the two that carry the regression rationale
for an assertion and drops the rest.

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-09-01 12:17:02 -07:00
mateo-berri
c21e895fe2 fix(proxy): handle CRLF and CR SSE frame terminators and flush held tail in anthropic stream restamper 2026-09-01 12:16:27 -07:00
mateo-berri
b0751169eb fix(cost): bill off-peak rates for deployments that set only off_peak_pricing
Cost lookup selects the deployment-scoped cost map entry only when custom
pricing is detected and the entry carries a base pricing field. A deployment
whose model_info held nothing but off_peak_pricing failed both conditions, so
its schedule was silently ignored and every request billed at the shared
backend rate.

use_custom_pricing_for_model now also treats deployment-scoped pricing fields
in the metadata model_info as custom pricing, and the router inherits the
backend model's built-in base token rates onto such an entry at registration,
which also lets cache pricing inheritance apply. Regression tests cover the
registration, the detection, and the costed request end to end.
2026-09-01 12:14:46 -07:00
mateo-berri
2adae6b475 fix(prometheus): pass through router-originated labels when no proxy router exists 2026-09-01 12:14:27 -07:00
Mateo Wang
f3dbd253be
Merge pull request #38106 from Timik232/bugfix/streaming-stable-response-id
fix(streaming): keep response id stable across streamed chunks
2026-09-01 12:11:32 -07:00
mateo-berri
98ea5eaab4 fix(responses): correlate streamed tool call events on normalized item ids 2026-09-01 12:08:22 -07:00
mateo
d816b75dd4 feat(bedrock): gate forced tool_choice on supports_forced_tool_use in converse
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-01 19:05:03 +00:00
mateo
6513f5c539 test(utils): allow supports_forced_tool_use in model prices schema test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-01 19:05:03 +00:00
milan
ed4343a026 fix(bedrock): scope client_metadata drop to anthropic converse models
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-01 19:04:32 +00:00
mateo-berri
0042493bca Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_bedrock_invoke_native_structured_output 2026-09-01 11:50:05 -07:00
yuneng-jiang
2b1bd20834
Merge pull request #31125 from BerriAI/litellm_/stoic-jones-7de871
feat(proxy): default to the v2 migration resolver, keep v1 as an opt-out
2026-09-01 11:46:13 -07:00
mateo
3c9ce458fd feat(anthropic): gate forced tool_choice for Fable 5.1 behind supports_forced_tool_use
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-01 18:46:03 +00:00
Mateo Wang
caeb5d181e
Merge pull request #39074 from BerriAI/litellm_fix_websearch_tool_selection_test
test(websearch): register configured search tool in pre-request hook test
2026-09-01 11:45:29 -07:00
mateo-berri
24393be4a6 Merge branch 'litellm_internal_staging' into bugfix/streaming-stable-response-id 2026-09-01 11:45:27 -07:00
Devin AI
2063c29f5d fix(anthropic): upgrade legacy thinking to adaptive on adaptive-only models for chat and Bedrock Converse
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-01 18:45:26 +00:00
mateo
0610331aa1 test(reasoning-effort-grid): bump the cell count for the four new Fable 5.1 cells
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-01 18:28:33 +00:00
mateo
ab549a8da3 test(fallbacks): use an unmapped fable id now that 5.1 is in the cost map
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-01 18:25:18 +00:00
mateo-berri
f3792fb700 feat(dashscope): add qwencloud and qwen_ai_platform provider aliases 2026-09-01 11:20:36 -07:00
mateo-berri
1ba13fcc25 fix(cost): keep off_peak_pricing scoped to its deployment
register_model inserted the first deployment's off_peak_pricing dict by
reference into the shared backend cost-map entry, and later deployments
sharing that backend merged their schedules into the same object,
corrupting the first deployment's schedule and polluting the built-in
entry. Nested dicts now merge copy-on-write, and off_peak_pricing stays
off the shared backend keys.
2026-09-01 11:17:55 -07:00
mateo
fcc203e35b test(reasoning-effort-grid): let the Anthropic Fable 5.1 cells run
Access is confirmed on the Anthropic account, and all 11 cells pass live, so the
xfail marker would only hide real regressions.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-01 18:17:39 +00:00
mateo-berri
93a03a9ffd fix(openai): drop tool_choice when request has no tools on chat completions 2026-09-01 11:16:56 -07:00
Yuneng Jiang
201f60d19c
revert: restore search tool fallback when no router is configured
This reverts commit 65a46a5f32 (#38113)

That change made two edits that combine into a hard failure for SDK
users. It dropped the null-router guard in
_select_search_tool_from_router, so a missing router now yields an empty
search_tools list instead of returning early, and it turned the no-match
case in _select_search_tool_from_list from a debug-logged fallback into
a raised ValueError. It also added a call site in
async_pre_call_deployment_hook that invokes the selection purely for the
side effect of raising, discarding the return value

Used together, any SDK caller that sets search_tool_name and sends a web
search tool now raises "Configured search tool '<name>' was not found"
on every request. There is no way to satisfy the check off the proxy,
because search_tools is only ever populated from the proxy router, so
the SDK path cannot register one

tests/pass_through_unit_tests/test_websearch_interception_e2e.py caught
this, but #38113 only updated the handler unit tests, so the break
landed on staging

Reverting restores the previous behavior while we work out a fix that
keeps the stricter validation on the proxy path, where a silently
substituted search provider is the real problem worth rejecting, without
turning the SDK path into an unconditional error
2026-09-01 11:16:23 -07:00
mateo-berri
ab2c9aed0f fix(responses): normalize tool call id shapes across the anthropic bridge and openai replay
The chat-completions bridge emitted Responses output items whose item ids
were raw Anthropic tool ids (toolu_/srvtoolu_), which OpenAI rejects on
replay with "Expected an ID that begins with 'fc'", breaking router
fallback conversations from gpt-5 to claude models.

Four fixes, composable and independently useful:
- emission: bridge output items get fc_/ctc_-prefixed item ids while
  call_id stays raw so tool_result pairing keeps working (streaming and
  non-streaming share the same helpers)
- openai replay: request transformation drops tool call item ids that do
  not match OpenAI's own shapes instead of forwarding them, gated to
  OpenAI and Azure, since the API accepts the items with no id at all
- anthropic replay: a replayed srvtoolu_ call whose paired server tool
  result is unavailable degrades to a plain client tool_use instead of a
  dangling server_tool_use that 400s the client's tool_result
- tool-only turns no longer emit a message output item with output_text
  text null, matching native OpenAI output
2026-09-01 11:12:24 -07:00
mateo
fb93db7791 feat(models): add Claude Fable 5.1 across Anthropic, Bedrock, Vertex AI, and Azure AI
Adds claude-fable-5-1 cost map entries on the Anthropic API, Bedrock converse
(base, global, and us/eu geo inference profiles at the 10% regional premium),
Vertex AI, and Azure AI. Specs match Fable 5 (1M context, 128K output, $10/$50
per MTok, adaptive thinking always on, xhigh and max effort), except cache reads
land at $0.25 per MTok, a quarter of Fable 5's price and 0.025x base input
instead of the usual 0.1x.

Registers anthropic.claude-fable-5-1 in BEDROCK_CONVERSE_MODELS, lists the model
in the setup wizard, and extends the reasoning effort e2e grid. The partner cells
carry fail_reason markers until access on the CI accounts is confirmed.

Partner entries deliberately carry no deprecation_date: Anthropic publishes
retirement no sooner than 2027-09-01 for the first-party model, and the Foundry
and Vertex dates are not published yet.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-01 18:07:06 +00:00
mateo-berri
fc091c1248 fix(prometheus): keep team alias and team wildcard names out of the other bucket 2026-09-01 11:04:25 -07:00
mateo-berri
3b3099d78d fix(prometheus): bound requested_model label cardinality on client failure paths 2026-09-01 10:34:58 -07:00
Mateo Wang
ef7a984fbb
Merge pull request #38990 from BerriAI/devin_ai_1788201394-veo31-pricing-tiers
fix(registry): veo 3.1 pricing tiers + roll up open registry PRs (glm-5.2, Qwen3.8-Flash, gemma-4-31b, scribe_v2, fireworks/databricks deepseek v4) + deprecation dates
2026-09-01 10:32:59 -07:00
Mateo Wang
25b11301b2
Merge pull request #38891 from BerriAI/litellm_deflake_20260830
test: deflake MCP registry state, savings cost map, and MCP identity env reload tests
2026-09-01 10:15:29 -07:00
mateo-berri
06d4521fc0 fix(registry): add vertex veo 3.1 resolution tier pricing per vertex pricing page 2026-09-01 10:15:18 -07:00
Mateo Wang
6661e915a5
Merge pull request #38774 from BerriAI/litellm_fix_openai_embedding_encoding_format_omit
fix(embeddings): omit encoding_format when the client omits it on OpenAI-compatible calls
2026-09-01 10:13:38 -07:00
Mateo Wang
cfd9d27be2
Merge pull request #39038 from BerriAI/litellm_fix_pipeline_guardrail_list_clobber
fix(policy_engine): restore request guardrails list after pipeline allow
2026-09-01 10:13:30 -07:00
Mateo Wang
b86791b4b9
Merge pull request #38985 from BerriAI/litellm_fix_ocr_annotation_page_cost
fix(cost): bill OCR annotation pages via annotation_cost_per_page
2026-09-01 10:13:01 -07:00
mateo-berri
7abed91523 feat(cost): support day-of-week qualified off-peak windows 2026-09-01 10:11:25 -07:00
Mateo Wang
30bf592aaf
Merge pull request #35417 from BerriAI/litellm_fix_responses_bridge_tool_call_arguments_json
fix(responses): json-encode object tool call arguments in the chat completions bridge
2026-09-01 10:08:12 -07:00
milan
ab11613441 fix(bedrock): strip client_metadata from converse additionalModelRequestFields
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-01 16:50:24 +00:00
ryan-crabbe-berri
190b8c7d8e test(e2e): drop coverage registry cell for the alias-grant test 2026-09-01 09:03:47 -07:00
ryan-crabbe-berri
4118db5a50 test: drop litellm-internal patches from the cross-region alias test (TQ008)
The pure helpers express the same regression: the save-side drop must
leave the alias in place, and two registries must expand it to their
own ids. The full validate path is already covered by the
persists-verbatim test and the live e2e test.
2026-09-01 08:23:26 -07:00
ryan-crabbe-berri
7a761ccf5a test(e2e): cover alias MCP grant persisting verbatim on key generate
A single-instance run cannot reproduce the two-region setup, but the
regression is fully visible in one: the alias must survive to /key/info
unrewritten, and the alias-granted key must still list the server's
tools. The broken write path stored the resolved server id instead.
2026-09-01 08:10:41 -07:00
ryan-crabbe-berri
0d7035989c fix(mcp): persist alias MCP grants verbatim instead of rewriting to local server ids
Since PR #29128, key create/update/regenerate resolved every
object_permission.mcp_servers entry against the saving instance's
DB + config registry and persisted the resolved server ids. For
config-loaded servers the id is derived from a hash of the regional
URL, so in a shared-database multi-region deployment the rewrite baked
one region's ids into the row and every other region denied the key.
Grants written before v1.88.0 kept the raw alias and kept working,
which is why only newly provisioned keys broke.

Keep the validation and the stale-entry drop (the LIT-3278 fix), but
persist the caller's original identifiers for everything that resolves.
Read-time expand_permission_list already maps a name to each region's
local server id.
2026-09-01 08:10:36 -07:00
Devin AI
9a1aebc146 fix(registry): declare databricks deepseek cache-write rate at the input rate per repo convention
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-01 13:50:21 +00:00
Devin AI
7bfa0d7fb4 Registry audit: Fireworks DeepSeek V4 Flash 0731 pricing, Databricks DeepSeek V4 entries, provider deprecation dates
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-01 13:15:22 +00:00
Devin AI
5f44bdd1c1 test: trim mcp fixture docstring and reload comment
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-01 11:06:08 +00:00
Devin AI
95c7ca8801 fix(tests): restore module attributes after reload in mcp identity env tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-01 10:59:08 +00:00
Kris Xia
7ca035f310
fix(gemini): return enabled thinking content by default 2026-09-01 17:52:37 +08:00
Devin AI
2019c309ea merge: absorb litellm_internal_staging; drop python duplicate-issues closer fix superseded by ts rewrite
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-01 09:18:31 +00:00
Devin AI
d4fc54a11d chore(techdebt): clear fresh debt from the 2026-08-31 window
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-01 08:25:42 +00:00
Mateo Wang
ec3f8183c3
Merge pull request #39066 from BerriAI/litellm_aws_external_id_files_batches
fix(bedrock): forward aws_external_id in files and batches credential loading
2026-08-31 23:31:29 -07:00
Mateo Wang
9f6024bf97
Merge pull request #39044 from BerriAI/litellm_fix_bedrock_guardrail_token_log
fix(bedrock): mask signed request headers in guardrail debug log
2026-08-31 23:30:14 -07:00