Backport of #50255 to lts/1.108.x.
Cherry-picked from 6fc9555f83 (main), adapted by hand: the line's text-to-speech module has no Gemini TTS code, so the Gemini parts of the pick are dropped and chirp takes its host from the deployment's or call's own vertex_location only.
Sorts two imports ruff flags (I001) in redis_update_buffer.py and utils.py, the same hunks #50348 carries, and drops a duplicate GenericStreamingChunk import (F811) in a streaming test. No behavior change.
A /v1/responses batch row went through the Responses-to-Chat bridge, which
keeps only Responses fields, so the row's own drop_params was lost while its
service_tier still reached the Nova mapper. An unknown tier then failed the
file upload even with drop_params: true on the row.
The row's drop_params now rides along to the mapper, and the passthrough
branch keeps it out of the modelInput sent to Bedrock.
Sorts two imports ruff flags (I001) in redis_update_buffer.py and utils.py, the same hunks #50348 carries, and drops a duplicate GenericStreamingChunk import (F811) in a streaming test. No behavior change.
Converse used to drop any service_tier outside default, flex, and priority
with no error, rewrite auto to default, and drop reserved. Now reserved and
standard_only map to Bedrock tiers, auto sends no tier, and any other value
answers 400 unless drop_params is on. Nova batch rows honor the row's
drop_params.
Backport to lts/1.108.x of ce94d2894d (litellm_bedrock_converse_service_tier_values), whose main PR is pending.
Adapted to the line: lts has no REQUEST_PARAMS, so the call passes the value through TypeAdapter(object) and keeps the line's two-argument requestMetadata call.
Backport of #43086 to lts/1.108.x.
Cherry-picked from 08f2a98250 (main).
Conflicts resolved for the line: update_headers_with_filtered_beta keeps the line's (headers, provider) signature and gains only keep=, the messages handler passes keep= at the line's existing filter call, DEPLOYMENT_SCOPED_CAPABILITY_FIELDS adds supports_per_turn_control to the line's own set, the Invoke config (anthropic_claude3_transformation.py on the line) gains the _MODEL_INFO adapter main already had, and the test file gains main's ScriptedBedrockInvoke and _ANTHROPIC_MESSAGE helpers the new tests use.
DeepSeek now sends the params through its own tool cleanup when a hosted MCP
tool is dropped, so a tool_choice naming that tool and the tool selection params
beside legacy functions are cleared as the line does today.
The MCP security guardrail uses the gateway's saved toolset check, so a failed
read reports the name as unregistered with the policy result instead of raising.
Also fixes three lint errors the line already has, two import orders and one
duplicate test import, since make check cannot pass on this line without them.
Backport of #50276 to lts/1.108.x. tests/integration/_support/redaction_probe.py also sends the standard key the picked integration test reads (main's probe got it in #47192).
Cherry-picked from 3f13c1ca34 (main). Conflicts in litellm_logging.py and the prompt caching check test were main-only neighbours (_get_completed_success_callbacks and the tools/system affinity tests), resolved by adding only this PR's method and test.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Backport of #50274 to lts/1.108.x. The line also lacked the get/delete/cancel responses CallTypes members that main got in #49133, so a during_call guardrail made those routes 500 with 'aget_responses is not a valid CallTypes'. The six members are added here (CallTypes, schema.d.ts, the azure call type list), and tests/integration/conftest.py registers the shares_worker_state marker the picked integration test carries.
Cherry-picked from d295f76c4a (main). Conflicts in litellm/types/utils.py, schema.d.ts and three test files were neighbouring main-only call types and tests, resolved by adding only the compact_responses entries and tests from this PR.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Backport of #50218 to lts/1.108.x.
Cherry-picked from f25fff42a6 (main). Conflict in tests/unit/llms/openai/test_openai.py resolved by adding only this PR's test and imports, the neighbouring agentic-loop tests from main are not on this line.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Backport of #50132 to lts/1.108.x. The line had no _prisma_error_code helper (main got it in #49190), so it is added here verbatim for db_write_never_sent.
Cherry-picked from b080c59e78 (main). Conflict in tests/unit/proxy/db/test_exception_handler.py resolved by keeping only the tests this PR adds, the two neighbouring tests from #49190 are not on this line.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Adapted for lts/1.108.x: the line has no _async_routable_cooled_deployment_ids,
so its call site and the _pool test helper change are dropped, and the test
file gains the RouterErrors import it lacks on this line.
(cherry picked from commit cfe0100e1f)
The limits test now runs on Haiku 5.5 with thinking, the opt-in row whose
schemas the picks clean, since Sonnet 4.6 keeps the line's schema as sent.
The open object test checks Converse forwards the caller's schema once with
no JSON tool and answers with Bedrock's verdict. The /v1/messages test compares
against a copy taken before the call, so an in-place edit fails it.
The integration tests that expected an open-object schema to move to the
JSON tool on a native model now expect it to stay native, as the line does.
A Bedrock /v1/messages test checks a row-native profile forwards the caller's
schema unchanged, and a Converse handler test checks a sync stream error keeps
its status class and the line's raw body.
Backport line fix, not a pick. Every change keeps a request that worked on
lts/1.108.x working after the picks in this branch.
Models the cost map marks native on every request (Sonnet 4.6, Haiku 4.5,
Sonnet 4.5, Mythos 5.1) send the caller's schema the way the line did, on
Converse, Invoke and direct Anthropic. #46870 moved some of their schemas to
the forced json tool. That turned array roots and typed allOf roots into 400s,
gave prose for open objects with thinking on, and broke tool_choice without
tools. Opt-in models (Haiku 5.5, Sonnet 5.5, Sonnet 5, Opus 4.8) keep the new
native path and its schema checks.
bedrock_mantle/ chat has no retry, so Mantle ignores thinking when it picks
the path, and opt-in models stay on the json tool there as they did before.
The sync Converse stream keeps the line's error handling, so its error classes
and router retries stay the same. The retry reads the raw HTTP error there
instead. It also sends the caller's headers as they were before the first
attempt, so the structured output beta does not leak into it.
main has the same routing gaps.
Backport of #47100 to lts/1.108.x.
Cherry-picked from 52f5a28679 (main).
Conflicts resolved for the line. The line has no per-attempt timeout split, so
main's _native_output_attempt_timeout and timeout_for_optional_params, and the
two tests that cover them, are left out. Both attempts use the call's resolved
timeout, which on a Router stream is stream_timeout, the same limit the line's
faked stream has today. The cost map JSON hunks are left out because the line
already matches them.
* fix(openai): send chat completions through client.post so the SDK's per-message param transform stops holding the event loop
create() walks every message, content part and tool of the request through
openai._utils._transform before posting, 91 ms at 300k tokens and ~300 ms at
1M tokens on the event loop, and returns the same dict for the plain dicts
litellm builds. client.post is what create() calls after that walk, same
request, headers, retries, raw response and stream objects, so the chat
helpers now call it directly through the helper embeddings already used
* fix(openai): type the chat client.post result as LegacyAPIResponse so the type gate sees .headers and .parse
* fix(openai): assert the chat client.post result is a LegacyAPIResponse instead of casting it
* test(openai): stub client.post with a LegacyAPIResponse spec in the empty-response and no-leak tests
* test(core-utils): stub client.post with a LegacyAPIResponse spec in the agentic loop leak test
* test: every client.post stub on the OpenAI chat path returns a LegacyAPIResponse-spec'd mock
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
(cherry picked from commit e8550c1454)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(caching): key cached async clients by a per-loop token instead of id(loop)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(caching): drop deferred closes queued on a collected event loop
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(caching): pin that a close queued on a live loop waits for that loop
* test(caching): run loop-reuse tests until a loop address repeats
The collected-loop tests ran a fixed 50 loops and passed with the fix reverted whenever no loop reused an earlier address. They now stop at the first reuse and skip if none happens, through a shared helper.
* test(caching): cover the cache key fallback for loops without weak references
Also corrects the evicted client closer docstring: a dropped bucket's clients are left to the garbage collector.
* fix(http): free short-lived event loops whose handler finalizer close finished as the loop stopped
* fix(caching): share sync clients across event loops and catch a failed finalizer close
* fix(agentops): stop closing the shared sync http client after the auth call
---------
Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
(cherry picked from commit 4ce724e3df)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Backport of #49785 to lts/1.108.x.
Cherry-picked from fcef9cd0a3 (main).
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Backport of #48278 to lts/1.108.x.
Cherry-picked from 0b66a27528 (main).
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Backport of #49964 to lts/1.108.x.
Cherry-picked from 62f85afacf (main). The cost-map schema keys and the schema test were re-applied by hand onto the line's test_utils.py, and the priority-markup test the line does not have was left out.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Backport of #49965 to lts/1.108.x.
Cherry-picked from 2d0c720425 (main). The line keeps its inline session page query, so that read and the grouped count go through _query_spend_logs directly and _SpendLogsReader is not ported.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Backport of #48086 to lts/1.108.x.
Cherry-picked from dc0237ccd7 (main).
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Backport of #49462 to lts/1.108.x.
Cherry-picked from 948445c828 (main). get_proxy_wide_deployment_credentials hand-ported onto the line's _resolve_matching_deployment_api_key, since the line has no _get_deployment_credentials.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Backport of #49301 to lts/1.108.x.
Cherry-picked from 298911d61b (main).
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Backport of #48640 to lts/1.108.x.
Cherry-picked from 59b8c73930 (main). Dropped the pick's edit to the ValidationError branch the line does not have.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Backport of #49922 to lts/1.108.x.
Cherry-picked from 391c8013c9 (main).
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Backport of #48089 to lts/1.108.x.
Cherry-picked from e6a32bbeeb (main).
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Backport of #48943 to lts/1.108.x.
Cherry-picked from 96fe78f96e (main).
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Backport of #48913 to lts/1.108.x.
Cherry-picked from 84c4c5aa5c (main).
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Backport of #48644 to lts/1.108.x.
Cherry-picked from 8f31c7171d (main). Dropped the pick's edit to a stream-stall test the line does not have.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
