Commit graph

56 commits

Author SHA1 Message Date
Mateo Wang
66c8f5b1a9
fix(llms): join api_base and route with one shared helper (#47535)
* fix(llms): sanitize trailing slashes in provider get_complete_url

* test(llms): move trailing-slash url tests into each provider's mapped unit file

* fix(llms): strip trailing slashes in Perplexity search and Ollama embedding URLs and run the Topaz tests in CI

* fix(llms): strip trailing slashes in search, DashScope, VolcEngine, GigaChat and Gradient AI URL builders

* fix(fastcrw): strip a trailing slash from api_base and expect one slash in the GigaChat tests

* fix(gigachat): strip a trailing slash from the pass-through api_base

* fix(llms): join api_base and route with one shared helper

A deployment api_base that ends in / made several get_complete_url builders emit a double slash, which Envoy and other gateways answer with 404. Add join_api_base in url_utils and use it in those builders, with a regression test that drives every get_complete_url under litellm/llms with a trailing-slash api_base

* refactor(llms): move per-provider api_base strips onto join_api_base

Use the shared join_api_base helper in the search, chat, embedding, image variation, GigaChat and VolcEngine URL builders instead of a per-provider rstrip and suffix check, so one helper owns the single-slash join

* ci: run the trailing-slash URL builder test in the llms shard

* test(integration): cover trailing-slash api_base across provider URL builders

Wire cells for chat, streaming, messages, responses, embeddings, search and pass-through with and without a trailing slash, an owned-proxy module for the flag-gated providers, env bases and a worker kill, two chaos bursts, and a Topaz SDK cell

* fix(llms): send every provider route through join_api_base

Every place under litellm/llms and litellm/main.py that joined a caller-settable api_base with a route by concatenation or f-string now goes through join_api_base, so an api_base with a trailing slash sends the same upstream path as one without it

Routes made only of a caller-supplied id, model, or passthrough path stay on a plain join over an already normalized base, because the helper's suffix dedup would turn a lookup by id into a list or create call

The sweep test now also reaches the factory-made JSON provider builders, and a new loopback test drives the public entrypoints with an api_base ending in /gw/ and checks that every request path keeps /gw/ with no double slash

* fix(llms): keep the route skip only where a builder already had it

join_api_base is a pure join again. The new ensure_api_route skips the route only when the URL path
already ends in it, and it is used only where the merge base or the trailing-slash test PR head
already skipped. Evals cancel, Predibase, and Azure AI threads append their route again, EdenAI
model listing and Gemini RAG ingestion join through join_api_base, and Azure AI Anthropic checks the
path so a host named anthropic-* builds /anthropic/v1/messages

* fix(azure_ai): read /anthropic from the URL path when building the count_tokens URL

* test(llms): run the Gemini RAG trailing-slash case through aingest

The sync ingest wrapper needs a current event loop, which an earlier async test in the same worker clears

* test(llms): cover the Azure realtime, Bedrock rerank, and oobabooga joins with a trailing-slash api_base

* test(integration): cover api_base joins across providers, query strings and named hosts

* feat(url_utils): add join_api_base and ensure_api_route helpers

* fix(url_utils): keep the scheme slashes when a stripped suffix was the whole host

join_api_base and ensure_api_route now keep the // of a base left with only a scheme,
so http://v1 with /v1 stripped joins to http://v1/files as before. The Azure v1 client
base_url goes through ensure_api_route so a base ending in /openai/v1 no longer doubles it

* fix(azure_ai): build the count_tokens URL from the messages URL

* fix(azure_ai): match anthropic as a whole path segment in the messages URL

* fix(llms): keep main's Azure AI anthropic match and non-string Bedrock endpoints

Azure AI chat and messages go back to cutting the base at the first
/anthropic, as on main, and count_tokens keeps its path check, so a
gateway path like /anthropic-proxy builds the same URLs as main. A base
that already ends in /v1/messages still gets count_tokens next to it.

A non-string aws_bedrock_runtime_endpoint (an httpx.URL VPC endpoint) is
used as its text again in AgentCore, Bedrock responses and the RAGFlow
base, instead of falling back to the default host.

get_volcengine_base_url and GigaChat's get_api_base strip a trailing
slash again for outside callers. The cache-hit wire test is skipped as a
known proxy bug, since one cache hit writes two cache_hit spend rows.

* fix(llms): drop a slash before the api_base query and tighten the route test checks

* test(llms): annotate new test locals as Final and flatten the builder generator

* fix(url_utils): keep a scheme-only api_base hostless so it fails before sending

---------

Co-authored-by: Raushankumar0720 <raushan.singh.cg@gmail.com>
2026-10-21 06:59:32 +00:00
Priyansh Nandwana
c6fc5b01bb
fix(bedrock): send batch jobs to a configured control-plane endpoint (#47260)
* fix(bedrock): send batch jobs to a configured control-plane endpoint

Bedrock batch create, retrieve and cancel always built the regional
bedrock.{region} host, so a VPC endpoint or compatible gateway could not
be used. Add aws_bedrock_control_plane_endpoint as a deployment setting
read by the signed HTTP path and the boto3 path, carry it through the
proxy credential filter, and refuse it in request bodies like the other
endpoint overrides. api_base stays out of it because on a Bedrock
deployment it points at the runtime host.

Fixes #45056

* fix(bedrock): read the control-plane endpoint from typed kwargs in job status

The named parameter added one unknown-argument error in litellm/batches/main.py,
which unpacks untyped kwargs into this function

* docs(bedrock): explain batch control-plane endpoint configuration

* fix(bedrock): keep the batch control-plane endpoint out of chat request bodies

A deployment that sets aws_bedrock_control_plane_endpoint also serves chat,
and the unregistered key was swept into the Bedrock request body, so every
chat call on it failed with "Extra inputs are not permitted". Register it
with the other Bedrock batch connection fields so it stays a LiteLLM-owned
kwarg on every route while batches still read it from litellm_params.

* test(bedrock): verify endpoint signatures without internal mocks

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-10-20 12:30:16 -07:00
devin-ai-integration[bot]
dd20bb0ae9
fix(bedrock): apply the 16 maxTokens floor to Kimi K3 on Converse (#43875)
* fix(bedrock): apply the 16 maxTokens floor to Kimi K3 on Converse

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(e2e): use lane AWS credentials for the Kimi K3 messages test

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(e2e): bypass the proxy cache and declare test metadata for the Kimi K3 probe

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-10-20 13:01:45 +00:00
Mateo Wang
270d108ace
fix(anthropic): json_schema envelope edge cases on the Anthropic json tool path (#47478)
* test(anthropic): cover json_schema envelope edge cases on the Anthropic json tool path

* fix(anthropic): unwrap required-only keys, walk past tool-named envelopes, keep non-ASCII text, flush cut streams, check Vertex output_format

* fix(anthropic): send held json tool text on cut Bedrock Invoke streams and drop a non-object output_format under drop_params

A Bedrock Invoke stream that ends before the json tool block closes now sends the held text and closing brace, as the direct Anthropic stream already did.

A non-object output_format on the Anthropic messages route and its providers is dropped under drop_params and otherwise forwarded, so the provider answers its own 400. Vertex Claude names the proxy setting in its 400 and reads drop_params through the shared helper.

Adds integration cases for the json_schema envelope and the cut stream.

* fix(anthropic): drop a non-object output_format on chat, unwrap a whole envelope on a cut stream, escape lone surrogates

Chat completions to Anthropic, Azure AI, Bedrock Invoke and Vertex now drop a non-object output_format under drop_params, matching /v1/messages. A stream cut after a complete held envelope is unwrapped like a normal end. A json_schema answer holding a lone surrogate keeps its \u escape so the response can still be encoded

* fix(anthropic): join a split answer when the stream ends in its last block

A cut during the second json tool block now runs the held block through
the same flush as a block stop, so it joins the streamed keys or the
earlier split blocks instead of sending two objects back to back. The
Vertex output_format fold drops its unreachable drop_params branch,
since both parent configs drop a non-object value first.

* test(vertex): expect the LiteLLM 400 for a non-object output_format on /v1/messages
2026-10-20 08:38:40 +00:00
joshua-berri
3124797399
fix(bedrock): resolve profile pricing before applying custom rates (#47419)
* fix(bedrock): price Converse calls using the invoked model profile

* fix(pricing): retain registered rates across metadata updates

* fix(pricing): honor temporary registered prices for Bedrock profiles

* fix(bedrock): record invocation model ID for each completion attempt

* test(bedrock): recalculate fallback cost after metadata updates

* fix(pricing): preserve cache-only registered price overrides

* fix(bedrock): preserve priced overrides without widening shared pricing

* fix(bedrock): retain explicit pricing through token cost lookup

* fix(bedrock): layer cache overrides over invoked profile rates

* fix(bedrock): keep effective prices consistent across billing

* fix(bedrock): share profile pricing and custom rate resolution

* fix(router): remove unused pricing suppression

* fix(cost): preserve rates across mixed token thresholds

* fix(router): share deployment pricing during startup

* fix(bedrock): preserve Router pricing inheritance for profiles

* fix(bedrock): retain configured pricing for unmapped profiles

---------

Co-authored-by: Joshua Valluru <326636767+joshua-berri@users.noreply.github.com>
2026-10-20 00:10:49 -07:00
Mateo Wang
52f5a28679
fix(bedrock): send native structured output first on Opus 4.8 and Sonnet 5 and fall back to the json tool when Bedrock rejects it (#47100)
* fix(bedrock): send native structured output first on Opus 4.8 and Sonnet 5 and fall back to the json tool when Bedrock rejects it

* fix(bedrock): retry with the json tool when Bedrock refuses the native output schema

Count a 400 under output_config.format.schema: as a native output rejection, so a schema Bedrock cannot compile (like a pattern with a nested {n,m} group) gets the json tool retry instead of reaching the caller

* fix(bedrock): make the native output retry send what an opted-out deployment sends

The retry after a native output rejection now reruns the same optional param
mapping completion() ran, with the deployment model_info opted out of
supports_bedrock_native_structured_output, so tool_choice, effort and the json
tool schema match an opted-out deployment instead of a hand-built tool. Each
config keeps only the predicate that decides when native output came from the
opt-in key.

Bedrock batch records for Anthropic models map with the opt-in turned off,
since a batch job has no retry. Models on supports_native_structured_output
keep native output in batch

* fix(bedrock): build the messages retry from the caller's request and keep other errors unchanged

* fix(bedrock): give sync converse stream errors the upstream body as text

* fix(bedrock): let a malformed caller outputConfig reach Bedrock instead of raising

A Converse json_schema request whose caller outputConfig failed litellm's
shape check raised a pydantic error before the request was sent, so the
caller got a 500 where the merge base passed Bedrock's 400 through. The
opt-in check now treats that outputConfig as carrying no native output.

Also opts the vertex wire test's pinned Opus 4.8 json tool deployment out of
native output, and extends the Bedrock native output integration tests.

* fix(bedrock): send a refused caller outputConfig once and give opt-out advice only when native output is unsupported

A caller can send its own outputConfig on Converse. When Bedrock refused that
schema on an opt-in deployment, the retry still carried the caller's textFormat,
so Bedrock got the same request twice and the warning told the operator to turn
off an opt-in that was not the cause. The retry is now skipped when the opted-out
request would still carry a native textFormat, so Bedrock's 400 reaches the
caller after one call as on the merge base. The opt-out advice is logged only for
the account-level "Extra inputs are not permitted" rejection.

* fix(bedrock): keep the caller's continuation message on the native output retry

The Converse transform pops user_continue_message from litellm_params on the first attempt, so a retry built from those params afterwards dropped it. Both Bedrock chat routes now hand the fallback a copy taken before the first attempt.

* test(bedrock): pin additional_drop_params on the /v1/messages native output retry

* fix(bedrock): give the Converse native output retry the fake stream timeout
2026-10-18 00:06:07 +00:00
Mateo Wang
16645cc963
fix(router)!: bound fake-streamed calls by the request timeout, not stream_timeout (#47162)
* fix(router): bound fake-streamed calls by the request timeout, not stream_timeout

A stream=True call that the provider serves as one non-streaming request (Bedrock Converse for json_schema on models without native structured output, tools on some Llama models, ai21) gets no byte until generation ends, so stream_timeout acted as a total-duration cap. The Router now passes timeout and stream_timeout separately to chat completion calls, and completion() applies stream_timeout only when it reads a live stream.

* test(timeout): cover stream_timeout on live streams only and the router's timeout split

* fix(router): cover every fake-stream site and keep the target group's stream_timeout under prompt management

* fix(router): never shorten a fake stream below stream_timeout, and split timeouts on every surface

A faked stream now takes the longer of timeout and stream_timeout, so a caller
who raised stream_timeout to cover a slow fake stream keeps that headroom.
Azure o-series dispatch is mirrored when deciding whether a stream is faked.
text_completion, /v1/messages and /v1/responses through the Router now get
timeout and stream_timeout separately, like chat calls.

* fix(router): report the effective stream timeout, split adapter streams, and cover M365 and Rust messages

* fix(router): report and log the timeout each stream was bounded by

Native /v1/messages and /v1/responses now record the timeout they applied, so
x-litellm-timeout and the cooldown guard read the real limit instead of the raw
request value. A prompt-managed model also passes on only the caller's or its
own timeout, so the target deployment's timeout still wins.

* fix(router): keep the caller's request_timeout on a prompt-managed handoff and report the applied stream limit

* fix(router): give streams answered in one piece the longer limit and accept text timeouts on responses

* fix(router): give /v1/messages and /v1/responses streams answered in one piece the longer limit

* test(integration): cover faked, live and converted stream timeout limits per endpoint

* test(router): regression tests for faked-stream timeouts
2026-10-17 22:08:02 +00:00
Mateo Wang
82ca10cc2d
feat(bedrock): apply bedrock_request_metadata_fields on the /bedrock pass-through routes (#47040)
* feat(bedrock): apply bedrock_request_metadata_fields on the /bedrock pass-through routes

* fix(bedrock): move Invoke body requestMetadata into the signed header and match Bedrock's ASCII whitespace rule

* fix(bedrock): keep Invoke body requestMetadata in the body and answer bad metadata as AWS does

Invoke requests whose body already carries requestMetadata keep the resolved pairs in the body
instead of the 8,500-character signed header. Malformed caller metadata now answers 400 with the
ValidationException error type the AWS SDK expects, and drop_params drops bad pairs on both the
pass-through routes and /v1/chat/completions. The caller-pair dedupe is linear, and the chat
validator treats \s as ASCII whitespace, as Bedrock does

* fix(bedrock): match Bedrock's 400s and header limit on pass-through request metadata

Invoke metadata too long for the 8,500-character header now goes in the body,
Invoke requestMetadata null answers 400 like Bedrock, Converse type errors carry
SerializationException, violation messages name the failing key, and the reserved
user_api_key_ prefix is matched case-insensitively after trimming whitespace

* fix(bedrock): trim the Invoke metadata header to Bedrock's limit and drop bad requestMetadata under drop_params without the setting

* fix(bedrock): leave /bedrock pass-through bodies as sent when request metadata forwarding is off

Global drop_params no longer rewrites requestMetadata on the pass-through routes unless
bedrock_request_metadata_fields is set. Chat keeps dropping bad pairs under drop_params.
Adds a tests/integration wire test for the four pass-through actions and removes an unused
parameter from the Invoke header helper.

* test(bedrock): drive the pass-through metadata wire test with boto3's bearer key env var

* fix(bedrock): trim the pass-through invoke metadata header and stop Nova invoke from checking metadata it drops

* test(bedrock): cover pass-through metadata header trimming, malformed caller metadata, and chat drop_params on the wire

* fix(bedrock): match requestMetadata keys and values against the whole string so a trailing newline cannot pass the 256-character limit
2026-10-17 14:01:50 +00:00
Mateo Wang
2c5c9b8d84
fix(bedrock): use native structured output on Haiku 5.5 and Sonnet 5.5, and fix Fable 5 and Sonnet 4.6 schema 400s (#46870)
* fix(bedrock): use native structured output on every Claude model whose Bedrock profiles accept it

Bedrock accepts outputConfig structured output on every reachable profile of
Claude Opus 4.7, 4.8, 5, 5.5, Sonnet 5, 5.5, Haiku 5.5, and Fable 5.1, so
json_schema now goes out natively there instead of as a forced tool, which kept
Bedrock from returning adaptive thinking. Fable 5 rejects outputConfig, so its
keys go back to the tool path. GovCloud keys keep their flags, and the drift
tests let GovCloud lag its commercial base but never lead it.

* fix(bedrock): clean Claude schemas for native structured output and keep recursive schemas on the old path

Bedrock native output rejects minimum/maximum, minItems, uniqueItems, oneOf,
additionalProperties true, and recursive or unresolved $ref, all of which the
forced tool accepted. Claude schemas on the Bedrock native paths now go through
the cleaner the Anthropic native path already uses, and a schema with a
recursive or unresolved $ref keeps the forced tool, inline, or drop path it had.

* fix(bedrock): keep older installs on the json tool with a Bedrock-only native output key

Older installs fetch the model map from main, so flipping supports_native_structured_output
moved them onto native output with their old schema handling. The new
supports_bedrock_native_structured_output key is ignored by older code. Schemas native output
cannot hold (recursive refs, open objects such as dict fields) keep the json tool on the new
key, models on the old key keep their current behavior for those shapes, and the Anthropic
API, Vertex AI, and Azure keep their current behavior.

* chore(cost-map): regenerate the model prices schema for the Bedrock native output key

* fix(bedrock): keep native output to profiles every account accepts and route schemas Bedrock rejects to the json tool

Only Opus 5.5, Sonnet 5.5, Haiku 5.5 and Fable 5.1 keep the Bedrock native output key, since
Opus 4.8 and Sonnet 5 answer 400 on some accounts and Opus 5 garbles escaped characters.
Lookaround patterns and string formats outside Bedrock's list now keep the json tool, objects
typed as a union with null are closed, and the cleaner keeps minItems of 0 or 1.

* fix(bedrock): keep Opus 5.5 and Fable 5.1 on the json tool, since native output stops their adaptive thinking

* fix(bedrock): keep shared $defs in native schemas and send refused schemas to the json tool on every native model

Native output now keeps $defs instead of inlining each use, so a submodel reused across a schema counts once against Bedrock's optional and union limits. Schemas native output cannot hold use the json tool on Converse and Invoke chat for both native keys, and minItems is only kept when it is an integer 0 or 1

* fix(bedrock): keep the json tool unless thinking is requested and bound native output schemas

The Bedrock native output key now applies only when the call asks for thinking, so calls without
thinking keep the json tool as before. Schemas past Bedrock's documented union and optional counts,
or past a grammar size estimate fitted to live probes, keep the json tool on both map keys. Open
objects keep the json tool on the Anthropic API, Vertex AI and Azure AI, where native output returned
them empty. The $ref check no longer expands $defs, so doubling definitions no longer stall the request.

* fix(bedrock): resolve root-path $defs refs so nested Pydantic models keep native output on Invoke

* test(bedrock): cover native structured output and json tool routing through the proxy

Integration cells drive the proxy against a scripted Bedrock peer that enforces Bedrock's structured output schema rules, covering thinking with native output on Converse, Converse streaming, Invoke, /v1/responses and /v1/messages, the deployment opt-in, Fable 5, schema cleaning on every native entry, open objects, and malformed schemas. Vertex, Anthropic and Azure AI cells cover open objects on native-capable models and the cleaner's minItems and nullable object handling. The unit table now covers reasoning_effort none and disabled thinking

* fix(bedrock): keep native output for numeric formats and wide old-key schemas, and keep thinking in json mode

Check string formats only on nodes that can be strings, apply the 64-field cap only to the Bedrock-only key, carry thinking blocks into the json mode message, and send malformed json tool schemas as object schemas.

* fix(anthropic): keep native output for dict fields unless the json tool can be forced

* fix(anthropic): keep native output for dict fields when the caller sends tools

* test(bedrock): close the dangerous-tool-use test schema, since Bedrock native output requires closed objects

* test(anthropic): cover forced and unforced open-object routing, json mode thinking, numeric formats and wide schemas

Add proxy cells for open-object schemas that keep native output when the json tool cannot be forced (a model that refuses forced tools on the Anthropic API and Vertex AI, thinking, caller tools), for the thinking block a json mode answer keeps on Vertex AI and Bedrock Invoke, and for integer and number formats and 65-field schemas on each Bedrock map key. Point the forced open-object cells at the Anthropic API and Azure AI only, since no Vertex AI model in the cost map takes both native output and a forced tool

* fix(bedrock): keep the native output opt-in on the deployment that sets it, and route effort none open objects to native output

* test(bedrock): cover deployment-scoped native output opt-in, register_model and effort none open objects
2026-10-16 20:57:23 -07:00
AxelRay
388251a5b8
fix(bedrock): drop dangerous-tool-use beta without safeguards (#44471)
* fix(bedrock): drop dangerous beta without safeguards

* fix(bedrock): drop the dangerous-tool-use beta only next to structured outputs, keep it on Mantle

Bedrock rejects the pair with 'invalid beta flag' on Claude models before Opus 4.7, while requests without structured outputs and Bedrock Mantle accept the beta, so the drop now only applies to Invoke requests with output_config.format or a strict tool and no safeguards.

* fix(anthropic): keep the AnthropicThinkingParam import that main now uses

* fix(bedrock): drop the client's dangerous-tool-use beta whenever safeguards is absent

Bedrock rejects the beta with 'invalid beta flag' on Sonnet 4.5, Opus 4.5, and Opus 4.6 even without structured outputs, so dropping it only next to structured outputs left those beta-only requests failing. The beta does nothing without safeguards, so Invoke now drops it in every such request. Mantle keeps it. The thinking display helper goes back to main's version.

* fix(bedrock): drop the client's dangerous-tool-use beta on Invoke chat without safeguards

* test(bedrock): wire cells for the dangerous-tool-use beta on Invoke messages, chat and responses

* test(bedrock): type the new dangerous-tool-use beta tests

* test(bedrock_mantle): type the dangerous-tool-use beta Mantle test

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-10-16 21:00:37 +00:00
Mateo Wang
7ddc9f1c96
fix(bedrock)!: keep reasoning, thinking blocks and the json_mode finish_reason on fake streams (#46863)
* fix(bedrock): keep reasoning, thinking blocks and the json_mode finish_reason on fake streams

A Bedrock Converse model without native structured output fakes the stream
when response_format carries a json_schema. The fake stream transformed the
response without json_mode and rebuilt a chunk from text, tool_use and token
counts only, so it dropped reasoning_content, the signed thinking_blocks and
the reasoning token count, and ended on finish_reason tool_calls.

The three fake-stream sites now pass json_mode to _transform_response and
stream the resulting ModelResponse through the shared
convert_model_response_to_streaming converter, which now numbers each
streamed thinking block with its index the way real streams do

* fix(bedrock): parse text tool calls on Converse fake streams like the non-stream answer

* fix(responses): stream the answer text when one chat chunk carries reasoning and the answer

* fix(streaming): keep every field on the OpenAI and openai_like fake streams and split multi-block thinking

Azure o-series and openai_like fake streams now use the shared MockResponseIterator, so extra tool calls,
reasoning and cached and reasoning tokens survive; the Bedrock-local iterator is deleted.

The /v1/responses bridge also splits a chunk whose thinking block is signed or redacted with no reasoning
text, and the sync iterator closes the reasoning item before the answer opens, like the async one.

The /v1/messages bridge gives each thinking block in a chunk its own content block, starts a new block when
the thinking block index changes, and streams redacted thinking as redacted_thinking.

Snowflake replays drop the stream index from thinking blocks.

* fix(anthropic): stream each thinking block's text once on /v1/messages

An unsigned thinking chunk now opens its content block with empty text, since its thinking_delta carries the text. A multi-block chunk keeps each piece's thinking block, so an unsigned block followed by a signed one keeps its own text and index.

* docs(anthropic): describe which thinking block start carries its text

* fix(streaming): close merged reasoning before the answer when one chunk carries both

* test(streaming): cover fake-stream reasoning across the proxy bridges and type the Snowflake replay test
2026-10-16 17:27:57 +00:00
moyai-devin-berriai[bot]
a3251a3bfd
fix(responses): preserve provider request IDs in streaming failure logs (#45453)
* fix(responses): preserve provider request IDs in streaming failure logs

* fix(bedrock): log the provider request id for in-stream errors on bridged Bedrock streams

Bedrock models that run through the chat-completions bridge (Claude,
converse, invoke) raise in-stream failures from the event-stream decoder
as a BedrockError without headers, so their failure logs kept
error_provider_request_id null. The decoder now passes the stream's
request-id headers to those errors.

Both the native Responses iterator and the decoder pass only the
PROVIDER_REQUEST_ID_HEADERS subset, so error responses and failure logs
pick up the request id and none of the 200 response's other headers.
MidStreamFallbackError gets its response at construction instead of
mutating it afterwards.

---------

Co-authored-by: moyai-devin-berriai[bot] <336287033+moyai-devin-berriai[bot]@users.noreply.github.com>
Co-authored-by: yucheng-berri <293926326+yucheng-berri@users.noreply.github.com>
2026-10-16 00:29:11 -07:00
ryan-crabbe-berri
f470e418ea
fix(anthropic): keep signed empty thinking blocks and honor thinking.block_binding (#46342)
* fix(anthropic): treat adaptive thinking as enabled when mapping response_format

is_thinking_enabled only recognised thinking.type=enabled, so a request that
paired thinking.type=adaptive with response_format took the synthetic JSON
tool path with a forced tool_choice, which Anthropic rejects alongside
thinking. The same check also gates the Bedrock Converse and Databricks
forced tool_choice downgrades, so they now see adaptive thinking too.

* fix(anthropic): keep signed thinking blocks with empty text on chat completions

Models that default to display: omitted return signed thinking blocks whose
text is empty, and Anthropic rejects the next turn when they are not replayed
unchanged. is_unsignable_thinking_block dropped them along with unsigned
blocks, and the inline content path required non-empty text, so a tool-use
turn on Opus 4.7+ or the 5 families lost its thinking blocks and 400ed on the
tool-result turn. Only a missing signature now drops a block, and the
mid-conversation system placement mirrors that rule. The /v1/messages strip
helper is left to #41362.

* fix(anthropic): skip the thinking-strip retry when block_binding asks for an error

The /v1/messages retry loop strips thinking blocks and resends on any
invalid-thinking 400, including the binding mismatch a caller opted into
with thinking.block_binding.prefix_mismatch_behavior: error. The retry
decision now sees the request body and surfaces that 400 instead of
overriding the caller's choice. #42888 covers the litellm_params side.

* fix(bedrock): carry thinking.block_binding through the Invoke thinking rebuilds

Bedrock Invoke rebuilt the thinking param from scratch when display: updates
was requested, on both the /v1/messages and chat completions paths, and the
legacy enabled-to-adaptive translation plus the clear_thinking injection did
the same, so thinking.block_binding never reached Bedrock. Every rebuild now
spreads the caller's thinking and only overrides the keys it owns, and
AnthropicThinkingParam types block_binding. The thinking-binding-controls
beta was already on the Bedrock allowlist.

* fix(anthropic): keep forcing the response_format tool under adaptive thinking

This reverts ad9e861ae7. Anthropic, Bedrock Invoke and Vertex all accept a
forced tool_choice next to thinking.type=adaptive and only reject it with
type=enabled, so counting adaptive as enabled fixed no live failure. It did
drop the forced JSON tool, so Claude answered response_format requests in
prose, and it downgraded a required tool_choice to auto on Bedrock Converse

* test(e2e): drop the adaptive thinking tool round trip case

With adaptive thinking the model often skips thinking on the weather prompt, so
turn 1 carries no signed empty block and the case fails before the replay it is
meant to cover. Anthropic also answers the replay with a 200 whether or not the
block is forwarded, so the case could not tell the fix from main. The factory
unit tests pin the outgoing shape instead

* fix(anthropic): only skip the thinking-strip retry for a binding mismatch 400

prefix_mismatch_behavior: error asks Anthropic to reject a block bound to
another conversation. It says nothing about unrelated bad blocks such as a
null signature, so those still get the strip-and-retry self-heal. Also trims
the empty-thinking docstrings to what the code guarantees and types the new
test parameters

* refactor(anthropic): narrow thinking fields with the mapping validator instead of typing.cast

typing.cast is banned by the strict ruff gate (TID251), so the three new casts pushed that rule one over its merge-base count. The existing pydantic mapping validator gives the same typed view without an unchecked cast, and it now has a generic name since it is no longer only used for Claude Code payloads

* refactor(anthropic): read thinking settings through the validated mapping so basedpyright stays at its merge-base count

The Bedrock clear_thinking rewrite still narrowed with a cast and an untyped dict fallback, which added two reportUnknownVariableType errors. It now uses the same pydantic mapping validator, made public for that reuse, and the retry hook validates the request body itself so the handler passes a typed dict instead of an untyped one

---------

Co-authored-by: yuneng-jiang <yuneng@berri.ai>
2026-10-15 18:49:53 +00:00
Kerry Lu
0d741a9816
fix: index streamed thinking_blocks so the OpenAI SDK .stream() accumulator accepts them (#46611)
* fix: index streamed thinking_blocks so the OpenAI SDK .stream() accumulator accepts them

* fix: satisfy the type discipline and basedpyright gates for the thinking block index

* chore: sync the OpenAPI snapshot and schema.d.ts with the thinking block index field

---------

Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-14 22:21:45 -07:00
Mateo Wang
c0663d5202
fix(anthropic)!: translate the OpenAI tool_search tool on /v1/chat/completions for Claude models (#46389)
* fix(responses): translate OpenAI tool_search to Anthropic tool search for Claude models

* fix(responses): define the tool_search_call output item locally for older openai SDKs

* fix(responses): gate tool search on the Anthropic request builder and validate deferred tool fields

Bedrock Converse builds its own request and has no tool search, so the bridge now drops the tool_search tool there with a warning instead of sending a tool the transport loses, while Bedrock invoke, Anthropic, and Vertex keep it. A replayed tool_search_call with no stored result is dropped from the Anthropic request so a previous_response_id follow-up no longer answers 400. Only srvtoolu_ ids whose name is one of Anthropic's search tools count as search calls, so a caller function named like one stays a function call. A non-boolean defer_loading or a non-list allowed_callers now answers 400 instead of 500

* fix(responses): keep an answered search call on replay and send tools undeferred when no search tool goes out

A replayed Claude tool search call that the caller answered with a tool message kept its
tool_result in the user turn while its tool_use was dropped from the assistant turn, so
Anthropic answered 400 on the OpenAI-SDK tool loop. The search call is now dropped only when
no tool message answers it.

The Responses bridge now strips defer_loading from every tool whenever no tool search tool is
sent, whether the caller sent none or the route cannot run it, since a deferred tool with no
search tool can never be loaded. Namespace members had started carrying the flag to every
bridge provider, which providers that reject unknown tool keys answered 422.

The audit round-1 integration cells for the bridge wire and chaos paths ride in this commit.

* fix(responses): gate only the OpenAI tool_search shape and keep native Anthropic search tools intact

A native tool_search_tool_regex_20251119 or tool_search_tool_bm25_20251119 tool the caller sends through /v1/responses passes through the bridge on every route as before, and the other tools keep defer_loading whenever any Anthropic search tool goes out. The route gate now drops the OpenAI tool_search item before translation instead of filtering the translated regex tool, so bm25 counts as a search tool. answered_tool_call_ids walks the history with islice instead of a slice copy. The tool_search_call output item drops its created_by field, which OpenAI rejects on a replayed input item and the streaming item never carried

* fix(anthropic): translate the OpenAI tool_search tool on /v1/chat/completions for Claude models

* fix(responses): drop malformed tool flags under drop_params and keep stored search calls off non-Anthropic follow-ups

With drop_params on, a non-boolean defer_loading or a non-list allowed_callers on a Responses API tool (or a namespace member) is dropped before the bridge instead of answering 400, so only a request without drop_params gets the 400. A custom tool now carries defer_loading to the chat tool like a function tool does

A previous_response_id follow-up routed to a provider that does not build an Anthropic request no longer replays the stored Claude search call (srvtoolu_ id) as a client tool call. Mistral answered that history 400 "Unexpected tool call id". An assistant turn that held only the search call is dropped there. The Anthropic route keeps the call, since its factory pairs it with the stored result

The tool_search_call output model is renamed OutputToolSearchCall so the proxy's OpenAPI schema keeps the OpenAI SDK's ResponseToolSearchCall name. answered_tool_call_ids reads the history by index, so the lookup is linear in the history length. The streaming done event reads the response it is given, which is the same object it used to read from the iterator

* fix(bedrock): answer 400 for the OpenAI tool_search tool on Converse unless drop_params is set

* fix(responses): validate function allowed_callers, send one search tool, and keep replays clean off Anthropic

A function tool whose allowed_callers is not a list of strings now answers 400
instead of 500. The guardrail scan reads tools without malformed flags, so the
route decides between a 400 and a drop under drop_params. When the caller sends
a native Anthropic search tool beside tool_search, only the native one goes out.
Databricks Claude no longer counts as an Anthropic request route.

A previous_response_id follow-up off Anthropic also drops a search call the
client replayed as a function_call with a srvtoolu_ id, and its output. The
answered tool id scan in the Anthropic message factory runs once per assistant
run instead of once per message. A null created_by on tool_search_call and
tool_search_output output items is left out of the serialized response, so a
cache hit returns the same keys as the first call.

* fix(anthropic): send one search tool and judge tool search on the real model for chat tool_search

Several OpenAI tool_search items now collapse into one tool_search_tool_regex
tool, since Anthropic answers 400 when two tools share a name. A native
Anthropic search tool the caller already sent wins over the OpenAI item.

Bedrock invoke and Vertex AI now decide tool search support on the real model
before tool-based response_format swaps in a stub model name, so a Claude 5.5
request with response_format no longer gets a 400 on a route that runs tool
search.

* fix(anthropic): name the proxy drop_params setting in the tool_search 400

* fix(guardrails): keep OpenAI tool_search out of the Responses guardrail tool scan

The scan translated tool_search into Anthropic's regex search tool on every
route, so a guardrail returning only function tools removed tool_search from
native OpenAI requests and OpenAI answered 400 for the deferred tools

* fix(anthropic): run tool search on azure_ai, gate it on Databricks Claude, and strip deferral without recursion

Anthropic's Foundry docs list no tool search exclusion for either hosting
option, so the claude-tool-search rule now fills azure_ai. Databricks Claude
models route their tools through the same unsupported-tool-search gate, so a
tool_search tool answers 400 or drops under drop_params instead of a 500.
_without_deferral strips two fixed levels instead of walking recursively.

* fix(anthropic): drop answered tool search replays beside client calls, place system before dropped search turns, flag tool search on azure_ai

* fix(anthropic): keep answered tool search replays paired with their answers

* fix(anthropic): drop Claude tool search calls from history sent to other routes, name the fix in the 400

A second turn that carries Claude's srvtoolu_ search call and its result now has both removed
before completion() sends it to a route that does not build an Anthropic request, including the
responses bridge. The tool search 400 now names model_info supports_tool_search for a model the
cost map does not know, and bedrock/invoke for a Converse model that runs tool search there.

* fix(anthropic): place mid-conversation system messages in one pass and scan tool_search_call in output guardrails

* fix(chat): keep answered Claude tool search calls in history sent to other providers

Only a srvtoolu_ search call with no tool result is dropped before a
non-Anthropic route, so an answered pair keeps the caller's shape where
OpenAI and Gemini take it. The history check no longer copies the list
through a TypeAdapter.

* fix(guardrails): send empty incomplete tool_search_call arguments when a rewrite is not a JSON object

* fix(chat): drop unanswered Claude tool search calls only on routes that reject them

OpenAI, Azure OpenAI and Bedrock Converse answer 400 for a srvtoolu_ call
with no tool result, while Gemini answers 200. Keep the history as sent
everywhere else so routes that already work see what the caller sent.

* fix(responses): keep the caller's defer_loading on Claude routes when no server tool_search is sent

* test(integration): cover chat tool_search routing, history drop and load on the wire

Audit round 1 of 3 at 9150723c84: 58 cells across the Claude routes, Converse, Databricks, the drop_params sources, replayed history on openai, azure, Converse and gemini, and a burst, an outage and a worker kill

* fix(guardrails): deliver streamed rewrites when OpenAI's tool_search_call has no call_id

* fix(responses): answer 400 to a tool_search execution that is not server or client

OpenAI's own Responses API answers 400 invalid_value to it. With drop_params the malformed
execution is dropped like the other malformed tool flags, which leaves a server search

* test(responses): cover the 400 and drop_params paths for a malformed tool_search execution

* test(responses): cover a Claude search call replayed to Bedrock Converse with cache marks

* fix(anthropic): map only a server or unset tool_search execution to Anthropic tool search, so a malformed value answers 400

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(anthropic): judge the chat tool_search gate with a strict server check and keep the Responses bridge classification

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* style(responses): sort the tool_search __all__

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-15 04:47:05 +00:00
ryan-crabbe-berri
6c78089d34
fix(bedrock): use base_model for Claude capability checks on application inference profile ARNs (#46569) 2026-10-15 03:02:57 +00:00
ryan-crabbe-berri
e889d9d653
fix(bedrock)!: replay redacted_thinking on Converse and keep its data as Claude's own string (#46532)
* fix(bedrock): send redacted_thinking back to Converse as reasoningContent.redactedContent

Replaying an assistant turn that carried a redacted_thinking block to Bedrock
Converse turned it into an empty unsigned reasoningText, which the assistant
content builder then dropped as blank text. Bedrock returned 200 so callers
never noticed. Map redacted_thinking to redactedContent in both the
thinking_blocks and inline content paths of the sync and async builders

* fix(bedrock): drop blank redacted_thinking data and keep non-str signatures signed on Converse replay

* fix(bedrock): replay redacted_thinking to Converse only when the target is Claude

Nova, Llama, DeepSeek and gpt-oss reject reasoningContent.redactedContent with a 400, so a Claude
history replayed to them through a Router fallback or model switch keeps dropping the redacted block
the way main does. Claude is resolved from the Bedrock base model of the target, and an application
inference profile ARN counts as Claude when its deployment's base_model names a Claude model.

Adds integration contracts for the replayed Converse request shape, stream and non-stream, with a
tool call, and for non-Claude and ARN targets.

* test(e2e): cover Bedrock Converse replaying redacted_thinking live through the proxy

Turn 1 triggers redacted thinking on Claude Sonnet 4.6, turn 2 replays it, and a live
edge between the proxy and Bedrock checks the replayed Converse body carries the same
reasoningContent.redactedContent. Covers nonstream, stream and a tool call round trip

* fix(bedrock): validate litellm_params before reading the deployment base_model

litellm_params arrives as an untyped dict, so passing it straight into the typed
helper grew reportUnknownArgumentType. Validating it into Mapping[str, object] first
keeps the read typed without growing the type-check gate

* fix(bedrock): wrap non-Converse redacted_thinking data and let a named model id win over base_model

* test(bedrock): cover redacted_thinking minted outside Converse, model_id and the sync Router base_model path

* fix(bedrock): match bare claude- base_model names without reading the untyped cost map

* fix(bedrock): keep redacted_thinking data as Claude's own string on every route

Converse hands redactedContent back base64-wrapped, and that wrapper used to leak out as data,
so a Converse turn replayed to Invoke or the Anthropic API was rejected. Converse replies now
decode it, Converse requests wrap it exactly once, and the Anthropic-shaped builders unwrap
histories saved with the old wrapping. Responses API encrypted reasoning is no longer sent to
Claude on Converse, which rejected it with a 400
2026-10-14 19:08:14 -07:00
devin-ai-integration[bot]
47e4ed88c2
fix(anthropic): keep refusal stop_reason and stop_details on the logged response (#46104)
* fix(anthropic): keep refusal stop_reason and stop_details on the logged response

* fix(anthropic): keep the refusal stop-fields boundary tolerant of a non-dict upstream body

* test(anthropic): use Anthropic's documented refusal example as the unit-test fixture

* fix(streaming): keep provider fields on async stream cache hits and received_model_id on synthesized finish chunks

* fix(caching): replay cached completion_tokens_details on stream cache hits and skip non-object cached provider fields

* test(integration): cover Anthropic-family refusal stop_details on the wire, the cache and under chaos

* fix(caching): skip a cached completion_tokens_details that fails validation on stream replay

* fix(anthropic): keep refusal stop fields on a JSON-mode non-stream response

Also adds the round 3 audit cells for JSON-mode refusals, the proxy response serialization, and wrong-typed cached usage details on both stream replays

* test(anthropic): expect content_filter on streamed JSON tool refusals after main's stream change

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-10-14 14:42:49 -07:00
Kerry Lu
83980c9d63
fix(router): read the first stream chunk before returning so a failure before any token runs through the retry and fallback budget (#46493)
* fix(router): read the first stream chunk before returning so a failure before any token runs through the retry and fallback budget

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(router): keep only the raising call inside pytest.raises for the pre-content stream tests

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(router): yield the primed first chunk from a helper generator so the stream iterators keep their complexity

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(bedrock): expect the empty invoke stream to be reported from router.completion now that the first chunk is read there

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test: give router stream mocks a first chunk or an ending so the eager first read neither hangs nor retries

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(router): name the first-chunk helpers after what they do (_read_first_chunk, _FirstChunk, _chunks_with_first)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(router): drop the string annotation on _FirstChunk.chunk, ModelResponseStream is imported at runtime

---------

Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-14 14:19:25 -07:00
Mateo Wang
457b5d26c7
fix(anthropic): stream json_schema answers as content on Bedrock Mantle and Claude Platform, and join split answers (#46331)
* fix(vertex_ai): unwrap tool-emulated json_schema envelopes and send output_config.format for Claude

Vertex AI Claude deployments that emulate response_format through a forced
tool sometimes nest the schema object under an extra key such as
`parameters` or `input`, and that envelope reached message.content with a
200. The converter now unwraps exactly one nested object that matches the
schema, leaving a payload that is itself schema-shaped alone.

Native structured-output rows sent the deprecated top-level `output_format`,
which Vertex rejects with a 400. The Vertex request now carries it as
`output_config.format`, with an explicit caller value winning.

* fix(anthropic): unwrap tool-emulated json_schema envelopes on streams and tool-name wrappers

* test(anthropic): expect the json tool description through a shared constant

* fix(anthropic): drop the json tool description that makes sonnet-5 omit required keys

* fix(anthropic): keep schema-accepted answers whole and decide held streams at the second key

An answer the json_schema accepts is returned as is before any envelope rule runs, the tool-name
wrapper is unwrapped only when the schema rejects it, a malformed schema turns the unwrap off, a
streamed answer whose first key is a schema property named name or tool_name waits for its second
key before streaming, the held fragments go through a StringIO, and a prefix that stays undecided
past 4 KB is held to the end

* test(anthropic): keep the stream family cells to providers whose stream iterator gets the schema

* fix(vertex_ai): unwrap claude json_schema answers nested under up to three wrapper keys

A tool-emulated json_schema answer was unwrapped only when the real object sat one key down. In live QA, 1 of 58 answers (/v1/responses, claude-sonnet-5) came back as {"parameter": {"value": {...}}} and reached the client still wrapped

The unwrap now follows single-key wrappers up to three levels down and stops at the first object that has a key the schema declares. That object is returned only when it has every required key. A payload wrapped deeper than three levels, a level with more than one key, and a level whose key is one of the schema's own property names all come back exactly as they arrived

The streamed path needs no change because the buffered answer goes through the same unwrap when the tool block closes, and a test now pins that both paths agree

* fix(bedrock): unwrap streamed claude json_schema answers on claude platform and mantle

A streamed json_schema answer on bedrock/claude_platform and Bedrock Mantle Claude reached the client as tool_calls deltas holding the json_tool_call wrapper, with finish_reason tool_calls and no content. The shared HTTP handler built the async stream iterator without json_mode, and neither stream path passed the response_format schema

The handler now asks the provider config for the iterator through get_model_response_iterator_for_request, which carries json_mode and the request's optional params on both the sync and async paths. Claude Platform and Mantle override it to hand the json tool schema to the Anthropic stream iterator, so the streamed answer is the bare schema object with finish_reason stop, the same as the non-streamed one

Databricks is the only other provider on this handler whose iterator reads json_mode, so its async Claude json mode stream now returns the answer as content like its sync stream already did

The stream family integration test covers claude_platform_unmapped and bedrock_mantle again

* fix(anthropic): unwrap json tool envelopes by structure instead of schema validation

The unwrap now reads only the schema's top-level property names. A lone
undeclared key holding an object, alone or beside a name equal to the json
tool's own name, is the envelope, and the unwrap repeats while the inner
object shares no key with the schema. jsonschema and referencing are no
longer imported on the request path. The stream holds only an answer that
opens with the tool's name or with an undeclared key holding an object, and a
schema without top-level properties streams as it arrives.

* fix(llms): keep other providers' async streams as they were

The request-aware iterator hook passed json_mode on async streams to every provider on the shared HTTP handler, which changed Databricks and any other provider whose iterator reads it. The default hook now makes the same two calls the handler made before, so only Claude Platform and Bedrock Mantle change, and the Databricks test that pinned the wider change is removed

* fix(anthropic): walk json tool envelopes iteratively

* test(llms): pin which stream iterators get json_mode from the shared handler

A provider config that does not override the request-aware iterator hook still gets json_mode on a sync stream and still does not get it on an async stream, the same as before the hook existed

* fix(anthropic): keep json tool arguments too deep to decode instead of raising

* fix(anthropic): unwrap json tool envelopes beside undeclared string labels

* fix(anthropic): keep json answers whole on schemas that allow extra keys and stream held text as sent

A schema that allows undeclared keys and requires none now keeps an undeclared wrapper key on both the stream and non-stream paths. When no envelope is found, the held stream text is now sent exactly as the model wrote it. On Vertex /v1/messages, output_format now wins over output_config.format when both are set, which matches the Anthropic adapter.

* fix(anthropic): join a json_schema answer the model splits into one tool block per field

With tool emulation the model sometimes answers with one json_tool_call block per schema field, each naming the field and holding its value. The non-stream path returned only the first block and the stream sent every block's text back to back, so the client got one field or invalid JSON. Both paths now build one schema object from those blocks, and any other set of blocks keeps today's output

* refactor(anthropic): set the streamed finish reason in one expression

* test(anthropic): cover label envelopes, dynamic-key schemas, as-written held streams and output_format precedence

* fix(anthropic): unwrap a streamed values wrapper like the non-stream answer

* fix(anthropic): join streamed split answers on schemas without required keys, keep client tool calls and length on json_schema streams

* fix(anthropic): join streamed split answers that carry text labels on schemas without required keys

* test(bedrock): script the wrapped json_schema stream with the wrapper Bedrock Mantle sends

* test(bedrock): script the wrapped json_schema stream with the wrapper Bedrock Mantle sent raw

* fix(anthropic): join a split json_schema answer whose fields all hold the same string

* fix(anthropic): join a split json_schema answer sent as one partial object per field

* fix(anthropic): join field blocks after an empty streamed answer without a leading comma
2026-10-14 11:32:59 -07:00
devin-ai-integration[bot]
2f3ccbe7c7
feat(decisions)!: send images to Cloudflare Clef through its images field (#45734)
* feat(decisions): send images to Cloudflare Clef

OpenAI-format input_image parts and the System One images field now reach Cloudflare clef and clef-flash in its images field, with remote URLs downloaded and inlined first. Providers that cannot take images drop them under drop_params and answer 400 otherwise.

* fix(decisions): follow drop_params for images and use the call timeout for image downloads

* fix(decisions): drop a malformed images entry on its own under drop_params

* fix(decisions): refuse images next to input whatever drop_params says

* fix(decisions): bound clef image fetches, honor drop_params for images, and log caller URLs

Refuse more than 4 images for Cloudflare clef before any download, keep the
caller's timeout as one deadline across image fetch retries, accept images
next to input under drop_params, validate image strings and objects, and pass
the caller's image URLs (not inlined bytes) to logging callbacks.

* fix(decisions): check input_image URLs and clef image sizes

* fix(decisions): accept data URL media type parameters and check clef image types

* fix(decisions): count clef images after dropping unsupported types and flag clef vision

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-10-14 12:14:48 +00:00
devin-ai-integration[bot]
c581c85559
fix(vertex_ai): unwrap tool-emulated json_schema envelopes and send output_config.format for Claude (#45871)
* fix(vertex_ai): unwrap tool-emulated json_schema envelopes and send output_config.format for Claude

Vertex AI Claude deployments that emulate response_format through a forced
tool sometimes nest the schema object under an extra key such as
`parameters` or `input`, and that envelope reached message.content with a
200. The converter now unwraps exactly one nested object that matches the
schema, leaving a payload that is itself schema-shaped alone.

Native structured-output rows sent the deprecated top-level `output_format`,
which Vertex rejects with a 400. The Vertex request now carries it as
`output_config.format`, with an explicit caller value winning.

* fix(anthropic): unwrap tool-emulated json_schema envelopes on streams and tool-name wrappers

* test(anthropic): expect the json tool description through a shared constant

* fix(anthropic): drop the json tool description that makes sonnet-5 omit required keys

* fix(anthropic): keep schema-accepted answers whole and decide held streams at the second key

An answer the json_schema accepts is returned as is before any envelope rule runs, the tool-name
wrapper is unwrapped only when the schema rejects it, a malformed schema turns the unwrap off, a
streamed answer whose first key is a schema property named name or tool_name waits for its second
key before streaming, the held fragments go through a StringIO, and a prefix that stays undecided
past 4 KB is held to the end

* test(anthropic): keep the stream family cells to providers whose stream iterator gets the schema

* fix(anthropic): unwrap json tool envelopes by structure instead of schema validation

The unwrap now reads only the schema's top-level property names. A lone
undeclared key holding an object, alone or beside a name equal to the json
tool's own name, is the envelope, and the unwrap repeats while the inner
object shares no key with the schema. jsonschema and referencing are no
longer imported on the request path. The stream holds only an answer that
opens with the tool's name or with an undeclared key holding an object, and a
schema without top-level properties streams as it arrives.

* fix(anthropic): walk json tool envelopes iteratively

* fix(anthropic): keep json tool arguments too deep to decode instead of raising

* fix(anthropic): unwrap json tool envelopes beside undeclared string labels

* fix(anthropic): keep json answers whole on schemas that allow extra keys and stream held text as sent

A schema that allows undeclared keys and requires none now keeps an undeclared wrapper key on both the stream and non-stream paths. When no envelope is found, the held stream text is now sent exactly as the model wrote it. On Vertex /v1/messages, output_format now wins over output_config.format when both are set, which matches the Anthropic adapter.

* test(anthropic): cover label envelopes, dynamic-key schemas, as-written held streams and output_format precedence

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-10-14 00:12:19 +00:00
Mateo Wang
0ebec123fd
chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388)
* chore(typing): validate provider payloads at the boundary to clear 464 basedpyright Any errors

Replaces untyped reads of decoded JSON in 48 provider, guardrail and proxy
modules with one pydantic validation at the point the payload enters the
module, so every value read afterwards carries a real type.

reportAny 6111 -> 5663 and reportExplicitAny 1416 -> 1400. No basedpyright,
ruff-strict or type-discipline rule is higher repo-wide or in any single
file, and no cast, ignore or suppression marker was added.

* chore(typing): replace 47 Any annotations that every caller already satisfies with object

Each annotation here is a parameter or return type whose value is only
stored, compared, logged or forwarded, so object describes it exactly and
no function body changes. A site was kept only if no basedpyright,
ruff-strict or type-discipline rule rose in any file that references it.

reportExplicitAny 1400 -> 1353 and reportAny 5663 -> 5655.

* chore(typing): type upstream responses and loose annotations across providers, proxy and integrations

Second pass of the Any reduction. Decoded upstream JSON is validated once at
the provider boundary and read through typed views afterwards, annotations
that every caller already satisfies move from Any to a concrete type, and the
nine search transformations validate each result item on its own so error
text no longer carries the item index.

No cast, no type: ignore, no suppression marker and no budget file edit

* ci(unit): run the brave, google_pse, nlp_cloud and opensandbox tests in the providers shard
2026-10-13 15:35:23 -07:00
devin-ai-integration[bot]
249add6f0d
refactor(bedrock): rename is_claude_4_5_on_bedrock to supports_bedrock_prompt_cache_ttl (#46058)
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-12 18:34:33 +00:00
devin-ai-integration[bot]
e0398455d0
test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759)
* test: migrate selected llm translation contracts

* test: migrate anthropic completion translation tests

* test: migrate bedrock and cohere translation tests

* test: tighten bedrock translation migration

* test: migrate azure and openai model translation tests

* test: migrate openai and small-provider translation tests

* test: migrate gemini and vertex translation tests

* test: migrate optional params and rerank translation tests

* test: migrate huggingface, json and openrouter translation tests

* test: fix openrouter integration expectations

* test: migrate databricks, xai, cometapi and fireworks translation tests

* test: migrate ovhcloud, prompt and small provider translation tests

* test: split reasoning effort grid into unit and e2e

* test: migrate router and realtime guardrail translation tests

* test: migrate google interactions translation tests

* test: migrate a2a, evals, skills and hosted vllm translation tests

* test: remove remaining llm translation suite

* test: tighten migrated llm translation unit tests and land missing rows

Restores rows the first pass dropped or weakened (bedrock agentcore and openwebui thinking stream, anthropic prefix, citations and tool_choice, azure agents handler, huggingface bare api_base, optional params num_retries, alternating roles, rerank response type), replaces truthiness and call-count checks with whole-value assertions, and moves tests to the mirror file of the code they cover

* test: move cohere, databricks, fireworks and gemini rows to mirror files over mocked HTTP

Restores three TestCohereV2Transform methods the first pass nested out of collection, replaces direct transform calls with respx-backed completion calls, and drops the databricks and fireworks files that sat outside their mirror paths

* test: close lost-intent gaps found in review of the llm translation migration

Points the hosted-mock guard at the triton test's new integration path, restores exact rerank result assertions for infinity, drops the reasoning e2e fallback that passed for any multi-line answer, and adds the mixtral invoke system prompt variants, eval deletion, and async pdf url download that the legacy suite exercised

* test: assert provider acceptance in the reasoning effort e2e

The live half of the split grid proves each route accepts reasoning_effort. claude-fable-5-1 on the direct route answers without surfacing reasoning content, so a reasoning-presence check either fails on head or needs a fallback that passes for any answer. Wire shape stays covered by the unit grid

* test: cover async and stream-end paths the legacy translation suite exercised

Adds async variants for bedrock converse file and thinking content, bedrock embedding timeout and region resolution, interaction stream termination and success logging, xai reasoning token folding, and bytes chunks with a prefix before data:. Also scopes the vertex safety settings global in test_vertex_safety_settings with monkeypatch so it stops leaking into gemini request bodies on the same worker

* test: type the new llm translation e2e calls and mark moved fixture dirs as packages

* test: name the reasoning effort e2e models as module constants

* test: assert reasoning content, register the new gemini coverage ids and give the guardrail config e2e a unique name

* test: block the prompt the shared guardrail denies and drop the reasoning presence check

* test: drop the duplicate litellm import left by the main sync

---------

Co-authored-by: yuneng <yuneng@berri.ai>
2026-10-11 07:58:53 +00:00
devin-ai-integration[bot]
1dbde0845e
test: migrate SDK and proxy-booting legacy CircleCI tests to their tiers (#45758)
* test: move guardrails, image, audio, batches, search legacy tests to unit

* test: restore the custom guardrail and remaining chunk 1 unit rows

* test: move the ocr matrix to e2e

* test: drop comments from moved tests and fix vertex ocr credentials

* test: move litellm utils and unified google legacy tests to their tiers

* test: move logging callback and pass-through unit legacy tests to their tiers

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test: move store_model_in_db legacy tests and start the admin ui migration

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test: finish the proxy job rows and drop their CircleCI jobs

Moves the remaining multi-instance, openai endpoint, otel, pass-through and
top-level proxy tests into tests/integration with scripted upstreams, rewrites
the weak replacements to assert the outbound wire and exact responses, and
removes the build_and_test job group this PR owns from CircleCI

* test: tighten the key generation and error log assertions, rename migrated unit files to their mapped names

* test: shard the pagerduty unit tests and record the rust harness file on the coverage allowlist

* test: drive the SSO sign-in tests through a scripted IdP and isolate the cost center stand-in

* test: type the e2e beta header, prompt cache and secret manager values

* test: assert the missing model error message on the OpenAI routes

* test: drop the deleted otel base class import and restore mcp redaction and exact langfuse usage checks

* test: make the langfuse fixture directory a package for the layout check

* test: restore collected no-match cases, real langfuse and gcs payload checks, and gate the aws secret lane

* test: keep the ocr e2e pdf fixture next to the test

* test: put meta below covers on the streaming tool search e2e

* test: run the OCR matrix against a scripted upstream and fix two runner-only e2e failures

* test: count only OCR calls in the OCR matrix recorder

* test: pin OCR parse bodies and billed cost, group new e2e cases into classes

* ci: restore the python image anchor dropped in the main sync

---------

Co-authored-by: yuneng <yuneng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-11 00:17:21 -07:00
devin-ai-integration[bot]
a2e334febe
test: migrate local_testing SDK and provider tests out of build_and_test (#45760)
* test: migrate local_testing SDK and provider tests out of build_and_test

* test: cover sagemaker region, hf template and nova paths end to end

Region rows now go through litellm.completion on the sagemaker/ handler that
the legacy tests exercised, the prompt template test mocks the Hugging Face
tokenizer config instead of switching to custom_prompt_dict, streaming tests
assert role, finish reason and body, and Nova single turn and streaming run
through litellm.completion with a recorded request body

* test: tighten migrated integration tests for local OpenAI routes, timeouts and air-gapped boot

Split the catch-all providers migration file by behavior. The fake endpoint
route tests now hit the owned upstream directly, as the legacy harness tests
did. LM Studio asserts the outbound key, path and stripped model. The embedding
timeout test asserts the proxy maps a deployment timeout to a 408 and forwards
the read timeout to the provider client, instead of timing out the test client

The Azure streaming tool-call test also asserts the second identical stream is
served from the Redis cache with the same reassembled tool call

The Docker network test always skipped in CI because the security job has no
litellm/litellm image and no egress to pull one, and it tested a published image
rather than the checkout. It now boots the checkout proxy with every outbound
HTTP connection refused and asserts readiness, the bundled cost map fallback and
that the cost map host is the only egress attempted

* test: tighten the migrated e2e rows for local_testing SDK and provider tests

Drop the duplicated Anthropic covers marker and the Anthropic cell wrongly claimed by
the Bedrock and Vertex cache tests. Keep the file-wide Anthropic model as it is on main.
Send the 2023-06-01 version header to Anthropic through the deployment's extra_headers
so the header actually reaches the provider, and mark the creation-token case 1h then
5m like the legacy test did

Reuse the Bedrock file's own registration for the pydantic response test, swap the
hand-built PNGs for the shared cat.jpg fixture, assert the SageMaker Nova image answer
names the cat and that top_logprobs=3 is honored, and register the SageMaker Nova opt-in
markers the same way as the other opt-in markers

* test: restore lost assertions in migrated test_main tests

Restore the role alternation check in test_completion_anthropic_hanging, the
embedding and streaming response-header checks, the logit_bias drop check and
the gemini system and safety settings checks. Add an azure 429 retry-after
test for test_exception_with_headers, trigger the router fallback with
mock_timeout like the legacy test, and stop the mock timeout tests sleeping 3s

* test: restore lost legacy assertions in migrated vertex ai unit tests

* test: tighten migrated cost, streaming and embedding tests and scope the hosted mock guard

* test: strengthen migrated anthropic, azure, bedrock and cohere provider tests

Assert full request bodies and usage instead of presence checks, restore the
dummy-tool and custom_llm_provider assertions the legacy tests made, add the
async and per-model variants the legacy parametrization covered, split
streamed tool arguments across deltas, and move the Azure chat streaming
tests to the azure.py mirror file

* test: tighten migrated llms provider tests and cover legacy intents lost in the move

Strengthen the migrated fireworks, gemini, mistral, together, openai text completion, openrouter, perplexity and openai error tests so they assert what the legacy tests proved (cost, provider tag, usage-only-on-last-chunk, function_call rebuild, reasoning param, sync/async/stream error fields). Replace the huggingface and jina tests that doubled litellm code with respx end-to-end tests. Add the tests the pre-existing mapped tests never covered: cohere 400 mapping, token-array text completion, hosted_vllm/llamafile/lm_studio env api base embeddings, watsonx embedding auth, ollama ssl_verify client and ollama vision prompt

* test: cover residual legacy paths for anthropic tool streams, watsonx async stream, ollama http images and caller async embedding clients

* test: restore single request assertions in the deep_infra and azure_ai embedding tests

* test: drop a no-op streaming cost flag from the openai streaming cost test

* test: cover residual legacy-test lines in caching, main, utils, replicate, vertex and bedrock invoke

* test: cover streamed audio assembly, choice membership, untranslated responses events and the stream log guard

* test: pin cost rates in unit tests, use sdk for cohere e2e, drop vertex llama e2e

* test: send the vertex maps e2e request through the openai sdk

* test: drop the local_testing jobs that select no tests after the migrations

* test: wait for the streamed cache entry and keep custom Mistral rates test-local

* test: keep untouched Mistral test lines as they were

---------

Co-authored-by: yuneng <yuneng@berri.ai>
2026-10-10 23:14:25 +00:00
Shubham Jain
985574f3dc
fix(bedrock): keep GPT tool-result images beside the tool result (#45579)
* fix(bedrock): keep GPT tool-result images beside the tool result

Bedrock gpt-6.1-sol rejects an image nested in toolResult.content. Put it on the same user turn instead.

* fix(bedrock): read tool-result image support from the price map

OpenAI GPT rows on Bedrock Converse reject an image inside toolResult.content, so those images sit beside the tool result.

* fix: sync price-map test schema and import order

CI validates model_prices_and_context_window.json against an inline schema in test_utils, and ruff checks import order in converse_transformation.

* fix(bedrock): satisfy basedpyright in tool-result image helper

Drop Final assignments inside loops and use TypedDict key checks so the lint type-check gate stays within its merge-base ceiling.

* test(bedrock): cover tool-result image placement helper

Exercise Claude no-op paths, text-only GPT tool results, status preservation, and split text/image tool content for codecov.

* refactor(bedrock): reuse the shared tool-result image placeholder text

---------

Co-authored-by: ryan-crabbe-berri <ryan@berri.ai>
2026-10-09 18:02:23 -07:00
devin-ai-integration[bot]
9f5de05b9d
fix(bedrock): drop stop sequences for GPT 5.6 and newer instead of forwarding them to a 400 (#45664)
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-09 15:00:24 -07:00
devin-ai-integration[bot]
8080601a95
feat(bedrock): default Grok on Bedrock to native Chat Completions, mint unique tool call ids and drop stop (#45473)
* fix(bedrock): route Grok 4.7 tools with reasoning to native Chat Completions

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(bedrock): mint unique tool call ids on native Chat Completions

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(bedrock): send a blank text block for Converse tool results with no supported content

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* feat(bedrock): default Grok on Bedrock to native Chat Completions and leave Converse unchanged

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(bedrock): gate tool call id minting to Grok on native Chat Completions

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(bedrock): check the Grok tool call id gate at both call sites

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(bedrock): remint positional tool call ids on native Chat Completions for every model

AWS returned call_0, call_1 for GPT 5.6 as well as Grok on 2026-10-09 (and unique ids for GPT 5.6 earlier the same day), so the id format is not fixed per model. Drop the Grok-only gate and remint any call_<digits> id on the route

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(bedrock): drop stop sequences for Grok instead of forwarding them to a 400

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-09 20:56:03 +00:00
yuneng-jiang
3250ccc802
test: move offline logging, secret manager and provider tests from legacy dirs into tests/unit (#45554)
Moves 58 legacy nodes (cyberark, logging callback manager, langfuse handler and helpers, SQS, app crypto, bedrock nova and invoke, cloudflare, litellm_proxy, nvidia nim, replicate, triton, convert_dict_to_response, get_model_info, vcr live-call probe) into the tests/unit files that mirror the modules they exercise. Calls that reached huggingface.co, langfuse or 0.0.0.0 are refused at the HTTP boundary with respx. Tests that need credentials, a license, a spawned server, sleeps, a subprocess, or assert nothing stay in their legacy files
2026-10-09 09:48:10 -07:00
joshua-berri
911aff2752
refactor(sdk): separate core AWS and tokenizer dependencies (#44447)
* refactor(sdk): separate core AWS and tokenizer dependencies

* fix(sdk): preserve runtime tokenizer alias compatibility

* test(sdk): compare tokenizer installs through existing entry point

* fix(sdk): preserve bearer headers and optional dependency interfaces

* fix(sdk): retain safe tokenizer fallback diagnostics

* test(sdk): inject tokenizer dependency for fallback diagnostics

* fix(core): preserve AWS dependency errors with retries

* fix(core): centralize optional AWS dependency handling

* refactor(core): reuse optional import helper for Invoke streams

* test(core): compare complete shared dependency requirements

* fix(core): keep tokenizer logging import compatible with main

* fix(core): preserve native token decoding and Responses dependency errors

* test(core): isolate optional dependency import failures

* fix(sdk): retain typed exception message access

* test(sdk): scope HTTP verification environment changes

* fix(sdk): preserve bearer request typing across Bedrock callers

---------

Co-authored-by: Joshua Valluru <326636767+joshua-berri@users.noreply.github.com>
2026-10-08 19:07:57 -07:00
devin-ai-integration[bot]
6a2d3edce3
fix(bedrock): split the <reasoning> tag for gpt-oss only on native Chat Completions (#44432)
* fix(bedrock): split the <reasoning> tag for gpt-oss only on native Chat Completions

The native Chat Completions route moved a leading <reasoning>...</reasoning>
block into reasoning_content for every model, while only gpt-oss writes its
reasoning inline. A GPT 5.6 or Grok answer that starts with a literal
<reasoning> tag lost that text, streaming and non-streaming alike. Both paths
now split only when the model id is gpt-oss.

* fix(bedrock): drive the inline reasoning split from a cost-map flag

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-10-08 16:32:07 -07:00
devin-ai-integration[bot]
4f62bbfd8b
fix(bedrock): remove the bare xai.grok-4.7 cost-map row that AWS cannot invoke (#45460)
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-08 23:02:31 +00:00
devin-ai-integration[bot]
fa2c8984ba
test: move the unit half of 126 mixed legacy files into tests/unit (#45090)
* test: move the unit half of 126 mixed legacy files into tests/unit

* test: restore litellm globals that moved tests set

* test: finalize migration test cleanup

* test: restore original bodies of moved legacy tests

The move into tests/unit had rewritten 612 test bodies, and some of the rewrites dropped assertions. Each moved test now carries its original body from the legacy file, with only the imports, helpers, fake provider credentials and monkeypatched env it needs to run under tests/unit

test_timeout_streaming goes back to tests/local_testing because it needs the fake OpenAI endpoint server. The image payload fixture moves with its only user, and two tests that leaked global state (a registered model cost entry and queued logging tasks) are now isolated

* test: drop module imports shadowed by restored local imports

* test: assert on LiteLLM output in no-assertion moved tests and isolate leaks

Twenty no-assertion candidates get one assertion on the value LiteLLM returns, with the original lines unchanged. Four tests go back to their legacy files because they only check types or imports, write into the working directory, or cannot assert without a body change

Two moved tests leaked globals into later tests in the same worker, so monkeypatch fixtures now restore the retry-after header parser and the end user cost tracking flags

* test: drain queued logging tasks before the Phoenix span test

The moved Phoenix test counted spans from logging tasks that earlier tests had queued, so the drain fixture moves to tests/unit/conftest.py and both it and the Datadog batch test use it. test_factory_function goes back to its legacy file because its returned wrapper calls the real Assistants API and cannot be asserted on without a body change

---------

Co-authored-by: yuneng <yuneng@berri.ai>
2026-10-07 14:07:43 -07:00
devin-ai-integration[bot]
e2971e0af4
refactor(llms): expose public names for private provider helpers (#45037)
* refactor(litellm): migrate private usage in llms

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(litellm): preserve Bedrock batch signature marker

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(litellm): migrate llms private usage symbols

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(litellm): preserve UUID Watsonx project IDs

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(litellm): retarget llms mocks to public names

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(litellm): limit llms changes to renames and forwarders

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(litellm): keep GCS mock client patching private Vertex auth methods

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(litellm): assert forwarder arguments and type forwarder signatures

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-07 08:49:25 -07:00
devin-ai-integration[bot]
dfe4df8c8a
test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807)
* test: delete unconditionally skipped legacy tests

* test: move whole-unit legacy test files into tests/unit

* test: keep moved legacy tests free of import-time global state

* test: keep the module-level invocation scan pointed at tests/local_testing

* ci: drop the agent_testing CircleCI job emptied by the move

* test: fix moved-test isolation and router coverage

* test: add Tinyfish search package marker

* test: isolate moved tests from logger state leaks

* test: isolate Helicone logging fixture state

* test: isolate Vertex pass-through credentials between moved tests

* test: cancel S3 periodic flush tasks started by moved tests

* ci: restore CircleCI assistant test selection after move

* test: prevent Datadog datetime import shadowing

* ci: drop the litellm_assistants_api_testing CircleCI job emptied by the move

* test: deduplicate imports in rebased unit tests

* test: remove duplicate passthrough router patch import

* test: remove moved legacy source files after rebase

* test: align moved tests with rebased main

* test: carry main's legacy-file edits into moved destinations

* test: make the moved cost map fallback tests assert the fetch and the backup

The four fallback cases only checked the result was non-empty, so they still
passed with integrity validation disabled. They now inject a mock client, assert
one fetch happened, and assert the result is exactly the local backup with the
fallback reason recorded.

---------

Co-authored-by: yuneng <yuneng@berri.ai>
2026-10-07 05:02:28 +00:00
devin-ai-integration[bot]
c3a23fe499
refactor: expose core private helpers under public names (#44871)
* refactor: expose core private symbols with compatibility aliases

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* types: narrow core migration diagnostics

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix: preserve runtime behavior in core symbol migration

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix: preserve core private usage migration behavior

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix: preserve private value rebinding compatibility

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix: restore optional imports and cover public helpers

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test: exempt router property from call coverage

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test: align recursive detector ignore names

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-07 00:05:27 +00:00
devin-ai-integration[bot]
f81a3f5243
fix(responses): report truncated bridged output as incomplete and echo request params (#44460)
* fix Bedrock json mode bug

* fix finish reason when max tokens hit

* fix hardcoded completion for streaming responses

* fix: echo Responses API request params onto bridged response

* refactor(responses): drop explanatory comments and build null finish_reason test without mutation

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(responses): type the terminal event and Bedrock mock helpers

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(responses): keep truncated WebSocket turns in history and drop invalid echoed params

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Mrinal Chanshetty <mrinal@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-05 11:27:12 -07:00
devin-ai-integration[bot]
1b98748528
fix(bedrock): honor the per-request timeout on Converse and Invoke streaming (internal copy of #38210) (#44134)
* fix(bedrock): propagate timeout to streaming requests

* test(bedrock): prove streaming fails at the request timeout against a slow upstream

* test(bedrock): simulate the slow upstream in process instead of over a local socket

* test(bedrock): audit the Converse and Invoke stream timeout on the proxy

Two integration files drive the per-request timeout on Bedrock streams
through the real proxy against an owned wire peer: the wire file covers
every surface (chat, messages, responses, invoke, pass-through), the
sad, edge and precedence rows, and the chaos file covers bursts, a
dropping upstream, a killed worker and a proxy stopped mid-burst.

The wire peer gains Reply.drop_connection so a cell can close the
socket before any response, and the harness's graceful stop grace is
now INTEGRATION_PROXY_STOP_SECONDS (default unchanged at 30), since a
two-worker supervisor's interpreter finalization takes longer than that
on a loaded box.

* test(bedrock): pin the fallback audit cell to one proxy worker

The fallback cell created both deployments through /model/new on one
worker and sent the chat request to the other, whose registry
read-through loads only the requested model, so the fallback target
was unknown there until the periodic DB poll. The cell now warms the
fallback model and sends the request over one keep-alive client, so
one TCP connection stays with one uvicorn worker, and it expects the
fallback upstream to see both requests.

---------

Co-authored-by: Sainyam Kapoor <hello@sainyam.me>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-10-03 19:29:15 +00:00
devin-ai-integration[bot]
6c32384d8c
fix(bedrock): stop emitting Converse cachePoint blocks for Kimi K3 (#44292)
* fix(bedrock): stop emitting Converse cachePoint blocks for Kimi K3

Bedrock prices Kimi K3 cache reads through implicit caching but Converse
rejects the explicit cachePoint marker ("This model doesn't support the
cachePoint field"), so any cache_control on the request answered 400.
Mark the three K3 rows supports_prompt_cache_breakpoint: false and have
bedrock_model_accepts_cache_points honor that flag before falling back to
supports_prompt_caching, keeping cached-token pricing intact.

* test(bedrock): assert cache points per request section

* fix(bedrock): honor a deployment's cache breakpoint flag for unmapped models

* fix(bedrock): read a converse-routed deployment's cache breakpoint flag

* refactor(bedrock): look up cache breakpoint flags by key

* test(bedrock): add the Kimi K3 cache point wire audit

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-10-03 18:11:46 +00:00
Aasif Multani
54260bec88
fix(bedrock): honor unsupported reasoning effort levels for OpenAI GPT models (#44183)
Bedrock GPT-5.6 rejects reasoning effort minimal with 400 unsupported_value. The model map already
sets supports_minimal_reasoning_effort=false for these models, but neither the Converse path nor the
native Responses path read it, so the value was forwarded. Drop it under drop_params and raise
UnsupportedParamsError otherwise, matching how the OpenAI GPT-5 path treats explicitly disabled
effort levels

Co-authored-by: Aasif-Multani <20943280+Aasif-Multani@users.noreply.github.com>
2026-10-02 21:03:58 -07:00
devin-ai-integration[bot]
233db9f285
feat(bedrock): serve gpt-5.6+ chat completions natively by default, with chat_completions/ opt-in for gpt-oss and grok (#44307)
* feat(bedrock): send grok chat completions through runtime openai path

Unspecified bedrock grok was rewritten to Converse. Chat completions now hit bedrock-runtime /openai/v1/chat/completions, and converse/ still uses Converse

* feat(bedrock): serve gpt-oss and gpt-5.6 chat completions on runtime's native openai path

* fix(bedrock): route gpt-oss response_format to Converse and decide the route once from the raw request

* fix(bedrock): serve region-path and GovCloud gpt-oss ids on native Chat Completions

The cost-map parity tests require every regional variant of a flagged id to carry the same supports_ flags, so the six us-gov gpt-oss entries now carry the native-route flags too. A region path in the model name (bedrock/us-gov-west-1/openai.gpt-oss-20b-1:0) is routing, not a different model: the route is looked up on the id after the path, the path's region picks the endpoint and the SigV4 scope, an explicit aws_region_name still wins, and the body carries the bare id AWS expects

* fix(bedrock): keep params AWS refuses natively off the chat completions route

Drop the params each family 400s or 503s on runtime Chat Completions from the native config's supported list (GPT-5.6 penalties, stop, and logprobs, Grok penalties, gpt-oss logit_bias) so drop_params drops them as Converse did, gate legacy functions on GPT-5.6 the same way as tools, and send an Anthropic-style thinking block to Converse, the only route that forwards it

* fix(bedrock): keep schema-less json_object on Converse for the chat completions models

* fix(bedrock): keep every json_object response_format on Converse for the chat completions models

* fix(rust): declare the bedrock runtime chat completions flags on ModelInfo

* fix(bedrock): opt into the native chat completions route through supported_endpoints

* docs(cost-map): describe the bedrock native chat completions capability flags

* revert: docs(cost-map): describe the bedrock native chat completions capability flags

This reverts commit 4101c0ceb2.

cost-map-guard runs main's schema generator under pull_request_target and compares
its output to the PR's committed schema, so a PR that changes the generator's output
cannot pass that required check until the generator change lands on main first. The
descriptions move to a follow-up that lands the generator change ahead of the schema

* test(bedrock): move the native chat completions tests under tests/unit

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(bedrock): drop reasoning_effort none for grok on the native chat completions route

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(bedrock): keep converse extension params on the converse route

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(bedrock): inline http image urls and keep stop on converse for native chat completions

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(bedrock): share the sync remote media inliner

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(image-handling): infer the image mime type when the server sends a generic content type

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(bedrock): stop sending aws_bedrock_project_id as OpenAI-Project on the runtime chat completions route

* feat(bedrock): make native chat completions an opt-in bedrock/chat_completions/ route

Bare Bedrock OpenAI and Grok model ids stay on Converse as on main. The
bedrock/chat_completions/<model> prefix opts a deployment into bedrock-runtime's
/openai/v1/chat/completions, and a request carrying a Converse-only param
still falls back to Converse. The cost map no longer decides the route.

* fix(bedrock): keep chat_completions/-prefixed deployments on the native Responses surface

* fix(bedrock): keep provider response headers on the runtime chat completions route

* feat(bedrock): serve gpt-5.6 and newer on runtime chat completions by default

Unprefixed bedrock/<gpt-5.6+> models whose cost-map row lists /v1/chat/completions
now route to the native OpenAI-compatible endpoint; converse/ pins Converse and
chat_completions/ still opts gpt-oss and Grok in. Guardrails, application inference
profile ARNs, and tools with reasoning keep falling back to Converse per request.
Hoist the remote-media url comprehension into a single-clause helper.

* fix(bedrock): refuse temperature and top_p natively on GPT 5.6 and newer like Converse does

AWS answers temperature and top_p with a 400 on the native Chat Completions endpoint for the GPT 5.6+ models, the same models whose Converse route already dropped both under drop_params via supports_sampling_params: false. The native config now honors that price-map flag, the gpt-6 and gpt-6.1 rows carry it, and the gpt-6 family joins gpt-5 in refusing frequency_penalty, presence_penalty, logprobs, and top_logprobs before the request reaches AWS.

* fix(bedrock): refuse GPT sampling and logprob params natively only while reasoning is on

On bedrock-runtime's native chat completions endpoint, GPT-5.x and GPT-6.x
accept temperature, top_p, frequency_penalty, presence_penalty, logprobs,
and top_logprobs once reasoning_effort is "none", and refuse them with any
other effort or when the effort is unset. The previous commit refused the
sampling params unconditionally from the cost map's supports_sampling_params
flag, which lost the reasoning-off case and never covered the penalties or
logprobs. The refusal now keys on the model being a GPT id and reasoning
being active, raises a 400 UnsupportedParamsError naming the params unless
drop_params drops them, and lets everything through under "none". Grok and
gpt-oss keep their unconditional family refusals.

* refactor(bedrock): keep the Converse route-prefix strip inside the bedrock llms module

* fix(bedrock): forward a non-string reasoning_effort on the native route instead of crashing

A list or dict reasoning_effort hit a frozenset membership test in
without_refused_reasoning_effort and raised TypeError, which the proxy
surfaced as a 500 APIConnectionError with no upstream call. The value is
now left alone unless it is a string Bedrock's native endpoint refuses,
so AWS answers the malformed value with its own 400 like it does for an int

* fix(bedrock): route overlong GPT version digits to Converse and send native chat completions to the runtime endpoint

A model id with more than 4300 version digits raised ValueError in the route check; the digits are now bounded so such ids fall back to Converse. The native chat completions URL now follows Converse's precedence: aws_bedrock_runtime_endpoint (or AWS_BEDROCK_RUNTIME_ENDPOINT) wins over api_base, so a deployment that sets both keeps sending to the same host

* fix(bedrock): route model_id overrides to Converse and never send an empty bearer natively

A deployment whose litellm_params carry model_id (an application inference profile or provisioned throughput ARN) went to the native Chat Completions route with the base model in the URL and model_id left in the body. It now takes Converse like the bedrock/arn:... model form, which encodes the override into the request URL

A blank api_key on a SigV4 deployment became an Authorization header reading Bearer with nothing after it on the native route, since the OpenAI-like header builder writes any non-None key and the signer keeps a non-AWS4 Authorization header. validate_environment now resolves the key through bedrock_bearer_token, so a blank key is signed with SigV4 the way Converse signs it

* test(bedrock): audit the native GPT chat completions route on the integration rig

Adds the /audit cells for the runtime chat completions route: the scripted Bedrock runtime peer, the happy and fallback wire tests, the sad-path and regex worst-case tests, the chaos burst tests, the Messages adapter tests, and the Responses native-route tests. Tests only, no product diff.

* test(bedrock): harden the runtime chat completions audit cells

The chaos peer's shared counter and process now come from the same spawn context, since a fork-context Value handed to a spawn-context process raises on Linux. The peer-kill test waits for the first six answers to reach the client before killing the peer instead of counting accepted requests. The Responses wire tests look the spend row up under both the ciphertext id the caller received and the issued id behind it, matching the chaos file's rule for the pre-encryption row

* fix(bedrock): refuse or drop a non-string reasoning_effort before the native chat completions call

A reasoning_effort sent as an int, a list, or an object on a GPT 5.6+ deployment the native
route serves now answers 400 from litellm before any wire request, naming the type and the
drop_params way out, and is dropped under drop_params so AWS applies its default effort, the
way Converse dropped it on main. The tip since a0cef91f0b forwarded it for AWS to refuse

* test(bedrock): pin router retries off and give the chaos bursts config deployments on an owned proxy

* test(bedrock): wait for the replacement worker before tearing down the sigkill chaos proxy

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo <mateo@berri.ai>
2026-10-02 19:55:54 -07:00
devin-ai-integration[bot]
5dbe4f95e8
feat(bedrock): drop lookaround regex patterns from tool schemas for Converse models that reject them (#44138)
* feat(bedrock): drop lookaround regex patterns from tool schemas for Converse models that reject them

* fix(bedrock): rename the lookaround flag to supports_regex_lookaround and keep dropped patternProperties names allowed

The cost-map flag becomes a generic supports_regex_lookaround capability, which the
cost-map schema admits as a supports_* boolean, and the Converse transform now owns
the drop decision instead of the shared tools factory. A patternProperties key dropped
from an object closed by additionalProperties: false leaves its value schema as that
object's additionalProperties, so the names it allowed stay allowed, on the OpenAI
non-Python-regex drop too. tool_with_sanitized_parameters also cleans Anthropic-shape
tools (input_schema).

* fix(router): keep a deployment's supports_regex_lookaround off the shared cost-map entry

A deployment's model_info.supports_regex_lookaround was written to the shared
bedrock/<model> cost-map key, so every sibling deployment of that model id
inherited one deployment's choice. The flag now stays under the deployment's
own id, which is what the Converse lookaround check reads first, and the
shared entry keeps the cost map's value

* test(bedrock): audit the Converse lookaround drop on the wire across endpoints, SDKs, flags and chaos

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-10-02 19:47:25 -07:00
devin-ai-integration[bot]
8b4de39ad7
fix(bedrock): surface unrecognized converse-stream event frames instead of an empty turn (#44112)
* fix(bedrock): surface unrecognized converse-stream event frames instead of an empty turn

* fix(bedrock): track distinct unknown stream event types and cover the stream error builder

* test(bedrock): audit converse-stream event frames on the proxy and the sync decoder

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-10-02 16:06:26 -07:00
devin-ai-integration[bot]
610576d247
fix(bedrock): accept Converse messages with no content key (#43936)
* fix(bedrock): accept Converse messages with no content key

A user or tool message whose content key is missing (or null, which the
message cleanup strips) made every Bedrock Converse request fail with
APIConnectionError 'content' before reaching Bedrock. The Converse
transform now reads content with .get for those messages, as it already
did for assistant messages: a content-less user message adds no block
and a content-less tool message becomes a toolResult with empty content.
The str branch also sends the continue message text instead of the
original whitespace-only text.

* fix(bedrock): send the continue message for a content-less user turn

* test(bedrock): type the content-less Converse message test parameters

* fix(bedrock): accept a Converse system message with no content key

* refactor(bedrock): read the system message content with get

* test(integration): audit Bedrock Converse messages without content

Adds the /audit cells for a chat message whose content key is missing or
null on a Converse-routed Bedrock deployment: happy, sad, edge, and chaos
rows through the OpenAI SDK, the Anthropic SDK, and raw httpx against the
scripted upstream, asserting the caller's response, the body the peer
received, and the spend row. The owned-proxy readiness deadline in the
integration harness is now INTEGRATION_PROXY_READY_SECONDS (default 70).

* test(integration): bound stray spend rows in the mid-burst restart cell

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-10-01 15:14:34 -07:00
shrey-berri
c2ae483782
fix(bedrock): add beta for thinking display updates (#43832) 2026-10-01 09:21:10 -07:00
shrey-berri
e494723105
fix(params): validate stream_chunk_size once, before any provider call (#43222)
* fix(params): validate stream_chunk_size once and carry it as typed control options

Checks stream_chunk_size at the top of completion() and acompletion(), accepts
digit strings, returns a 400 naming the param unless drop_params is set, and
stores the checked value under _litellm_control. Bedrock Converse and Invoke
read it from litellm_params; the Bedrock-only checker and the dead Invoke pops
are gone. Owned-kwarg filtering now runs through one helper everywhere.

Refs LIT-8317

* test(bedrock): drop tests for the removed stream_chunk_size_from helper

Refs LIT-8317

* fix(params): check stream_chunk_size before the MCP gateway branch

Refs LIT-8317

* fix(params): return assert_never in the exhaustive control-options match

Refs LIT-8317

* fix(params): address council review of the control options change

Read all_litellm_params live so names registered after import stay
LiteLLM-owned, make litellm_params a required keyword on the stream
wrapper hooks, give digit strings and ints the same 18-digit range,
share the default-chunking test table, test the Responses bridge through
litellm.responses, and revert formatting-only churn in existing tests.

Refs LIT-8317

* fix(params): address the second council review of control options

Keep the Responses bridge on its original all_litellm_params forwarding,
narrow _int_from_decimal_string inline so it type-checks, bound nested
huge ints in the error message, store _litellm_control only when a value
is set, simplify the parser to its single field, drop the one-caller
wrapper, and tighten the tests.

Refs LIT-8317

* fix(params): keep the 18-digit length check on stream_chunk_size strings

A 19-character string with leading zeros such as 0000000000000000001 would
otherwise pass as 1, although the rule and the error message say at most
18 digits.

Refs LIT-8317

* test(params): tidy control options tests after council sign-off

Move the Responses bridge test into the existing bridge test file, drop the
rebind test that pinned an implementation detail, assert through
stored_control_options instead of the storage key, and cover
drop_params="true" through Bedrock streaming.

Refs LIT-8317

* test(params): wrap a chunking test row that went past 120 characters

Refs LIT-8317
2026-09-26 23:01:20 +00:00
devin-ai-integration[bot]
b6fcd03848
fix(bedrock): surface a converse-stream 200 that decodes to no events as a 502 instead of an empty turn (#43213)
* fix(bedrock): surface a converse-stream 200 that decodes to no events as a 502 instead of an empty turn

* fix(bedrock): quote the body head only when a stream decoded no events

The leftover-bytes error keeps the byte and event counts, the content type and the request id but no longer quotes the first bytes of a stream that already decoded events, since that head is the start of a healthy stream and can hold model output. The anthropic_messages empty-stream warning no longer prints the request's model name.

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-09-25 15:18:03 -07:00
yuneng-jiang
5e6dc89ba1
test: move tests/test_litellm/llms into tests/unit/llms (#43191)
* ci: run the unit_selection.sh shard files on every event instead of only fork pull requests

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ci: rename fork-flag to unit-flag now that it applies on every event

* test: move tests/test_litellm root and small trees into tests/unit

Pure renames, no content changes. Follow-up commits in this PR fix
references, merge the three files that already existed in tests/unit,
keep live-provider tests in tests/test_litellm and wire CI.

* test: carry tests/test_litellm conftest isolation into tests/unit

Callback lists, routing fallbacks, cached HTTP clients, logger state, AWS,
proxy-URL and keychain env, and session-end client cleanup now reset for
unit tests too. The environment isolation owns its MonkeyPatch so a test's
own monkeypatch is undone before the model-cost teardown runs.

* test: merge, split and prune the moved root and small-tree tests

Merge batches/test_batch_utils.py and the chat_completions and messages
dispatch tests into the files that already existed in tests/unit. Keep
the live Gemini interactions tests, the async image-fetch format test and
the OpenAI embedding scorer test in tests/test_litellm since they need
real network or keys. Put test_router.py under tests/unit/test_router so
the existing package no longer shadows it. Delete eight tests the audit
found superseded by stronger ones kept in this move.

* ci: run the moved root and small-tree tests under their legacy flags

Add the misc and responses-caching-types flags to unit_selection.sh and
CircleCI, extend enterprise-routing and mcp-integration, and point the
legacy GHA shards, Makefile, redis-compat workflow, merge smoke manifest
and change classifier at the new paths.

* test: make the new tests/unit directories packages

tests/unit/test_package_layout.py requires every directory to carry an
__init__.py, and without one the moved and retained
test_litellm_responses_bridge.py modules collide on import.

* test: scope the unit socket block to tests/unit in shared sessions

The GHA shards collect the legacy test-path and the unit selection in one
pytest session. The unit conftest's loopback-only block leaked into legacy
modules that reach the network at import. The legacy conftest now lifts the
restriction at collect and setup time, and the unit conftest re-applies it
when collecting its own modules.

* test: move tests/test_litellm/llms into tests/unit/llms

Rename-only. Moves the provider tests and the fine-tuning fixtures they
load, mirroring the old paths. Follow-up commits merge, split and wire them.

* test: merge, split and prune the moved llms tests

Merges the Databricks chat transformation tests into the existing unit
file, keeps the tests that need real keys or the network in
tests/test_litellm, deletes the audited tests a stronger unit test
already covers, and points imports at tests.unit.llms.

* ci: run the moved llms tests under their legacy flags

The Vertex AI and All Other Providers shards keep their legacy test-path
for the retained files and add the llm-vertex-ai and llm-other-providers
unit selections. CircleCI gets matching unit jobs.

* test: make the tests/unit/llms directories packages

Adds __init__.py to the moved dirs and drops the legacy ones whose
directories no longer hold tests.

* test: drop script runners and path hacks the llms split left dangling

The __main__ runners in the split openai_like files and the Databricks e2e
runner called tests that now live in the other half of the split or were
deleted. The retained legacy halves also no longer need sys.path edits.

* test: give the shard-script tests their own GITHUB_OUTPUT

They only passed where the runner set it. The CircleCI unit job's env
allowlist drops it, so the script's redirect failed there.

* test: point the router and module-deletion checks at tests/unit

router_code_coverage and code_qa_check_tests only searched tests/test_litellm,
so the moved router tests no longer counted. The two silent-experiment tests
the audit deleted were the only direct callers of those methods; they are
replaced with tests that assert the forwarded shadow request and the
recursion guard.

* test: keep the Databricks manual e2e runner and fix the SageMaker Nova run path

The Databricks e2e file is a manual script whose main() calls the tests
that were pruned, so pruning them broke the documented run. It is back to
its main version. The SageMaker Nova docstring now points at the file's
real location in tests/local_testing.

* test: keep the job's UNIT_FLAG out of the shard-script tests

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-25 12:43:23 -07:00