Resolves the conflicts with the WebSocket request defaults from main (PR #41881):
the relay keeps both custom_llm_provider and request_defaults, and a masked
response.create frame is re-serialized when the defaults changed it.
Keeps the first-frame routing hints (input, previous_response_id) out of the
deployment request defaults so they never get injected into later frames on the
same connection, with a regression test.
Pass-through requests inject the proxy span into upstream headers since #40669, which
replaced an explicit x-pass-traceparent with an unrelated trace and dropped its
x-pass-tracestate. Keep the caller's context when the carrier already names a
different trace, and keep the proxy child span for same-trace or missing headers.
Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Candidates.finishReason listed eleven values while the mapping key set
carried twenty-one, so typed fixtures could not spell the reasons this
PR handles. GeminiFinishReason is now the one list, the key set derives
from it, and a test checks every documented reason has an explicit
mapping instead of falling through to "stop"
* fix(proxy): name the blocking guardrail in x-litellm-applied-guardrails
When a guardrail hook raises, the common ProxyLogging dispatch (sequential and parallel pre_call, pipeline block, during_call and post_call metrics wrapper, streaming iterator wrapper) now records that guardrail in applied_guardrails before re-raising, and pre_call_hook folds request-declared guardrails in on its raising path. Buffered streams rebuild their response headers after the first chunk so a post_call block reached while buffering carries the blocker too
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): attribute only the raising layer in stream and pipeline blocks
The streaming wrapper caught every exception crossing its boundary and named its own
callback, so a block by an inner guardrail or a provider stream failure also named every
outer guardrail. The wrapper now runs the hook over an upstream boundary that remembers
the exception it raised, and skips attribution when the same exception passes through
Pipeline blocks converted from SensitiveDataRouteException or ModifyResponseException into
a generic guardrail_pipeline_error now still record the blocking step's guardrail
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(proxy): drop explanatory docstrings from the stream attribution helpers
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Replace the module-level LATEST_EDITS list with per-example callback
registry isolation, and import litellm names with from-imports in the
legacy callback shim so the module uses one import style.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The payload boundary of callbacks-legacy gets a model-based proptest: for any
JSON body, caller keywords and callback edit, keywords the route sends unchanged
reach pre_call as the caller's own objects, and the wire is the body pre_call
received as the callback left it. A parametrized test pins that a keyword the
bridge never reads keeps its identity through setup, the deployment hook,
check_limits and prepare.
Behaviour owned by the real Logging object is pinned end to end in the OCR
tests: a hypothesis version of the body property over HTTP, sync hooks seeing no
running event loop, retained payloads staying intact after the call, success
callbacks sharing one standard logging payload, and state stashed before a
blocking deployment hook raises reaching both failure callback families
Every legacy callback call from callbacks-legacy now goes through one typed
Python shim, litellm.rust_bridge.legacy_callbacks, the only Python module
the crate reaches. Before, the crate called Logging methods, litellm.utils
hooks, the logging worker, the executor and several litellm globals
directly, and its tests retyped those signatures by hand, so an outdated
fake could accept a call the real code rejects. python_contract.json lists
each shim function's parameters: a Python test pins it to the real
signatures and a Rust test pins it to the Rust enum.
The lifecycle contract changes to match the Python @client wrapper:
- the driver emits CallEvent::Started before begin, so every host sees one
start time
- RequestContext carries the route-resolved api_key, so legacy pre_call and
post_call receive it, and post_call's additional_args match the Python OCR
path
- Passthrough and its re-aliasing are gone
- async deployment hooks always run, and the "no callbacks" shortcut that
skipped the logging payload is removed, as in the Python path
The OCR api_key is a SecretValue from the wire request onward, so Debug
output upstream of the callback contract cannot leak it.
host-python's RouteHost now classifies native failures once through
classify, and host ops return HostOpError. The OCR route host keeps main's
public errors by sending both through the existing Python map_failure.
A deployment whose reasoning_effort is an object is copied through as
reasoning the way the HTTP mapper does it instead of being dropped, and
the relay re-asserts the response.create frame type after merging
extra_body so a type key inside it can never replace it. The lazy
OpenAPI snapshot goes back to main: the earlier regeneration came from a
Python 3.14 interpreter dedenting docstrings, which CI on 3.12 rejects
A content-less candidate is now kept as a choice whenever it carries a
finishReason, with the raw value on the choice's provider_specific_fields.
NO_IMAGE, IMAGE_RECITATION, IMAGE_OTHER and ESCALATION map to content_filter;
UNEXPECTED_TOOL_CALL and MISSING_THOUGHT_SIGNATURE map to stop. The
/v1/responses bridge reports content_filter and refusal as incomplete with
incomplete_details, and tool calls and reasoning no longer leak from one
candidate into the next.
The httpx.Response built for a Rust upstream failure had no request attached,
so constructing openai.AuthenticationError raised RuntimeError inside the
exception mapper and every bad-key OCR call surfaced as APIConnectionError 500
instead of AuthenticationError 401 (the Python path already returned 401)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Authorization and pricing read the first model and language query value, but the raw query was forwarded, so Deepgram (which honours the last repeated value) could be sent a model the key was never allowed. Later duplicates of those two keys are now dropped before the upstream URL is built
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
A caller could pick a model with only a pre-recorded registry row, or no row at all, and the session would be billed at the pre-recorded rate or logged at zero cost, so budgets did not apply. The route now closes the WebSocket with 1008 before dialing Deepgram unless deepgram/streaming/<model> (or the -multilingual row for language=multi) is an exact registry hit, and the logging handler applies the same check so a registry change under a live session records the duration with no cost instead of a substitute rate
Regression tests cover the route refusal, an operator-supplied streaming row for another model being accepted, and the handler never substituting the pre-recorded rate
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
