Pass-through requests inject the proxy span into upstream headers since #40669, which
replaced an explicit x-pass-traceparent with an unrelated trace and dropped its
x-pass-tracestate. Keep the caller's context when the carrier already names a
different trace, and keep the proxy child span for same-trace or missing headers.
Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Candidates.finishReason listed eleven values while the mapping key set
carried twenty-one, so typed fixtures could not spell the reasons this
PR handles. GeminiFinishReason is now the one list, the key set derives
from it, and a test checks every documented reason has an explicit
mapping instead of falling through to "stop"
* fix(proxy): name the blocking guardrail in x-litellm-applied-guardrails
When a guardrail hook raises, the common ProxyLogging dispatch (sequential and parallel pre_call, pipeline block, during_call and post_call metrics wrapper, streaming iterator wrapper) now records that guardrail in applied_guardrails before re-raising, and pre_call_hook folds request-declared guardrails in on its raising path. Buffered streams rebuild their response headers after the first chunk so a post_call block reached while buffering carries the blocker too
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): attribute only the raising layer in stream and pipeline blocks
The streaming wrapper caught every exception crossing its boundary and named its own
callback, so a block by an inner guardrail or a provider stream failure also named every
outer guardrail. The wrapper now runs the hook over an upstream boundary that remembers
the exception it raised, and skips attribution when the same exception passes through
Pipeline blocks converted from SensitiveDataRouteException or ModifyResponseException into
a generic guardrail_pipeline_error now still record the blocking step's guardrail
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(proxy): drop explanatory docstrings from the stream attribution helpers
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Replace the module-level LATEST_EDITS list with per-example callback
registry isolation, and import litellm names with from-imports in the
legacy callback shim so the module uses one import style.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
core/src/machine was route-neutral runtime code sitting among route surfaces, and the workspace had two modules named machine. It now lives in litellm-host beside the contract it implements, so core only holds routes. The OCR conversion from MachineFault moves to litellm-llms because the orphan rule no longer allows it in core
The payload boundary of callbacks-legacy gets a model-based proptest: for any
JSON body, caller keywords and callback edit, keywords the route sends unchanged
reach pre_call as the caller's own objects, and the wire is the body pre_call
received as the callback left it. A parametrized test pins that a keyword the
bridge never reads keeps its identity through setup, the deployment hook,
check_limits and prepare.
Behaviour owned by the real Logging object is pinned end to end in the OCR
tests: a hypothesis version of the body property over HTTP, sync hooks seeing no
running event loop, retained payloads staying intact after the call, success
callbacks sharing one standard logging payload, and state stashed before a
blocking deployment hook raises reaching both failure callback families
Every legacy callback call from callbacks-legacy now goes through one typed
Python shim, litellm.rust_bridge.legacy_callbacks, the only Python module
the crate reaches. Before, the crate called Logging methods, litellm.utils
hooks, the logging worker, the executor and several litellm globals
directly, and its tests retyped those signatures by hand, so an outdated
fake could accept a call the real code rejects. python_contract.json lists
each shim function's parameters: a Python test pins it to the real
signatures and a Rust test pins it to the Rust enum.
The lifecycle contract changes to match the Python @client wrapper:
- the driver emits CallEvent::Started before begin, so every host sees one
start time
- RequestContext carries the route-resolved api_key, so legacy pre_call and
post_call receive it, and post_call's additional_args match the Python OCR
path
- Passthrough and its re-aliasing are gone
- async deployment hooks always run, and the "no callbacks" shortcut that
skipped the logging payload is removed, as in the Python path
The OCR api_key is a SecretValue from the wire request onward, so Debug
output upstream of the callback contract cannot leak it.
host-python's RouteHost now classifies native failures once through
classify, and host ops return HostOpError. The OCR route host keeps main's
public errors by sending both through the existing Python map_failure.
The match statement in _incomplete_details_for_finish_reason tripped CodeQL's
mixed explicit and implicit returns alert (code-scanning 12640). A module-level
MappingProxyType keyed by finish reason gives the same three mappings with one
explicit return path
Rotating the Speech-to-Text stream at 240 s no longer ends the active turn:
the turn and its billed seconds continue on the new stream, forced at 280 s.
Bound the request and event queues (64 and 256) so a slow peer applies
backpressure instead of growing memory. Route a model to the Chirp realtime
path from its cost-map entry (mode audio_transcription plus /v1/realtime)
instead of a hardcoded name. Return on every branch of the recv and
transform helpers (CodeQL mixed returns), have the shared protocol helper take
the provider's error class so the Meta tests assert MuseProtocolError again,
and pin google-cloud-speech in the ci group so unit shards import it.
mcp_allowed_clients entries become {alias, value} objects: the value is what the JWT claim or header must equal, the alias is the name the dashboard and logs show. The Network Settings section is renamed Allowed Clients with one alias/value row per client
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Rename McpError/isError/inputSchema-style references to the SDK 2
spellings, parse the JSONRPCMessage union with a TypeAdapter, and drive
the SDK transports off httpx2 MockTransport injection where respx can no
longer intercept. Adjust for SDK 2 behavior: the initialize handshake
negotiates handshake-era protocol versions only, an empty SSE stream
surfaces CONNECTION_CLOSED, non-2xx tool responses surface INTERNAL_ERROR
MCPError instead of HTTPStatusError, and the SDK read timeout carries the
JSON-RPC REQUEST_TIMEOUT code.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
A deployment whose reasoning_effort is an object is copied through as
reasoning the way the HTTP mapper does it instead of being dropped, and
the relay re-asserts the response.create frame type after merging
extra_body so a type key inside it can never replace it. The lazy
OpenAPI snapshot goes back to main: the earlier regeneration came from a
Python 3.14 interpreter dedenting docstrings, which CI on 3.12 rejects
McpError -> MCPError (new code/message/data constructor), camelCase model
attributes and constructor kwargs -> snake_case, RequestResponder ->
ClientSession message handler receiving ServerNotification | Exception,
RequestContext -> ClientRequestContext, read_timeout_seconds -> float,
server_capabilities property, JSONRPCMessage union parsed via TypeAdapter,
and httpx -> httpx2 for every object handed to the SDK transports
(MCPSigV4Auth, the httpx client factory, outbound_credentials auth
classes and resolver return types). Helpers that serve both litellm httpx
clients and the SDK's httpx2 transport accept both response types.
The SDK read-timeout code is now the JSON-RPC REQUEST_TIMEOUT (-32001)
instead of HTTP 408; as_mcp_read_timeout keeps the TimeoutError context
discriminator. Upstream transport exceptions and responses found in
exception trees are matched as httpx2 alongside httpx.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Replace the bespoke dependency-install CI gate with a real migration:
require mcp>=2.2.0,<3 alongside httpx2>=2.5.0,<3 and pydantic>=2.12.0,<3
in the proxy and mcp extras, drop langchain-mcp-adapters (pins mcp<2)
from the dev group, and remove the dependency-install workflow and
tests/mcp_dependency_tests that only exercised the old pins.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
