The streaming pipeline step only takes a post-call hook on routes whose
translation assembles the streamed response (chat completions, Responses,
Messages). On /v1/completions, the Gemini streamGenerateContent route, and
A2A streams the pipeline is skipped with the merge-base warning and the hook
runs on its own afterwards, instead of getting a None response while the
header says the guardrail ran. A guardrail that overrides
async_post_call_streaming_iterator_hook next to its post-call hook keeps its
native per-chunk path rather than running buffered through the adapter
Reasoning items the Responses API returned for a /v1/messages turn were rebuilt
from their summary text on every replay, so the prompt the model saw changed
between turns and the prompt cache never matched. The bridge now asks for
reasoning.encrypted_content, carries it in the thinking signature (or as a
redacted_thinking block when there is no summary), and replays it verbatim as
the reasoning item's encrypted_content. Anthropic replay paths drop those
tagged blocks so a cross-model resume never forwards OpenAI bytes to Anthropic
Replace the thread semaphore around HuggingFace encodes with an anyio CapacityLimiter
applied at every offloaded count site through offload_token_count, so waiting counts no
longer hold slots in the shared 40-thread pool and inline counts on the event loop never
block on the bound. Rename TOKEN_COUNTER_MAX_CONCURRENT_HF_ENCODES to
TOKEN_COUNTER_MAX_CONCURRENT_COUNTS
get_optional_params_image_gen only forwarded the per-call drop_params flag
to provider configs, so litellm_settings drop_params: true never dropped
the n the MAI generations endpoint ignores. The MAI edits config also
advertised and forwarded size, which that endpoint ignores. Non-numeric
and non-positive n now surface as a 400 instead of a 500 or a pass-through.
Every test this PR adds now annotates its fixture and parametrize
parameters. The submit-time warning for a tag-matched deferred policy
names only the policies, since a wildcard attachment pattern would let
caller-provided tag text reach the log.
A guardrail that removes or adds a tool call on an ended stream used to be
silently ignored: every handler substitutes the original list on a count
mismatch and the executor skipped its observer once the translation could
deliver rewrites. The executor now tracks the count change on the observer
and releases the original chunks with the discard warning on every
translation, matching what the merge base did for any tool call rewrite
* test(mcp): exercise /mcp/proxy authorization against the real registry instead of patched manager methods
* fix(mcp): preserve proxy logging and authorization coverage
* test(mcp): respect the proxy FastAPI import boundary
The bridges derived prompt_cache_key as the first 64 chars of metadata.user_id.
Claude Code packs a JSON object into that field whose prefix is the per-install
device_id, so every session and subagent on one machine shared a single key,
and a plain end-user id pinned all of that user's conversations to one slot.
Parse the JSON and use session_id; send no key otherwise so the provider falls
back to its own prompt-prefix hashing. An explicit prompt_cache_key still wins.
Fixes#39145
* fix(otel v2): restore the Datadog auth span and the last-wins callback merge
Move @tracer.wrap() back onto user_api_key_auth so USE_DDTRACE=true emits the
auth span again, and let a failure entry's callback_vars take part in the
destination merge so the resolver picks the same account the runtime parser does
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(otel v2): drop docstrings from the two regression tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* ci: rerun proxy-infra after the flaky test_check_migration process-tree test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
A translation that hands the hook no assembled response leaves the stream
as it is, and a rewrite the translation cannot rescan is released as the
original stream with a warning.
