The request crossed the boundary four times: Python bind_request into a
dataclass, Rust re-reading each field back with its own precedence rules,
a BridgeOcrRequest copy, then the core request. The dataclass was retained
for the whole call only so map_failure could read model and kwargs at the
end.
Add a route-agnostic Signature binder that reproduces Python's positional
and keyword binding rules and TypeError messages, and project straight
from the bound arguments into the core request. This removes the Python
LiteLLMOcrRequest dataclass, bind_request, PythonOcrInput, BridgeOcrRequest,
and the retained boundary_request, document, api_key and callback_inputs
Python objects. api_key for pre_call now comes from core's resolved
connection, matching the legacy handler, and core no longer carries
retained_fields or retains_document since the bridge stopped re-aliasing.
The bridge kept a Rust-side shadow of Logging's callback registries and
elided pre_call/post_call/success_handler when it thought nothing was
listening. That forked the logging contract and hid a bug: with callbacks
present, the during-call hook re-inserted the caller's original document
and unmapped optional params over the provider-transformed body.
Call the real Logging handlers unconditionally, like the Python wrapper
does, and hand callbacks a fresh dict built from core's composed body
instead of re-aliasing caller objects into it.
Trimming the streamed buffer to the retained tail could drop a category
exception phrase that suppresses a later keyword, or the identifier word
of an unfinished sentence that a conditional category pairs with a later
block word. Refuse the cut while either would leave the buffer so the
bounded scan masks and blocks exactly like a scan of the full text
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The streaming post-call hook rescanned the whole accumulated choice buffer on every chunk, so scan cost grew quadratically with output length. Keep a bounded per-choice buffer instead: once it exceeds twice the scan context, drop the head when masking the head and tail separately yields the same output as masking the whole buffer, so no pattern, phrase or exception straddles the cut. Detections from the dropped head are kept and merged, deduplicated, into the final log row
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The first cache-enabled litellm-e2e build (211) showed three gaps in the shared provider cache:
Every OpenAI response carries Cloudflare bot-management Set-Cookie headers, and the capture rejected any response with Set-Cookie, so no OpenAI response was ever recorded (179 of 372 misses rejected). The edge already withholds Set-Cookie from the proxy, so drop it before validating and storing instead of rejecting.
The provider prompt-caching tests need fresh provider state: a replayed priming response reports cache creation rather than a cache read, and the TPM test then trips the key limit. Mark both modules provider_live.
TestApiBaseSeam::test_live_mode_returns_none ran inside the cache-enabled runner and saw the shared edge; isolate it from E2E_PROVIDER_CACHE.
The reset job moved from zeroing spend to an atomic decrement of the amount
it cleared, so every batched write now carries {"decrement": <cleared>}
instead of 0. Four tests still pinned 0 and had been failing since, which
also meant they no longer checked the amount at all. Assert the decrement
equals each row's own pre-reset spend, so a wrong amount fails the test.
