A converted-stream request whose cache entry is a plain (non-stream) object is
replayed as that plain object, so nothing later fires the success callbacks.
Decide deferral from the replayed result's type instead of the request kwargs.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
A deployment hook (Headroom, code interpreter, web search) can downgrade
kwargs["stream"] to False while the caller still expects to iterate the
result. The cache handler keyed stream replay and callback deferral off
the raw flag, so a cache hit returned a plain object to a caller that
iterates, and the Responses iterator never persisted the converted
stream in the first place. Key both off the conversion marker as well
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Deployment hooks such as Headroom downgrade stream=True to a non-streaming provider call and the agentic loop then hands back a CustomStreamWrapper (or MockResponsesAPIStreamingIterator for Responses). wrapper_async still saw kwargs["stream"] is False, so it took the non-streaming success path with a lazy stream object: no standard_logging_object was built, the proxy cost callback raised failed_tracking_spend, and the wrapper's own end-of-stream dispatch was deduped away. Treat a lazy stream result as streaming for logging regardless of the downgraded kwarg. Regression in v1.99.0 via #35017
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
httpx.Headers.items() comma-joins repeated header names, so the rebuilt
response iterates multi_items() and keeps every value, matching what the
raw openai client exposes on e.response.headers
The streaming bridge restored the namespace before deciding whether a tool call was a custom tool, so a namespaced function sharing a short name with a nested custom tool streamed back as a custom_tool_call. Classify on the raw chat tool name first, the way the non-streaming path already does.
The guardrail merge only stripped the namespace prefix and grammar suffix from the ends of the edited description, so a guardrail appending text after the grammar block left the block in the member description and the chat conversion appended it a second time. Strip the first occurrence of each instead.
The proxy edge test file no longer carries any test of this change, and
the remaining diff was the scoped format gate reflowing the whole file to
the 120 limit, so it goes back to the merge base bytes
A mapped litellm_proxy exception now attaches an httpx.Response that
carries the proxy's response headers whenever the handler attached a
header-less synthetic one, on every status branch and on the relay
path. BadRequestError keeps its base-class contract: .headers stays the
proxy-supplied channel, so the proxy edge keeps forwarding an upstream
proxy's headers under the llm_provider- prefix and the date and server
edge change is no longer needed.
Only image-only unscoped content stays unrecorded; text or tool content
removed by scoping is recorded as not_run even when an image sits beside it.
Also keeps the type-discipline budget flat by returning the reason from the
helper and annotating the accumulator lists _extract_inputs requires.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The one-hour cache write fallback now takes the applied cache write rate, so an off-peak write price carries into it instead of the input rate
The cost estimate test for a cost-map model without cache prices now expects writes at the input rate, which is what the proxy bills
The recording logger in the deferred guardrail test types its callback parameters
A chat row whose content carried an empty text part counted two slots in the
chat completions handler while Prompt Security read one text out of the
modified row, so the structured rewrite was dropped and the request got the
named rejection. One shared helper now lists a row's slot texts and both the
slot count and the modified-row reader use it.
With a post-call guardrail the proxy defers async success logging, and every nested wrapper on a
/v1/messages call bridged to the Responses API overwrote the stored closure, so the spend log was
built from the outermost Anthropic-shaped reply under Responses semantics and recorded the prompt
tokens without the cache hit. The first wrapper to exit now keeps the slot, which is the innermost
provider response, the same one the non-deferred path logs.
The flat cost path also billed cache-creation tokens at 0 when the model had no
cache_creation_input_token_cost. It now falls back to the input rate, and the 1h rate to the
creation rate, matching the tiered path and the custom pricing helper.
The three guardrail translation handlers imported the exception from the proxy policy engine through a function-local import, which CodeQL flagged as a cyclic import. The exception and its helper now live next to the handlers in the shared guardrail translation utils, and the tests import it from there.
The Prompt Security modify-mode helper is also restructured into early-return TypedDict displays so the LIT002 budget stays at its limit
