* fix(proxy): end a chat completions stream with an error event when a guardrail blocks mid-stream
A streaming guardrail that raised an HTTPException after the first chunk re-raised it inside the SSE generator, so the 200 body was cut off with no finish_reason, no error event and no [DONE]. Build the error frame through sse_error_payload instead, keeping the status, message and structured detail, and encode values JSON cannot represent as strings so the frame always goes out.
* fix(proxy): keep early guardrail blocks on their HTTP status and end mid-stream errors with [DONE]
A guardrail block before the first chunk is raised again, so its response stays the
400 event-stream body it is on main. A mid-stream error event now carries the failure
hook's replacement error when one is returned, is followed by data: [DONE] on
OpenAI-format streams, and leaves the Gemini route's raise unchanged. Error details
JSON cannot encode are written as strings instead of breaking the event.
* fix(proxy): keep a per-chunk guardrail block on the first chunk on its event-stream 400
* fix(proxy): treat a keepalive ping as a started stream so a later guardrail block ends in-band
* fix(proxy): encode non-JSON guardrail details in early stream error frames
An early guardrail block whose detail holds a value JSON cannot encode sent
400 headers and then cut the body. The error frame now writes such a value as
its string form, the same as the mid-stream event. The keepalive test now runs
on a virtual clock instead of real sleeps.
* fix(proxy): show an in-band stream error event as an error in the CLI, Python client, and dashboard playgrounds
ChatClient.completions_stream raises StreamError on a data frame carrying an error, so litellm-proxy chat prints the block reason and keeps the partial reply out of history. The prompt playground and the Interactions playground read the same frame through a shared streamErrorMessage helper. Adds the audit's integration cells for chat stream error events, and makes the endpoint tests' failure-hook mocks return None so they check the real error payload.
* fix(proxy): print stream error text verbatim in the CLI and read split error events in the prompt playground
The CLI printed the server's error message as Rich markup, so a closing tag like [/pii] in a guardrail message crashed lite chat. The prompt playground now buffers partial lines across reads, ChatClient raises only on a non-empty error, StreamError redacts keys like the other client exceptions, and StreamError is exported from litellm.proxy.client
* fix(proxy): print CLI stream errors without Rich highlighting or emoji codes
* test(proxy): cover the CLI's plain-text 400 error print
* test(proxy): annotate the new client stream test locals as Final
* fix(proxy): send the failure hook replacement on Responses streams and before the first chunk
* fix(proxy): end a Responses stream with response.failed when it fails on a built-in exception
* fix(proxy): keep a failure hook replacement before the first chunk in-band so bare stream routes still answer it
A non-HTTP failure whose hook returns an HTTP error was raised before the first
chunk, which create_response turned into the hook's status but which a bare
StreamingResponse route such as /queue/chat/completions answered as a 200 with
zero bytes. Only the original HTTP errors raise now. The replacement goes out as
the stream's error event, which create_response still answers with the hook's
status as a JSON error.
Adds audit round 2's integration cells for failure hook replacements on chat and
Responses streams, built-in exceptions on Responses streams, and the Python client.
* test(proxy): cover a queue stream whose failure hook replaces a failure before the first chunk
The audit cell posts to /queue/chat/completions with a callback that raises before the first chunk and a failure hook that answers 403. It checks for a 200 event stream whose only event is the replacement, then one [DONE], with the callback's own message absent
* fix(proxy): streaming responses hand the event loop back every 50 ms instead of holding it for the whole buffered stream
* fix(proxy): yield to the event loop on every streamed chunk, no hold budget or clock
* docs(proxy): say why the streaming loop yields on every chunk
* fix(proxy): streaming loop yields once it has held the event loop 50 ms, clock read inline
* docs(proxy): streaming loop comment names the 50 ms hold
* Revert "docs(proxy): streaming loop comment names the 50 ms hold"
This reverts commit cbf1371873.
* refactor(proxy): streaming loop hold budget is _STREAM_LOOP_HOLD_MS = 50
* Revert "refactor(proxy): streaming loop hold budget is _STREAM_LOOP_HOLD_MS = 50"
This reverts commit 356454def6.
* docs(proxy): streaming loop comment notes the yield is a floor, not a count of releases
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(proxy): expose public names for private proxy helpers
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): keep original class names behind public aliases
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): preserve internal callback filtering
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): keep _PROXY_ class names for managed files hooks
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): keep old private names in package exports
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): preserve recursive auth helper name
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): align MCP limiter tests with server enforcement
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): match main's MCP limiter tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): keep old private names bound in importing modules
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): preserve compatibility imports through strict lint
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): use exact pyright suppression in password helper test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(proxy): add reasons to compatibility import noqa comments
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): update IN-list baseline for renamed helpers
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): stamp the client alias on a copy of each streamed chunk so pricing sees the deployment model
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(spend): streamed alias matching a capability rule bills the deployment price
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(spend): assert every streamed chunk carries the client alias
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(logging): log the client alias on the priced streamed response, the same as non-streamed
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): move proxy_server, _experimental and db tests into tests/unit/proxy
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): keep tuple identity in proxy state restore and fix misc target paths
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yuneng <yuneng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-01 18:14:24 +00:00
Renamed from tests/test_litellm/proxy/proxy_server/test_streaming_helpers.py (Browse further)
