The transport now has every data command Python sends, request-scoped and
explicit batches on the one pipeline engine RedisBatch already used,
MULTI/EXEC batches, pub/sub, and the admin and health commands. RedisBatch
declares onto the same engine, so moved and unmoved callers in one request
still share one round trip. Cluster mget goes through the cache's
mget_nonatomic path instead of a cross-slot MGET.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Rework RedisTransport into plain get, mget, set, delete and eval methods,
split the breaker step (run_under_breaker over the existing
_run_under_circuit_breaker) from the events step (RedisServiceEvents.observe,
which skips the stack walk and event when no service_callback is set), and
add eval with EVALSHA plus EVAL fallback. Fold in the store layer and the
response polling store, and replace the redis-access skill references with
the migration plan and PR sketches.
* fix(proxy): end a chat completions stream with an error event when a guardrail blocks mid-stream
A streaming guardrail that raised an HTTPException after the first chunk re-raised it inside the SSE generator, so the 200 body was cut off with no finish_reason, no error event and no [DONE]. Build the error frame through sse_error_payload instead, keeping the status, message and structured detail, and encode values JSON cannot represent as strings so the frame always goes out.
* fix(proxy): keep early guardrail blocks on their HTTP status and end mid-stream errors with [DONE]
A guardrail block before the first chunk is raised again, so its response stays the
400 event-stream body it is on main. A mid-stream error event now carries the failure
hook's replacement error when one is returned, is followed by data: [DONE] on
OpenAI-format streams, and leaves the Gemini route's raise unchanged. Error details
JSON cannot encode are written as strings instead of breaking the event.
* fix(proxy): keep a per-chunk guardrail block on the first chunk on its event-stream 400
* fix(proxy): treat a keepalive ping as a started stream so a later guardrail block ends in-band
* fix(proxy): encode non-JSON guardrail details in early stream error frames
An early guardrail block whose detail holds a value JSON cannot encode sent
400 headers and then cut the body. The error frame now writes such a value as
its string form, the same as the mid-stream event. The keepalive test now runs
on a virtual clock instead of real sleeps.
* fix(proxy): show an in-band stream error event as an error in the CLI, Python client, and dashboard playgrounds
ChatClient.completions_stream raises StreamError on a data frame carrying an error, so litellm-proxy chat prints the block reason and keeps the partial reply out of history. The prompt playground and the Interactions playground read the same frame through a shared streamErrorMessage helper. Adds the audit's integration cells for chat stream error events, and makes the endpoint tests' failure-hook mocks return None so they check the real error payload.
* fix(proxy): print stream error text verbatim in the CLI and read split error events in the prompt playground
The CLI printed the server's error message as Rich markup, so a closing tag like [/pii] in a guardrail message crashed lite chat. The prompt playground now buffers partial lines across reads, ChatClient raises only on a non-empty error, StreamError redacts keys like the other client exceptions, and StreamError is exported from litellm.proxy.client
* fix(proxy): print CLI stream errors without Rich highlighting or emoji codes
* test(proxy): cover the CLI's plain-text 400 error print
* test(proxy): annotate the new client stream test locals as Final
* fix(proxy): send the failure hook replacement on Responses streams and before the first chunk
* fix(proxy): end a Responses stream with response.failed when it fails on a built-in exception
* fix(proxy): keep a failure hook replacement before the first chunk in-band so bare stream routes still answer it
A non-HTTP failure whose hook returns an HTTP error was raised before the first
chunk, which create_response turned into the hook's status but which a bare
StreamingResponse route such as /queue/chat/completions answered as a 200 with
zero bytes. Only the original HTTP errors raise now. The replacement goes out as
the stream's error event, which create_response still answers with the hook's
status as a JSON error.
Adds audit round 2's integration cells for failure hook replacements on chat and
Responses streams, built-in exceptions on Responses streams, and the Python client.
* test(proxy): cover a queue stream whose failure hook replaces a failure before the first chunk
The audit cell posts to /queue/chat/completions with a callback that raises before the first chunk and a failure hook that answers 403. It checks for a 200 event stream whose only event is the replacement, then one [DONE], with the callback's own message absent
* fix(params): validate stream_chunk_size once and carry it as typed control options
Checks stream_chunk_size at the top of completion() and acompletion(), accepts
digit strings, returns a 400 naming the param unless drop_params is set, and
stores the checked value under _litellm_control. Bedrock Converse and Invoke
read it from litellm_params; the Bedrock-only checker and the dead Invoke pops
are gone. Owned-kwarg filtering now runs through one helper everywhere.
Refs LIT-8317
* test(bedrock): drop tests for the removed stream_chunk_size_from helper
Refs LIT-8317
* fix(params): check stream_chunk_size before the MCP gateway branch
Refs LIT-8317
* fix(params): return assert_never in the exhaustive control-options match
Refs LIT-8317
* fix(params): address council review of the control options change
Read all_litellm_params live so names registered after import stay
LiteLLM-owned, make litellm_params a required keyword on the stream
wrapper hooks, give digit strings and ints the same 18-digit range,
share the default-chunking test table, test the Responses bridge through
litellm.responses, and revert formatting-only churn in existing tests.
Refs LIT-8317
* fix(params): address the second council review of control options
Keep the Responses bridge on its original all_litellm_params forwarding,
narrow _int_from_decimal_string inline so it type-checks, bound nested
huge ints in the error message, store _litellm_control only when a value
is set, simplify the parser to its single field, drop the one-caller
wrapper, and tighten the tests.
Refs LIT-8317
* fix(params): keep the 18-digit length check on stream_chunk_size strings
A 19-character string with leading zeros such as 0000000000000000001 would
otherwise pass as 1, although the rule and the error message say at most
18 digits.
Refs LIT-8317
* test(params): tidy control options tests after council sign-off
Move the Responses bridge test into the existing bridge test file, drop the
rebind test that pinned an implementation detail, assert through
stored_control_options instead of the storage key, and cover
drop_params="true" through Bedrock streaming.
Refs LIT-8317
* test(params): wrap a chunking test row that went past 120 characters
Refs LIT-8317
* fix(params): carry stream_chunk_size through litellm_params instead of provider params
* test(integration): fence stream_chunk_size out of every provider request body
* test(bedrock): type parametrized stream chunk test params
* test(integration): drop the contracts manifest resurrected by the main merge
* test(bedrock): type the stream_chunk_size test helpers
* test(params): finish AGENTS.md typing pass on stream_chunk_size tests
* test(integration): drop the covers marker from the stream_chunk_size wire test
---------
Co-authored-by: shrey kharbanda <shreshth@berri.ai>
