mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-30 06:43:27 +00:00
032342d2b7
* fix(responses): resume background response streams with stream and starting_after
Retrieving a response with stream=true and starting_after=N now returns the
replayed event stream from OpenAI and Azure instead of one JSON object, on the
SDK (get_responses / aget_responses) and on the proxy's GET /v1/responses/{id}
Providers that cannot replay a stored stream drop the params under drop_params
and raise UnsupportedParamsError without it. Malformed values answer 400 with
the param named. A replay that reaches its terminal event settles the
background response's bill once and passes the provider's usage through
without estimating one. An empty replay (cursor at the end) ends normally
* fix(responses): encrypt replayed stream ids and resume Azure AI Foundry streams
A replayed background stream now gets the same response id encryption as a
created one, so the key that made the response can reuse the id. Azure AI
Foundry deployments resume streams like Azure ones, and the chat-bridged
Responses iterator closes its wrapped stream instead of raising.
* fix(responses): drop malformed resume params under drop_params and keep JSON answers to a streamed retrieve
A bad stream flag drops to a plain retrieve and a bad cursor to a full replay when drop_params is on, and a cursor without stream is ignored even when malformed. A streamed retrieve that the upstream answers with JSON returns the response object instead of an empty stream. get_responses forwards only the include keys from extra_query.
* fix(responses): give proxy stream resumes the deployment stream_timeout and document retrieve query params
* fix(responses): keep deployment stream off retrieves and reject oversized cursors
* fix(responses): keep router default stream off plain retrieves
* test(responses): black box coverage for resume refusals, drop_params, stream timeout and billing
* test(responses): replay a real stream in the billing test and type the fake wrapper chunks
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
|
||
|---|---|---|
| .. | ||
| cloudformation_stack | ||
| dist | ||
| enterprise_hooks | ||
| enterprise_ui | ||
| litellm_enterprise | ||
| __init__.py | ||
| LICENSE.md | ||
| pyproject.toml | ||
| README.md | ||
LiteLLM Enterprise
Code in this folder is licensed under a commercial license. Please review the LICENSE file within the /enterprise folder
These features are covered under the LiteLLM Enterprise contract
👉 Using in an Enterprise / Need specific features ? Meet with us here
See all Enterprise Features here 👉 Docs