mirror of
https://github.com/BerriAI/litellm.git
synced 2026-11-07 07:58:09 +00:00
032342d2b7
* fix(responses): resume background response streams with stream and starting_after
Retrieving a response with stream=true and starting_after=N now returns the
replayed event stream from OpenAI and Azure instead of one JSON object, on the
SDK (get_responses / aget_responses) and on the proxy's GET /v1/responses/{id}
Providers that cannot replay a stored stream drop the params under drop_params
and raise UnsupportedParamsError without it. Malformed values answer 400 with
the param named. A replay that reaches its terminal event settles the
background response's bill once and passes the provider's usage through
without estimating one. An empty replay (cursor at the end) ends normally
* fix(responses): encrypt replayed stream ids and resume Azure AI Foundry streams
A replayed background stream now gets the same response id encryption as a
created one, so the key that made the response can reuse the id. Azure AI
Foundry deployments resume streams like Azure ones, and the chat-bridged
Responses iterator closes its wrapped stream instead of raising.
* fix(responses): drop malformed resume params under drop_params and keep JSON answers to a streamed retrieve
A bad stream flag drops to a plain retrieve and a bad cursor to a full replay when drop_params is on, and a cursor without stream is ignored even when malformed. A streamed retrieve that the upstream answers with JSON returns the response object instead of an empty stream. get_responses forwards only the include keys from extra_query.
* fix(responses): give proxy stream resumes the deployment stream_timeout and document retrieve query params
* fix(responses): keep deployment stream off retrieves and reject oversized cursors
* fix(responses): keep router default stream off plain retrieves
* test(responses): black box coverage for resume refusals, drop_params, stream timeout and billing
* test(responses): replay a real stream in the billing test and type the fake wrapper chunks
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
|
||
|---|---|---|
| .. | ||
| html_forms | ||
| __init__.py | ||
| test_admin_ui_utils.py | ||
| test_auth_cache_invalidation_pubsub.py | ||
| test_cache_aware_routing.py | ||
| test_cache_codec.py | ||
| test_callback_config_validation.py | ||
| test_callback_utils.py | ||
| test_check_batch_cost.py | ||
| test_check_responses_cost.py | ||
| test_codex_model_catalog.py | ||
| test_config_sync_pubsub.py | ||
| test_credential_hydration.py | ||
| test_custom_openapi_spec.py | ||
| test_debug_utils.py | ||
| test_discoverable_model_filter.py | ||
| test_encrypt_decrypt_utils.py | ||
| test_error_body_call_id.py | ||
| test_expired_ui_session_key_cleanup_manager.py | ||
| test_get_routes.py | ||
| test_http_parsing_utils.py | ||
| test_json_merge_patch.py | ||
| test_key_rotation_e2e.py | ||
| test_key_rotation_integration.py | ||
| test_key_rotation_lock.py | ||
| test_key_rotation_manager.py | ||
| test_load_config_utils.py | ||
| test_model_deprecation.py | ||
| test_model_listing_utils.py | ||
| test_openai_endpoint_utils.py | ||
| test_openai_error_payload.py | ||
| test_openapi_route.py | ||
| test_openapi_schema_compat.py | ||
| test_periodic_reload_schedule.py | ||
| test_proxy_encrypt_decrypt.py | ||
| test_rbac_utils.py | ||
| test_realtime_cache.py | ||
| test_registry_read_through.py | ||
| test_request_tag_mappings.py | ||
| test_reset_budget_job.py | ||
| test_resource_ownership.py | ||
| test_scheduled_job_stagger.py | ||
| test_semantic_text_index.py | ||
| test_slow_gc_logger.py | ||
| test_sse_keepalive.py | ||
| test_static_asset_utils.py | ||
| test_swagger_utils.py | ||
| test_timezone_utils.py | ||
| test_token_count_log.py | ||
| test_token_count_settings.py | ||
| test_upsert_budget_membership.py | ||
| test_user_api_key_cache.py | ||
| test_validation_error_body.py | ||