mirror of
https://github.com/BerriAI/litellm.git
synced 2026-11-07 07:58:09 +00:00
bc1309f18e
* fix(responses): map streamed flex_unavailable errors to 429 not 500 When OpenAI has no flex capacity, a service_tier flex request fails with type resource_unavailable / code flex_unavailable. Non-streamed, this arrives as HTTP 429 and litellm raises RateLimitError. Streamed /v1/responses delivers the same error as an in-stream event, and neither field was in _ERROR_CODE_HTTP_STATUS, so it defaulted to 500 InternalServerError. The HTTP status then depended only on stream: one provider outage looked like a 429 or a 500. The 500 reads as a server error in metrics and alerting, and callers that retry on 429 do not retry it. Map both resource_unavailable and flex_unavailable to 429 so the streamed path matches the non-streamed path. Fixes #49736 * test(responses): cover streamed flex outage end to end for error and response.failed events Streams the flex_unavailable payload from #49736 through litellm.aresponses with an injected client and asserts a 429 RateLimitError for both in-stream event kinds, matching the non-streamed path * test(responses): drop explanatory comment from flex status mapping test * test(integration): cover streamed flex outage status on responses and chat bridge Drives the flex_unavailable payload from #49736 through a real proxy as an in-stream error event and as response.failed. The native /v1/responses stream fails with rate_limit_exceeded and logs 429, and the chat bridge answers a 429 throttling error, where both reported a 500 before the mapping fix --------- Co-authored-by: BinYi LIU <bliu1@berkeley.edu> |
||
|---|---|---|
| .. | ||
| faked_responses_case.py | ||
| test_assistants_evals_stream.py | ||
| test_azure_streaming_function_calls.py | ||
| test_bedrock_guardrail_background_stream_scan.py | ||
| test_chat_stream_guardrail_error_events.py | ||
| test_decision_model_buffering.py | ||
| test_decision_model_pipeline_buffering.py | ||
| test_fake_stream_reasoning_wire.py | ||
| test_faked_responses_stream_bridges.py | ||
| test_faked_responses_stream_cache_replay.py | ||
| test_faked_responses_stream_chaos.py | ||
| test_faked_responses_stream_guardrail_blocks.py | ||
| test_faked_responses_stream_items.py | ||
| test_file_content_streaming.py | ||
| test_google_genai_stream_usage_settings_wire.py | ||
| test_interception_converted_stream_options.py | ||
| test_mid_stream_fallback_continuation.py | ||
| test_noma_v2_streaming_settings.py | ||
| test_openai_like_stream_usage_wire.py | ||
| test_openai_sdk_plain_chunk_fields.py | ||
| test_openai_stream_passthrough_fields.py | ||
| test_responses_bridge_stream_chaos.py | ||
| test_responses_bridge_stream_errors.py | ||
| test_responses_top_level_error_events.py | ||
| test_stream_clean_eof_without_finish_reason.py | ||
| test_stream_contracts.py | ||
| test_stream_open_error_provider_headers_proxy.py | ||
| test_stream_parallel_slot_release.py | ||
| test_text_completion_stream_guardrail_rebuild.py | ||
| test_ttft_keepalive.py | ||