Yassin Kortam cb3a7accdd fix(streaming): surface in-body error payloads on OpenAI-compatible streams (#32237) ... * fix(streaming): surface in-body error payloads on OpenAI-compatible streams vLLM and sglang return HTTP 200 streams whose SSE body carries the error, e.g. data: {"error": {"message": "...", "code": 400}}. The OpenAI-compatible chunk parser had no detection for this shape: since #23931 the payload parsed into an empty chunk (choices=[]) and the stream ended silently with 200, losing the provider's error and never attempting configured fallbacks. Detect the payload in OpenAIChatCompletionStreamingHandler.chunk_parser and raise OpenAIError with the upstream message and status code. The existing mid-stream gate then applies: 4xx surface directly to the client, 5xx wrap into MidStreamFallbackError so the router can run configured fallbacks. Fixes #25492 * fix(streaming): serialize messageless error payloads as JSON Address review feedback: an error dict without a message field now serializes via json.dumps instead of Python dict repr		2026-07-06 08:13:25 -07:00
..
a2a_protocol	fix(a2a): record agent cost_per_query and input tokens on native send path (#31979)	2026-07-03 09:25:42 +05:30
anthropic_interface	style: unify ruff format width on 120 (#31518)	2026-06-27 12:39:29 -07:00
assistants	style: unify ruff format width on 120 (#31518)	2026-06-27 12:39:29 -07:00
batch_completion	style: unify ruff format width on 120 (#31518)	2026-06-27 12:39:29 -07:00
batches	style: unify ruff format width on 120 (#31518)	2026-06-27 12:39:29 -07:00
caching	chore(lint): zero out crash-class pyright rules and ban new type: ignore comments (#32152)	2026-07-04 16:56:12 -07:00
completion_extras	style: unify ruff format width on 120 (#31518)	2026-06-27 12:39:29 -07:00
compression	style: unify ruff format width on 120 (#31518)	2026-06-27 12:39:29 -07:00
containers	style: unify ruff format width on 120 (#31518)	2026-06-27 12:39:29 -07:00
endpoints/speech/speech_to_completion_bridge	style: unify ruff format width on 120 (#31518)	2026-06-27 12:39:29 -07:00
evals	style: unify ruff format width on 120 (#31518)	2026-06-27 12:39:29 -07:00
experimental_mcp_client	feat(mcp): make token_exchange (OBO) production-ready - discovery threading + audit hardening + RFC 9728 challenge (#31622)	2026-07-03 17:12:25 -07:00
files	style: unify ruff format width on 120 (#31518)	2026-06-27 12:39:29 -07:00
fine_tuning	chore(lint): zero out crash-class pyright rules and ban new type: ignore comments (#32152)	2026-07-04 16:56:12 -07:00
google_genai	style: unify ruff format width on 120 (#31518)	2026-06-27 12:39:29 -07:00
images	style: unify ruff format width on 120 (#31518)	2026-06-27 12:39:29 -07:00
integrations	fix: merge websearch tool params (#32162)	2026-07-04 19:24:35 -07:00
interactions	style: unify ruff format width on 120 (#31518)	2026-06-27 12:39:29 -07:00
litellm_core_utils	feat(router): add separate ITPM/OTPM deployment rate limits (#31952)	2026-07-05 21:58:35 +05:30
llms	fix(streaming): surface in-body error payloads on OpenAI-compatible streams (#32237)	2026-07-06 08:13:25 -07:00
models	feat(proxy): add key-level budget_fallbacks to reroute requests when a per-model budget is exceeded (#31783)	2026-07-03 12:20:12 -07:00
ocr	style: unify ruff format width on 120 (#31518)	2026-06-27 12:39:29 -07:00
passthrough	style: unify ruff format width on 120 (#31518)	2026-06-27 12:39:29 -07:00
proxy	feat(router): add separate ITPM/OTPM deployment rate limits (#31952)	2026-07-05 21:58:35 +05:30
proxy_auth	style: unify ruff format width on 120 (#31518)	2026-06-27 12:39:29 -07:00
rag	style: unify ruff format width on 120 (#31518)	2026-06-27 12:39:29 -07:00
realtime_api	fix(realtime): preserve nested transcription model and session-first model priority	2026-07-03 14:21:22 -07:00
repositories	fix(proxy): count only active users toward license seat limit (#31227)	2026-06-29 18:01:02 -07:00
rerank_api	style: unify ruff format width on 120 (#31518)	2026-06-27 12:39:29 -07:00
responses	fix(responses/mcp): execute follow-up tool calls instead of dropping the stream	2026-07-03 21:28:45 -07:00
router_strategy	chore(lint): zero out crash-class pyright rules and ban new type: ignore comments (#32152)	2026-07-04 16:56:12 -07:00
router_utils	feat(router): add separate ITPM/OTPM deployment rate limits (#31952)	2026-07-05 21:58:35 +05:30
rust_bridge	feat(ocr): thin Rust OCR Python bridge (#31368)	2026-06-25 18:42:59 -07:00
sandbox	style: unify ruff format width on 120 (#31518)	2026-06-27 12:39:29 -07:00
search	style: unify ruff format width on 120 (#31518)	2026-06-27 12:39:29 -07:00
secret_managers	style: unify ruff format width on 120 (#31518)	2026-06-27 12:39:29 -07:00
skills	style: unify ruff format width on 120 (#31518)	2026-06-27 12:39:29 -07:00
types	feat(router): add separate ITPM/OTPM deployment rate limits (#31952)	2026-07-05 21:58:35 +05:30
vector_store_files	style: unify ruff format width on 120 (#31518)	2026-06-27 12:39:29 -07:00
vector_stores	style: unify ruff format width on 120 (#31518)	2026-06-27 12:39:29 -07:00
videos	style: unify ruff format width on 120 (#31518)	2026-06-27 12:39:29 -07:00
__init__.py	feat(tencent): add Tencent TokenHub as a provider (#31903)	2026-07-02 18:31:59 -07:00
_internal_context.py	fix(proxy): harden request parameter handling	2026-04-16 01:38:12 +00:00
_lazy_imports.py	style: unify ruff format width on 120 (#31518)	2026-06-27 12:39:29 -07:00
_lazy_imports_registry.py	feat(tencent): add Tencent TokenHub as a provider (#31903)	2026-07-02 18:31:59 -07:00
_logging.py	style: unify ruff format width on 120 (#31518)	2026-06-27 12:39:29 -07:00
_redis.py	fix(redis): re-establish async cluster connections after a node restart (#31577)	2026-06-30 12:25:15 -07:00
_redis_credential_provider.py	style: unify ruff format width on 120 (#31518)	2026-06-27 12:39:29 -07:00
_service_logger.py	style: unify ruff format width on 120 (#31518)	2026-06-27 12:39:29 -07:00
_uuid.py	build(deps-dev): bump black to 26.3.1 and apply formatting (#28525)	2026-05-21 17:24:18 -07:00
_version.py
anthropic_beta_headers_config.json	chore(oss): litellm oss staging 120626 (#30292)	2026-06-12 09:49:25 -07:00
anthropic_beta_headers_manager.py	style: unify ruff format width on 120 (#31518)	2026-06-27 12:39:29 -07:00
blog_posts.json	docs(blog): add WebRTC blog post link	2026-03-13 17:54:33 +05:30
budget_manager.py	style: unify ruff format width on 120 (#31518)	2026-06-27 12:39:29 -07:00
constants.py	feat(tencent): add Tencent TokenHub as a provider (#31903)	2026-07-02 18:31:59 -07:00
cost.json
cost_calculator.py	chore(lint): zero out crash-class pyright rules and ban new type: ignore comments (#32152)	2026-07-04 16:56:12 -07:00
exceptions.py	feat: litellm oss staging (#31935)	2026-07-03 09:27:31 +05:30
main.py	chore(lint): zero out crash-class pyright rules and ban new type: ignore comments (#32152)	2026-07-04 16:56:12 -07:00
model_prices_and_context_window_backup.json	feat(tencent): add Tencent TokenHub as a provider (#31903)	2026-07-02 18:31:59 -07:00
policy_templates_backup.json	feat(add-new-block_code_execution-guardrail): prevent agent from executing code (#22154)	2026-02-25 22:02:14 -08:00
provider_endpoints_support_backup.json	chore: litellm oss staging (#30968)	2026-06-23 07:31:44 -07:00
py.typed
router.py	feat(router): add separate ITPM/OTPM deployment rate limits (#31952)	2026-07-05 21:58:35 +05:30
scheduler.py	style: unify ruff format width on 120 (#31518)	2026-06-27 12:39:29 -07:00
setup_wizard.py	feat(anthropic): add Claude Sonnet 5	2026-06-30 18:47:08 +00:00
timeout.py	style: unify ruff format width on 120 (#31518)	2026-06-27 12:39:29 -07:00
utils.py	feat(tencent): add Tencent TokenHub as a provider (#31903)	2026-07-02 18:31:59 -07:00