mirror of
https://github.com/BerriAI/litellm.git
synced 2026-11-14 09:03:16 +00:00
534a60d28d
* fix(proxy): name the deployment in stream error alerts on every provider and route The llm_exceptions alert for a cut stream on Anthropic chat, /v1/completions, native /v1/responses and native /v1/messages had no API Base, model_group or deployment line. The shared exception mapper now attaches its debug info at its single exit, mid-stream fallback errors inherit it from the error they wrap, and the native relays add it to the same error object they re-raise. * fix(proxy): keep the prompt out of attached alert debug info and name the deployment on more stream exits * fix(proxy): name the deployment on stream errors that arrive already mapped exception_type returned errors that were already litellm types before it attached the debug info, so a finish_reason=abort chunk or the max streaming duration timeout still sent an alert with no deployment. It now builds the info first and attaches the prompt-free copy on that early return too, and native Responses streams attach it on their generic error exit Also adds tests for the dropped `Messages: None` line and the mapping-crash exit * fix(proxy): read the router's deployment from litellm_metadata on Messages and Responses errors /v1/messages and /v1/responses keep the router's model_group, deployment and api_base in litellm_metadata, so their errors before the stream named only the model in alerts. The debug info now reads litellm_metadata first, falls back to metadata, and takes API Base from it when the call passes none get_first_chars_messages returns an empty string again for input that is not a mapping, as it did before this branch * fix(proxy): merge both metadata buckets when error alerts name the deployment A pre_call guardrail adds litellm_metadata to a chat request while the router keeps model_group and deployment in metadata, and generic routes such as moderations keep the key alias and team in metadata and the deployment in litellm_metadata. Reading only one bucket dropped lines the merge base showed. The debug info now merges both, and litellm_metadata wins when both carry the same key, so a caller's own metadata cannot replace the router's values * fix(proxy): keep a reused error's alert details on the request that raised it An exception instance raised for more than one request, such as an SDK mock_response exception or a handler that raises one shared error, kept the first request's Key Name, model_group and deployment in every later alert The attached details now record the call id that set them. A later call with a different id replaces them, and a layer that re-maps the error without the request keeps them * fix(proxy): drop alert details from an error object raised by more than one request One exception object raised by two concurrent requests (a shared mock_response error, or a handler that raises one error object for every request) let a request's alert show the other request's Key Name and deployment, since the later attach replaced the earlier one. An error object tagged by one call id and attached again under another now drops its details for good, so no alert names the wrong request. One request re-mapping its own error, with or without a call id, keeps the lines it already has --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| __init__.py | ||
| test_streaming_iterator.py | ||