mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-25 05:54:53 +00:00
ce4a6cbf5b
* perf(streaming): skip full-chunk dump and delta rebuild on openai-compatible chunks Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * perf(streaming): reuse the built choice and delta on plain openai-compatible chunks Co-authored-by: yassin <yassin@berri.ai> Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(streaming): copy upstream chunk attributes in upstream key order The scalar fast path iterated a frozenset of keys, so the SSE key order of chunk extras like service_tier and obfuscation changed with each worker's hash seed * test(streaming): count chunk_creator calls with a profiler instead of patching classes * test(streaming): check service_tier still passes through chunk_creator * fix(streaming): reuse the built choice for chunks with upstream choice extras * test(streaming): cover ModelResponseStream chunks on the choice reuse path * fix(streaming): keep main's delta rebuild for deltas with nested values Clearing role in place on a delta that holds tool calls left its nested fields unset, so chunk.model_dump() dropped keys like a custom tool call's name on later chunks. Deltas holding only scalars keep the in-place path. * perf(streaming): build each plain OpenAI chunk's delta once and skip pydantic attribute misses Later plain ChatCompletionChunks now build one Delta and one StreamingChoices, with no full-chunk model_dump, and read stored values instead of attribute probes that raise and catch AttributeError. Nested-value chunks no longer pay the dict(model) misses that made them slower than the merge base. * fix(streaming): read hidden params a chunk carries as a pydantic extra * test(streaming): pin logprobs, Azure filter extras, and audio and image deltas on later OpenAI chunks * test(streaming): cover role, reasoning, choice extras and billing on plain OpenAI SDK chunks --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: mateo <mateo@berri.ai> Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| faked_responses_case.py | ||
| test_assistants_evals_stream.py | ||
| test_azure_streaming_function_calls.py | ||
| test_chat_stream_guardrail_error_events.py | ||
| test_decision_model_buffering.py | ||
| test_decision_model_pipeline_buffering.py | ||
| test_fake_stream_reasoning_wire.py | ||
| test_faked_responses_stream_bridges.py | ||
| test_faked_responses_stream_cache_replay.py | ||
| test_faked_responses_stream_chaos.py | ||
| test_faked_responses_stream_guardrail_blocks.py | ||
| test_faked_responses_stream_items.py | ||
| test_file_content_streaming.py | ||
| test_google_genai_stream_usage_settings_wire.py | ||
| test_interception_converted_stream_options.py | ||
| test_mid_stream_fallback_continuation.py | ||
| test_noma_v2_streaming_settings.py | ||
| test_openai_like_stream_usage_wire.py | ||
| test_openai_sdk_plain_chunk_fields.py | ||
| test_openai_stream_passthrough_fields.py | ||
| test_responses_bridge_stream_chaos.py | ||
| test_responses_bridge_stream_errors.py | ||
| test_responses_top_level_error_events.py | ||
| test_stream_contracts.py | ||
| test_stream_open_error_provider_headers_proxy.py | ||
| test_stream_parallel_slot_release.py | ||
| test_ttft_keepalive.py | ||