scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-30 06:43:27 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 250
litellm/tests/unit/responses
History
Kerry Lu 22556f15bf fix(proxy): bill partial Responses API streams when the client disconnects (#47611) 
* fix(proxy): bill partial Responses API streams when the client disconnects

Streamed /v1/responses requests and the default-route OpenAI and Azure /v1/messages requests behind them logged a 499 with zero spend when the client went away mid-stream, while /v1/chat/completions already billed the chunks it had sent. The Responses API iterator now keeps the response.created payload and can hand the disconnect cleanup a response.incomplete event with counted usage and the served service_tier, and the messages-route byte stream and router wrappers delegate to it.

* fix(proxy): satisfy lint gates for partial Responses API disconnect billing

Make the disconnect helper public so the proxy, router and messages adapter can import it without reportPrivateUsage, type the SSE byte stream as an AsyncGenerator so aclose is known, read litellm_call_id off the logging object, and keep the new tests on Final locals with the recorder injected through dynamic_async_success_callbacks instead of litellm.callbacks.

* fix(router): bill the primary stream's tokens after a mid-stream hop and chat-bridge Responses streams on disconnect

The router's fallback wrapper now folds the primary Responses stream's relayed tokens into the fallback's disconnect event the way _combine_responses_fallback_usage does for a natural end of stream, and LiteLLMCompletionStreamingIterator (chat-only providers served through /v1/responses) builds its disconnect event from the chat completion chunks it collected. stamp_responses_usage_cost is now public so the bridge module can price that event.

* Potential fix for pull request finding 'CodeQL / Log Injection'

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

---------

Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

Backport of #47611 to stable/1.106.x
Cherry-picked from f0ae445cc4 (main)
2026-10-21 00:26:25 +00:00
..
litellm_completion_transformation fix(proxy): bill partial Responses API streams when the client disconnects (#47611) 2026-10-21 00:26:25 +00:00
mcp refactor(proxy): expose public names for private proxy helpers (#45170) 2026-10-07 18:59:27 -07:00
__init__.py
test_additional_tools.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_custom_tool_call.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_dispatch.py refactor(python-bridge): ship a signature base and read the resolved call (#45450) 2026-10-08 17:20:29 -07:00
test_metadata_codex_callback.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_no_duplicate_spend_logs.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_null_test_fix.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_responses_api_bridge_flag.py fix(params): validate stream_chunk_size once, before any provider call (#43222) 2026-09-26 23:01:20 +00:00
test_responses_api_lifecycle.py test: make 77 legacy live tests offline in litellm_utils, router_unit and responses dirs (#45298) 2026-10-08 02:37:55 -07:00
test_responses_api_request_body.py fix(bedrock_mantle): send OpenAI explicit prompt cache breakpoints for GPT-5.6 and newer (#38729) 2026-10-08 02:07:22 -07:00
test_responses_prompt_management.py refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
test_responses_router_cooldown.py refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
test_responses_streaming_iterator.py fix(proxy): bill partial Responses API streams when the client disconnects (#47611) 2026-10-21 00:26:25 +00:00
test_responses_supported_endpoints_passthrough.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_responses_utils.py refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
test_responses_websocket_all_providers.py refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
test_rust_bridge_websocket.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_sse_output_recovery.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_streaming_iterator.py fix(proxy): bill partial Responses API streams when the client disconnects (#47611) 2026-10-21 00:26:25 +00:00
test_streaming_iterator_error_events.py refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
test_text_format_conversion.py refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00