Kerry Lu 339fd90836 fix(proxy): streaming responses hand the event loop back every 50 ms instead of holding it for the whole buffered stream (#46925) ... * fix(proxy): streaming responses hand the event loop back every 50 ms instead of holding it for the whole buffered stream * fix(proxy): yield to the event loop on every streamed chunk, no hold budget or clock * docs(proxy): say why the streaming loop yields on every chunk * fix(proxy): streaming loop yields once it has held the event loop 50 ms, clock read inline * docs(proxy): streaming loop comment names the 50 ms hold * Revert "docs(proxy): streaming loop comment names the 50 ms hold" This reverts commit cbf1371873. * refactor(proxy): streaming loop hold budget is _STREAM_LOOP_HOLD_MS = 50 * Revert "refactor(proxy): streaming loop hold budget is _STREAM_LOOP_HOLD_MS = 50" This reverts commit 356454def6. * docs(proxy): streaming loop comment notes the yield is a floor, not a count of releases --------- Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>		2026-10-16 11:33:01 -07:00
..
.coverage_baseline	test(proxy): move proxy_server, _experimental and db tests into tests/unit/proxy (#44012)	2026-10-01 18:14:24 +00:00
__init__.py	test(proxy): move proxy_server, _experimental and db tests into tests/unit/proxy (#44012)	2026-10-01 18:14:24 +00:00
_coverage_check.py	test(proxy): move proxy_server, _experimental and db tests into tests/unit/proxy (#44012)	2026-10-01 18:14:24 +00:00
_pin_check.py	test(proxy): move proxy_server, _experimental and db tests into tests/unit/proxy (#44012)	2026-10-01 18:14:24 +00:00
conftest.py	test: repair stale and polluting tests red on scheduled main CI (#44229)	2026-10-02 21:32:17 +00:00
test_background_health.py	refactor(proxy): expose public names for private proxy helpers (#45170)	2026-10-07 18:59:27 -07:00
test_exception_handlers.py	fix(proxy): answer a rejected websocket handshake without crashing the HTTP exception handler (#45251)	2026-10-07 22:27:46 -07:00
test_harness_smoke.py	test(proxy): move proxy_server, _experimental and db tests into tests/unit/proxy (#44012)	2026-10-01 18:14:24 +00:00
test_lifecycle.py	fix(proxy): track and drain management audit log writes on shutdown (#43594)	2026-10-16 08:41:34 +00:00
test_openapi_customization.py	test(proxy): move proxy_server, _experimental and db tests into tests/unit/proxy (#44012)	2026-10-01 18:14:24 +00:00
test_proxy_config.py	fix(proxy): cooldown errors carry 429, Sentry starts at boot without request bodies, tracing failures log at ERROR, exporter counters exported (#46344)	2026-10-16 00:53:52 +00:00
test_routes_anthropic_beta.py	test(proxy): move proxy_server, _experimental and db tests into tests/unit/proxy (#44012)	2026-10-01 18:14:24 +00:00
test_routes_assistants.py	fix(proxy): keep the mapped status on assistants and threads route errors (#44994)	2026-10-09 16:55:15 -07:00
test_routes_audio.py	test(proxy): move proxy_server, _experimental and db tests into tests/unit/proxy (#44012)	2026-10-01 18:14:24 +00:00
test_routes_chat_completions.py	refactor(proxy): expose public names for private proxy helpers (#45170)	2026-10-07 18:59:27 -07:00
test_routes_completions.py	test(proxy): move proxy_server, _experimental and db tests into tests/unit/proxy (#44012)	2026-10-01 18:14:24 +00:00
test_routes_config.py	fix(proxy)!: delete callbacks configured under callbacks or failure_callback (#42740)	2026-10-16 09:22:22 -07:00
test_routes_embeddings.py	refactor(proxy): expose public names for private proxy helpers (#45170)	2026-10-07 18:59:27 -07:00
test_routes_invitation.py	refactor(proxy): expose public names for private proxy helpers (#45170)	2026-10-07 18:59:27 -07:00
test_routes_login_sso.py	test(cost): add tier 1 claude messages stream twins and recapture drifted usage shapes (#46690)	2026-10-15 17:11:13 +00:00
test_routes_misc.py	test(proxy): move proxy_server, _experimental and db tests into tests/unit/proxy (#44012)	2026-10-01 18:14:24 +00:00
test_routes_model_cost_map.py	fix(caching): key response cache by router model group in litellm_metadata (#44542)	2026-10-05 17:06:21 +00:00
test_routes_model_info.py	fix(proxy): list team access group models in /v1/model/info and keep key model lists narrow (#32599)	2026-10-15 10:23:03 -07:00
test_routes_model_metrics.py	test(proxy): move proxy_server, _experimental and db tests into tests/unit/proxy (#44012)	2026-10-01 18:14:24 +00:00
test_routes_models.py	feat(proxy): serve a Codex-native model catalog with per-model service_tiers from /v1/models (#44136)	2026-10-03 20:42:55 +00:00
test_routes_moderations.py	test(proxy): move proxy_server, _experimental and db tests into tests/unit/proxy (#44012)	2026-10-01 18:14:24 +00:00
test_routes_onboarding.py	test(proxy): move proxy_server, _experimental and db tests into tests/unit/proxy (#44012)	2026-10-01 18:14:24 +00:00
test_routes_queue.py	test(proxy): move proxy_server, _experimental and db tests into tests/unit/proxy (#44012)	2026-10-01 18:14:24 +00:00
test_routes_threads.py	fix(proxy): keep the mapped status on assistants and threads route errors (#44994)	2026-10-09 16:55:15 -07:00
test_routes_utils.py	fix(proxy)!: count prompt-only token_counter requests with the provider when call_endpoint is set (#46839)	2026-10-16 10:54:42 +00:00
test_spend_counters.py	fix(caching): make Redis Cluster commands wait for a pool slot instead of tripping the circuit breaker (#46488)	2026-10-14 15:27:00 -07:00
test_streaming_helpers.py	fix(proxy): streaming responses hand the event loop back every 50 ms instead of holding it for the whole buffered stream (#46925)	2026-10-16 11:33:01 -07:00
test_team_model_name_translation.py	fix(proxy): resolve model aliases to their target deployment for listing metadata (#41041)	2026-10-15 00:38:50 -07:00