scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-12 23:01:41 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 182
litellm/tests/test_litellm/proxy/proxy_server
History
Exact
Yassin Kortam 2cf88d9a37
fix(proxy): send SSE keepalives on assistants runs and A2A streams (#37368) 
Both surfaces wrote zero bytes for the whole time-to-first-token, so an
intermediary with an idle read timeout drops a healthy connection before the
first token. They reached neither keepalive engine, which is what #37322 left
open.

The streaming assistants run spends that wait inside the awaited call that
produces its response, since create_response buffers the first chunk, so it
takes the same open_sse_before_first_byte seam the native routes use. The A2A
route only contacts the upstream agent once its body iterator is first pulled,
so nothing is awaited before the response exists and the gap has to be filled
from inside the stream instead; wrap_sse_stream_with_keepalive_pings already
does that and now takes the filler as a parameter, so A2A gets an SSE comment
its JSON-RPC clients discard rather than Anthropic's ping event.

Off until an operator sets litellm_settings.sse_keepalive_ping_interval_seconds.
2026-08-18 16:22:05 -07:00
..
.coverage_baseline test(proxy): add harness for proxy_server.py behavior-pinning (#28827) 2026-05-25 20:26:44 -07:00
__init__.py test(proxy): add harness for proxy_server.py behavior-pinning (#28827) 2026-05-25 20:26:44 -07:00
_coverage_check.py test(proxy): add harness for proxy_server.py behavior-pinning (#28827) 2026-05-25 20:26:44 -07:00
_pin_check.py test(proxy): add harness for proxy_server.py behavior-pinning (#28827) 2026-05-25 20:26:44 -07:00
conftest.py test(proxy): add harness for proxy_server.py behavior-pinning (#28827) 2026-05-25 20:26:44 -07:00
test_background_health.py fix(router): tag-aware pre-routing strategy selection for shared model_name (#33691) 2026-07-17 09:26:07 -07:00
test_exception_handlers.py test(proxy): pin both branches of the validation exception handler 2026-07-27 09:59:47 -07:00
test_harness_smoke.py test(proxy): add harness for proxy_server.py behavior-pinning (#28827) 2026-05-25 20:26:44 -07:00
test_lifecycle.py feat: pre-adoption shadow eval for the auto-router (blind pairwise judge, derived state) (#36587) 2026-08-13 13:15:45 -07:00
test_openapi_customization.py tests(proxy_server): surface current behavior in tests (#29309) 2026-05-29 23:17:24 -07:00
test_proxy_config.py feat(complexity_router): custom classifier plugins via classifier_type 'custom' (#37249) 2026-08-18 14:09:19 -07:00
test_routes_anthropic_beta.py tests(proxy_server): surface current behavior in tests (#29309) 2026-05-29 23:17:24 -07:00
test_routes_assistants.py tests(proxy_server): surface current behavior in tests (#29309) 2026-05-29 23:17:24 -07:00
test_routes_audio.py fix(proxy): extend response headers hook to streaming, TTS, image gen, and pass-through (#24232) 2026-06-09 22:10:23 +02:00
test_routes_chat_completions.py tests(proxy_server): surface current behavior in tests (#29309) 2026-05-29 23:17:24 -07:00
test_routes_completions.py tests(proxy_server): surface current behavior in tests (#29309) 2026-05-29 23:17:24 -07:00
test_routes_config.py feat(proxy): make DB config-reload interval configurable via config.yaml and UI (#34130) 2026-07-21 22:03:22 +00:00
test_routes_embeddings.py tests(proxy_server): surface current behavior in tests (#29309) 2026-05-29 23:17:24 -07:00
test_routes_invitation.py tests(proxy_server): surface current behavior in tests (#29309) 2026-05-29 23:17:24 -07:00
test_routes_login_sso.py fix(email): stop duplicate legacy invitation email and fix its onboarding link (#36455) 2026-08-10 23:26:27 -07:00
test_routes_misc.py fix(router): tag-aware pre-routing strategy selection for shared model_name (#33691) 2026-07-17 09:26:07 -07:00
test_routes_model_cost_map.py fix(proxy): persist periodic reload schedule state so status survives restarts and fires without store_model_in_db (#35165) 2026-08-04 15:42:57 -07:00
test_routes_model_info.py feat(ui): give auto-routers their own tab on Models + Endpoints 2026-07-29 19:45:02 -07:00
test_routes_model_metrics.py tests(proxy_server): surface current behavior in tests (#29309) 2026-05-29 23:17:24 -07:00
test_routes_models.py fix(proxy): always emit the Anthropic /v1/models token limits, null when unknown (#36961) 2026-08-14 16:52:39 -07:00
test_routes_moderations.py tests(proxy_server): surface current behavior in tests (#29309) 2026-05-29 23:17:24 -07:00
test_routes_onboarding.py fix(email): stop duplicate legacy invitation email and fix its onboarding link (#36455) 2026-08-10 23:26:27 -07:00
test_routes_queue.py tests(proxy_server): surface current behavior in tests (#29309) 2026-05-29 23:17:24 -07:00
test_routes_threads.py tests(proxy_server): surface current behavior in tests (#29309) 2026-05-29 23:17:24 -07:00
test_routes_utils.py tests(proxy_server): surface current behavior in tests (#29309) 2026-05-29 23:17:24 -07:00
test_spend_counters.py perf(spend): gather independent per-scope spend-counter increments (#31578) 2026-06-30 12:07:47 -07:00
test_streaming_helpers.py fix(proxy): send SSE keepalives on assistants runs and A2A streams (#37368) 2026-08-18 16:22:05 -07:00
test_team_model_name_translation.py fix(proxy): expand config-defined model access groups when resolving team models for /v2/model/info (#34211) 2026-08-12 12:54:36 -07:00