litellm/tests/test_litellm/proxy/proxy_server
devin-ai-integration[bot] 9e18526887
feat(deploy): metrics sidecar and separate metrics port in Helm and Terraform (#40163)
* feat(deploy): expose SSE keepalive, pre-call checks and a metrics sidecar in Helm and Terraform

Typed reliability values on both Helm charts and the AWS/GCP Terraform
modules, a dedicated ClusterIP Service for the separate Prometheus port,
a /health route on the metrics server and dead-worker pruning so the
aggregate does not keep stale multiprocess samples.

Resolves LIT-7142

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(deploy): drop reliability config from Helm and Terraform, keep only the metrics sidecar

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(proxy): cover startup pruning of dead workers' live gauges and unsignalable pids

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-08 13:31:07 -07:00
..
.coverage_baseline test(proxy): add harness for proxy_server.py behavior-pinning (#28827) 2026-05-25 20:26:44 -07:00
__init__.py test(proxy): add harness for proxy_server.py behavior-pinning (#28827) 2026-05-25 20:26:44 -07:00
_coverage_check.py test(proxy): add harness for proxy_server.py behavior-pinning (#28827) 2026-05-25 20:26:44 -07:00
_pin_check.py test(proxy): add harness for proxy_server.py behavior-pinning (#28827) 2026-05-25 20:26:44 -07:00
conftest.py test(proxy): add harness for proxy_server.py behavior-pinning (#28827) 2026-05-25 20:26:44 -07:00
test_background_health.py feat(health): opt-in model-group allowlist for background health checks and health-check routing (#38539) 2026-08-27 12:25:56 -07:00
test_exception_handlers.py test(proxy): pin both branches of the validation exception handler 2026-07-27 09:59:47 -07:00
test_harness_smoke.py test(proxy): add harness for proxy_server.py behavior-pinning (#28827) 2026-05-25 20:26:44 -07:00
test_lifecycle.py feat(deploy): metrics sidecar and separate metrics port in Helm and Terraform (#40163) 2026-09-08 13:31:07 -07:00
test_openapi_customization.py tests(proxy_server): surface current behavior in tests (#29309) 2026-05-29 23:17:24 -07:00
test_proxy_config.py fix(proxy): load db credentials in the model reconcile so a worker never serves a model before its credential (#39876) 2026-09-08 10:08:24 -07:00
test_routes_anthropic_beta.py tests(proxy_server): surface current behavior in tests (#29309) 2026-05-29 23:17:24 -07:00
test_routes_assistants.py tests(proxy_server): surface current behavior in tests (#29309) 2026-05-29 23:17:24 -07:00
test_routes_audio.py fix(proxy): narrow audio_speech response before reading upstream content-type 2026-08-30 12:46:11 -07:00
test_routes_chat_completions.py tests(proxy_server): surface current behavior in tests (#29309) 2026-05-29 23:17:24 -07:00
test_routes_completions.py tests(proxy_server): surface current behavior in tests (#29309) 2026-05-29 23:17:24 -07:00
test_routes_config.py fix: reject constructor-managed router settings 2026-09-02 07:49:25 +00:00
test_routes_embeddings.py tests(proxy_server): surface current behavior in tests (#29309) 2026-05-29 23:17:24 -07:00
test_routes_invitation.py tests(proxy_server): surface current behavior in tests (#29309) 2026-05-29 23:17:24 -07:00
test_routes_login_sso.py feat(auth): enforce configurable password policy and SSO-only login (#39381) 2026-09-02 14:28:13 -07:00
test_routes_misc.py feat(ui): let admins supply a dark-mode variant of their custom logo (#37662) 2026-08-20 12:48:55 -07:00
test_routes_model_cost_map.py fix(proxy): persist periodic reload schedule state so status survives restarts and fires without store_model_in_db (#35165) 2026-08-04 15:42:57 -07:00
test_routes_model_info.py fix(ui): make Admin UI table pagination honor the selected page size 2026-09-03 23:59:58 +00:00
test_routes_model_metrics.py tests(proxy_server): surface current behavior in tests (#29309) 2026-05-29 23:17:24 -07:00
test_routes_models.py feat(proxy): honor model_info.display_name in the Anthropic-shaped /v1/models listing 2026-09-01 17:10:52 -07:00
test_routes_moderations.py tests(proxy_server): surface current behavior in tests (#29309) 2026-05-29 23:17:24 -07:00
test_routes_onboarding.py feat(auth): enforce configurable password policy and SSO-only login (#39381) 2026-09-02 14:28:13 -07:00
test_routes_queue.py tests(proxy_server): surface current behavior in tests (#29309) 2026-05-29 23:17:24 -07:00
test_routes_threads.py tests(proxy_server): surface current behavior in tests (#29309) 2026-05-29 23:17:24 -07:00
test_routes_utils.py fix(router): bare authenticating-provider names declare nothing 2026-08-31 13:34:14 -07:00
test_spend_counters.py test(proxy): parametrize the stale end-user counter case so no Final local sits in a loop 2026-09-04 18:00:40 -07:00
test_streaming_helpers.py fix(proxy): emit SSE keepalives on queue, rag, azure passthrough, usage chat and policy enrich streams (#39273) 2026-09-03 18:23:37 -07:00
test_team_model_name_translation.py feat(proxy): honor model_info.display_name in the Anthropic-shaped /v1/models listing 2026-09-01 17:10:52 -07:00