mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-15 23:31:29 +00:00
a8ae515bee
* fix(proxy_cli): reap orphaned prisma query-engine processes when a worker dies When the proxy runs multi-worker (uvicorn multiprocess supervisor or the gunicorn arbiter), a worker that crashes or is force-killed never runs its in-process atexit cleanup, so its prisma query-engine subprocess reparents to PID 1 and keeps its database connection pool established forever while the replacement worker opens a fresh pool. Active DB connections then grow past database_connection_pool_limit with every worker death. Run a reaper thread in the supervisor process that marks itself a child subreaper on Linux, scans for adopted query-engine children whose worker is gone, and terminates them with SIGTERM escalating to SIGKILL after a bounded grace period. Engines owned by live workers are children of those workers, never of the supervisor, so they are structurally out of reach. Resolves LIT-4449 Fixes https://github.com/BerriAI/litellm/issues/33023 * fix(proxy_cli): address review findings on the query-engine reaper Make start_query_engine_reaper idempotent, reap simultaneous orphans under one shared grace period instead of serially, and log when a PID survives SIGKILL. Also regenerate schema.d.ts for the update_team docstring line that documents the existing mcp_rpm_limit param (fixes the walk-order-dependent documentation CI failure) and avoid a cast in the prctl wrapper * test(proxy): fix reaper idempotency-test isolation and widen coverage The daemon-thread startup test now stubs threading.enumerate so a reaper thread left running by an earlier test in the same xdist worker cannot satisfy the idempotency guard and skip the code under test. Add coverage for stat-file truncation, non-numeric ppid, non-child reap, signal-to-dead-pid, subreaper capability, and reaper-loop resilience |
||
|---|---|---|
| .. | ||
| db_transaction_queue | ||
| mcp_server | ||
| conftest.py | ||
| test_check_migration.py | ||
| test_create_views.py | ||
| test_db_spend_update_writer.py | ||
| test_db_url_settings.py | ||
| test_exception_handler.py | ||
| test_exception_handler_reconnect_retry.py | ||
| test_prisma_client.py | ||
| test_prisma_planned_engine_restart.py | ||
| test_prisma_self_heal.py | ||
| test_query_engine_reaper.py | ||
| test_rds_iam_token_expiry.py | ||
| test_routing_prisma_wrapper.py | ||
| test_tool_registry_writer.py | ||