mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-20 00:11:50 +00:00
2d0823abf7
Every proxy process registers the same scheduler, and it registers it once per uvicorn worker process rather than once per pod, so a job with a shared side effect runs replicas times processes. The jobs that did elect an owner each hand-rolled it, and the dialects disagree on the one case that matters: acquire_lock reports contention and an unreachable Redis identically, so the PTU rollup ran unguarded while key rotation, spend log cleanup and UI session cleanup skipped, meaning a Redis outage silently stopped those three everywhere at once. Adds run_as_single_owner and claim_once_per_window, which name that decision as WhenLockUnavailable and carry the two lease shapes the proxy actually uses: a mutex held for one run, and a done-marker sized to a reporting window. The lease is now renewed while the body runs, so the TTL is a failover deadline rather than a run budget. On a live proxy that is the difference between a spend log cleanup losing its 60s lock partway through a multi-minute sweep, which a challenger could take at 75s, 120s and 180s, and holding it to completion. LITELLM_JOB_ROLE=serving registers none of these jobs, so an operator can run them on a dedicated worker deployment instead of on every replica taking traffic. A serving pod keeps the jobs that drain its own in-memory queues and refresh its own model registry, so it still writes spend. The default is unchanged, and an unrecognised value falls back to it with a warning. Both Helm charts gain an opt-in worker Deployment, off by default and pinned to one process. Live-Redis integration tests cover election, failover, renewal, rolling restart and once-per-window claims. Also fixes the ownership gauge, which only ever fired on a reentrant re-acquisition and so was driven to 0 by every release and never back up. |
||
|---|---|---|
| .. | ||
| db_transaction_queue | ||
| mcp_server | ||
| conftest.py | ||
| test_autorouter_session_rollup.py | ||
| test_check_migration.py | ||
| test_create_views.py | ||
| test_daily_spend_bulk_upsert.py | ||
| test_db_spend_update_writer.py | ||
| test_db_url_settings.py | ||
| test_exception_handler.py | ||
| test_exception_handler_reconnect_retry.py | ||
| test_gateway_request_tracking.py | ||
| test_prisma_client.py | ||
| test_prisma_planned_engine_restart.py | ||
| test_prisma_self_heal.py | ||
| test_query_engine_reaper.py | ||
| test_rds_iam_token_expiry.py | ||
| test_replica_identity.py | ||
| test_routing_prisma_wrapper.py | ||
| test_spend_log_batching.py | ||
| test_spend_log_tool_index.py | ||
| test_tool_registry_writer.py | ||