mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-16 23:41:43 +00:00
2d0823abf7
Every proxy process registers the same scheduler, and it registers it once per uvicorn worker process rather than once per pod, so a job with a shared side effect runs replicas times processes. The jobs that did elect an owner each hand-rolled it, and the dialects disagree on the one case that matters: acquire_lock reports contention and an unreachable Redis identically, so the PTU rollup ran unguarded while key rotation, spend log cleanup and UI session cleanup skipped, meaning a Redis outage silently stopped those three everywhere at once. Adds run_as_single_owner and claim_once_per_window, which name that decision as WhenLockUnavailable and carry the two lease shapes the proxy actually uses: a mutex held for one run, and a done-marker sized to a reporting window. The lease is now renewed while the body runs, so the TTL is a failover deadline rather than a run budget. On a live proxy that is the difference between a spend log cleanup losing its 60s lock partway through a multi-minute sweep, which a challenger could take at 75s, 120s and 180s, and holding it to completion. LITELLM_JOB_ROLE=serving registers none of these jobs, so an operator can run them on a dedicated worker deployment instead of on every replica taking traffic. A serving pod keeps the jobs that drain its own in-memory queues and refresh its own model registry, so it still writes spend. The default is unchanged, and an unrecognised value falls back to it with a warning. Both Helm charts gain an opt-in worker Deployment, off by default and pinned to one process. Live-Redis integration tests cover election, failover, renewal, rolling restart and once-per-window claims. Also fixes the ownership gauge, which only ever fired on a reentrant re-acquisition and so was driven to 0 by every release and never back up. |
||
|---|---|---|
| .. | ||
| html_forms | ||
| test_auth_cache_invalidation_pubsub.py | ||
| test_cache_codec.py | ||
| test_callback_utils.py | ||
| test_config_sync_pubsub.py | ||
| test_custom_openapi_spec.py | ||
| test_encrypt_decrypt_utils.py | ||
| test_expired_ui_session_key_cleanup_manager.py | ||
| test_get_routes.py | ||
| test_http_parsing_utils.py | ||
| test_json_merge_patch.py | ||
| test_key_rotation_e2e.py | ||
| test_key_rotation_integration.py | ||
| test_key_rotation_lock.py | ||
| test_key_rotation_manager.py | ||
| test_load_config_utils.py | ||
| test_openai_endpoint_utils.py | ||
| test_path_utils.py | ||
| test_periodic_reload_schedule.py | ||
| test_reset_budget_job.py | ||
| test_scheduled_job_stagger.py | ||
| test_single_owner_job.py | ||
| test_sse_keepalive.py | ||
| test_static_asset_utils.py | ||
| test_timezone_utils.py | ||
| test_upsert_budget_membership.py | ||
| test_user_api_key_cache.py | ||