mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-12 23:01:41 +00:00
b0626cad8c
APScheduler anchors an interval job at now + interval, so every scheduled background job registered in one proxy startup shares a single firing instant for the life of the process, and every replica a rollout brought up together shares that instant too. Each tick the spend flushes, budget reset sweep, config-in-DB reload, credential reload and cost pollers all hit Postgres at the same moment, on every pod, competing with request-path auth and budget queries. Shift each eligible job by a deterministic offset derived from sha256(job_id, identity), where identity covers the pod and the worker process. The offset lives in the trigger rather than in a one-off next_run_time, because a cron trigger recomputes each fire from the wall clock and would otherwise snap straight back onto the shared instant. An interval job is never offset by more than one of its own periods. Only schedules LiteLLM chose are shifted: interval jobs always, cron jobs only when the id is one of the product's own defaults, so an operator-supplied crontab keeps the instant it asks for. general_settings.scheduled_job_stagger turns it off, widens the window, replaces the identity, or pins a job. The applied offsets are logged once at startup and each fire logs its scheduled instant against its actual start. Resolves LIT-5433 |
||
|---|---|---|
| .. | ||
| litellm-dashboard | ||
| Dockerfile | ||
| nginx.conf | ||