mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-30 06:43:27 +00:00
57f6e1b39f
* fix(proxy): default the uvicorn worker healthcheck timeout to 60 s so multi-worker boots survive a slow import * fix(proxy): warn on uvicorn below 0.37 when multi-worker boots cannot get the longer healthcheck uvicorn 0.33 to 0.36 has no timeout_worker_healthcheck, so the 60 s default never reaches it and a slow worker start still crash-loops with no hint. Print an upgrade warning when the proxy runs more than one uvicorn worker on such a version, keep granian out of the uvicorn-only path, and type the new tests. * fix(proxy): name --run_granian in the healthcheck help text and add per-worker grace guidance to the helm chart * fix(proxy): keep the litellm command light so spawned workers answer uvicorn's first health ping Each worker uvicorn spawns re-runs the parent's main script before it starts answering health pings. The litellm console script imported the whole SDK at the top, so on a busy or CPU-limited host a new worker missed uvicorn's 5 s ping and was killed in a loop. The script now runs litellm_cli.main:proxy_server, which imports the proxy only when called, and gateway.launch imports LiteLLM inside main the same way. This replaces the earlier 60 s healthcheck default, so uvicorn's 5 s healthcheck and SIGTERM timing stay as they are. * fix(proxy): load the worker app before its log config and without GC pauses * fix(gateway): ship litellm_cli in the gateway image gateway.launch imports without_gc_pauses from litellm_cli, which the gateway image never copied, so python -m gateway.launch failed at import. Also build the gateway image when the launcher or litellm_cli changes. * fix(proxy): apply the worker log config before the proxy app import, as earlier releases did --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| actions | ||
| assets/scaledown | ||
| codeql | ||
| e2e-stack | ||
| ISSUE_TEMPLATE | ||
| observatory | ||
| prompts | ||
| PULL_REQUEST_TEMPLATE | ||
| screenshots | ||
| scripts | ||
| workflows | ||
| ci-coverage-allowlist.yml | ||
| CODEOWNERS | ||
| dependabot.yaml | ||
| deploy-on-aws.png | ||
| deploy-on-gcp.png | ||
| deploy-to-aws.png | ||
| FUNDING.yml | ||
| issue-labels.json | ||
| mutmut-coverage.rc | ||
| template.yaml | ||
