scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-08-28 05:25:59 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 8

fix: suggest Gunicorn instead of uvicorn when using max_requests_before_restart (#17788)

Browse source
This commit is contained in:
Alexsander Hamir 2025-12-10 13:09:00 -08:00 • committed by GitHub
parent 7ad2a58dcd
commit 439bb5bfe3
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 7 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

7
docs/my-website/docs/proxy/prod.md
View file

@ -81,6 +81,13 @@ CMD ["--port", "4000", "--config", "./proxy_server_config.yaml", "--num_workers"
export MAX_REQUESTS_BEFORE_RESTART=10000
```
> **Tip:** When using `--max_requests_before_restart`, the `--run_gunicorn` flag is more stable and mature as it uses Gunicorn's battle-tested worker recycling mechanism instead of Uvicorn's implementation.
```shell
# Use Gunicorn for more stable worker recycling
CMD ["--port", "4000", "--config", "./proxy_server_config.yaml", "--num_workers", "$(nproc)", "--run_gunicorn", "--max_requests_before_restart", "10000"]
```
## 4. Use Redis 'port','host', 'password'. NOT 'redis_url'