From 09992a61223af3192cc32b265c8c743e5d72088f Mon Sep 17 00:00:00 2001 From: Ishaan Jaff Date: Sat, 23 Mar 2024 19:36:26 -0700 Subject: [PATCH] (docs) prod best perf --- docs/my-website/docs/proxy/prod.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/docs/my-website/docs/proxy/prod.md b/docs/my-website/docs/proxy/prod.md index 568d37a7d13..a5d0b7033de 100644 --- a/docs/my-website/docs/proxy/prod.md +++ b/docs/my-website/docs/proxy/prod.md @@ -24,6 +24,13 @@ litellm_settings: set_verbose: True ``` +You should only see the following level of details in logs on the proxy server +```shell +# INFO: 192.168.2.205:11774 - "POST /chat/completions HTTP/1.1" 200 OK +# INFO: 192.168.2.205:34717 - "POST /chat/completions HTTP/1.1" 200 OK +# INFO: 192.168.2.205:29734 - "POST /chat/completions HTTP/1.1" 200 OK +``` + ## 2. On Kubernetes - Use 1 Uvicorn worker [Suggested CMD] Use this Docker `CMD`. This will start the proxy with 1 Uvicorn Async Worker