diff --git a/docs/my-website/docs/proxy/debugging.md b/docs/my-website/docs/proxy/debugging.md index 5cca6541763..fbcac24a4d6 100644 --- a/docs/my-website/docs/proxy/debugging.md +++ b/docs/my-website/docs/proxy/debugging.md @@ -11,13 +11,13 @@ The proxy also supports json logs. [See here](#json-logs) **via cli** -```bash +```bash showLineNumbers $ litellm --debug ``` **via env** -```python +```python showLineNumbers os.environ["LITELLM_LOG"] = "INFO" ``` @@ -25,25 +25,25 @@ os.environ["LITELLM_LOG"] = "INFO" **via cli** -```bash +```bash showLineNumbers $ litellm --detailed_debug ``` **via env** -```python +```python showLineNumbers os.environ["LITELLM_LOG"] = "DEBUG" ``` ### Debug Logs Run the proxy with `--detailed_debug` to view detailed debug logs -```shell +```shell showLineNumbers litellm --config /path/to/config.yaml --detailed_debug ``` When making requests you should see the POST request sent by LiteLLM to the LLM on the Terminal output -```shell +```shell showLineNumbers POST Request Sent from LiteLLM: curl -X POST \ https://api.openai.com/v1/chat/completions \ @@ -51,25 +51,63 @@ https://api.openai.com/v1/chat/completions \ -d '{"model": "gpt-3.5-turbo", "messages": [{"role": "user", "content": "this is a test request, write a short poem"}]}' ``` +## Debug single request + +Pass in `litellm_request_debug=True` in the request body + +```bash showLineNumbers +curl -L -X POST 'http://0.0.0.0:4000/chat/completions' \ +-H 'Content-Type: application/json' \ +-H 'Authorization: Bearer sk-1234' \ +-d '{ + "model":"fake-openai-endpoint", + "messages": [{"role": "user","content": "How many r in the word strawberry?"}], + "litellm_request_debug": true +}' +``` + +This will emit the raw request sent by LiteLLM to the API Provider and raw response received from the API Provider for **just** this request in the logs. + + +```bash showLineNumbers +INFO: Uvicorn running on http://0.0.0.0:4000 (Press CTRL+C to quit) +20:14:06 - LiteLLM:WARNING: litellm_logging.py:938 - + +POST Request Sent from LiteLLM: +curl -X POST \ +https://exampleopenaiendpoint-production.up.railway.app/chat/completions \ +-H 'Authorization: Be****ey' -H 'Content-Type: application/json' \ +-d '{'model': 'fake', 'messages': [{'role': 'user', 'content': 'How many r in the word strawberry?'}], 'stream': False}' + + +20:14:06 - LiteLLM:WARNING: litellm_logging.py:1015 - RAW RESPONSE: +{"id":"chatcmpl-817fc08f0d6c451485d571dab39b26a1","object":"chat.completion","created":1677652288,"model":"gpt-3.5-turbo-0301","system_fingerprint":"fp_44709d6fcb","choices":[{"index":0,"message":{"role":"assistant","content":"\n\nHello there, how may I assist you today?"},"logprobs":null,"finish_reason":"stop"}],"usage":{"prompt_tokens":9,"completion_tokens":12,"total_tokens":21}} + + +INFO: 127.0.0.1:56155 - "POST /chat/completions HTTP/1.1" 200 OK + +``` + + ## JSON LOGS Set `JSON_LOGS="True"` in your env: -```bash +```bash showLineNumbers export JSON_LOGS="True" ``` **OR** Set `json_logs: true` in your yaml: -```yaml +```yaml showLineNumbers litellm_settings: json_logs: true ``` Start proxy -```bash +```bash showLineNumbers $ litellm ``` @@ -80,7 +118,7 @@ The proxy will now all logs in json format. Turn off fastapi's default 'INFO' logs 1. Turn on 'json logs' -```yaml +```yaml showLineNumbers litellm_settings: json_logs: true ``` @@ -89,20 +127,20 @@ litellm_settings: Only get logs if an error occurs. -```bash +```bash showLineNumbers LITELLM_LOG="ERROR" ``` 3. Start proxy -```bash +```bash showLineNumbers $ litellm ``` Expected Output: -```bash +```bash showLineNumbers # no info statements ``` @@ -119,14 +157,14 @@ This can be caused due to all your models hitting rate limit errors, causing the How to control this? - Adjust the cooldown time -```yaml +```yaml showLineNumbers router_settings: cooldown_time: 0 # 👈 KEY CHANGE ``` - Disable Cooldowns [NOT RECOMMENDED] -```yaml +```yaml showLineNumbers router_settings: disable_cooldowns: True ```