mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-11 03:38:38 +00:00
docs(debugging.md): document new feature
Closes https://github.com/BerriAI/litellm/issues/13814
This commit is contained in:
parent
48619f2b7b
commit
c802c472b5
1 changed files with 53 additions and 15 deletions
|
|
@ -11,13 +11,13 @@ The proxy also supports json logs. [See here](#json-logs)
|
|||
|
||||
**via cli**
|
||||
|
||||
```bash
|
||||
```bash showLineNumbers
|
||||
$ litellm --debug
|
||||
```
|
||||
|
||||
**via env**
|
||||
|
||||
```python
|
||||
```python showLineNumbers
|
||||
os.environ["LITELLM_LOG"] = "INFO"
|
||||
```
|
||||
|
||||
|
|
@ -25,25 +25,25 @@ os.environ["LITELLM_LOG"] = "INFO"
|
|||
|
||||
**via cli**
|
||||
|
||||
```bash
|
||||
```bash showLineNumbers
|
||||
$ litellm --detailed_debug
|
||||
```
|
||||
|
||||
**via env**
|
||||
|
||||
```python
|
||||
```python showLineNumbers
|
||||
os.environ["LITELLM_LOG"] = "DEBUG"
|
||||
```
|
||||
|
||||
### Debug Logs
|
||||
|
||||
Run the proxy with `--detailed_debug` to view detailed debug logs
|
||||
```shell
|
||||
```shell showLineNumbers
|
||||
litellm --config /path/to/config.yaml --detailed_debug
|
||||
```
|
||||
|
||||
When making requests you should see the POST request sent by LiteLLM to the LLM on the Terminal output
|
||||
```shell
|
||||
```shell showLineNumbers
|
||||
POST Request Sent from LiteLLM:
|
||||
curl -X POST \
|
||||
https://api.openai.com/v1/chat/completions \
|
||||
|
|
@ -51,25 +51,63 @@ https://api.openai.com/v1/chat/completions \
|
|||
-d '{"model": "gpt-3.5-turbo", "messages": [{"role": "user", "content": "this is a test request, write a short poem"}]}'
|
||||
```
|
||||
|
||||
## Debug single request
|
||||
|
||||
Pass in `litellm_request_debug=True` in the request body
|
||||
|
||||
```bash showLineNumbers
|
||||
curl -L -X POST 'http://0.0.0.0:4000/chat/completions' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-H 'Authorization: Bearer sk-1234' \
|
||||
-d '{
|
||||
"model":"fake-openai-endpoint",
|
||||
"messages": [{"role": "user","content": "How many r in the word strawberry?"}],
|
||||
"litellm_request_debug": true
|
||||
}'
|
||||
```
|
||||
|
||||
This will emit the raw request sent by LiteLLM to the API Provider and raw response received from the API Provider for **just** this request in the logs.
|
||||
|
||||
|
||||
```bash showLineNumbers
|
||||
INFO: Uvicorn running on http://0.0.0.0:4000 (Press CTRL+C to quit)
|
||||
20:14:06 - LiteLLM:WARNING: litellm_logging.py:938 -
|
||||
|
||||
POST Request Sent from LiteLLM:
|
||||
curl -X POST \
|
||||
https://exampleopenaiendpoint-production.up.railway.app/chat/completions \
|
||||
-H 'Authorization: Be****ey' -H 'Content-Type: application/json' \
|
||||
-d '{'model': 'fake', 'messages': [{'role': 'user', 'content': 'How many r in the word strawberry?'}], 'stream': False}'
|
||||
|
||||
|
||||
20:14:06 - LiteLLM:WARNING: litellm_logging.py:1015 - RAW RESPONSE:
|
||||
{"id":"chatcmpl-817fc08f0d6c451485d571dab39b26a1","object":"chat.completion","created":1677652288,"model":"gpt-3.5-turbo-0301","system_fingerprint":"fp_44709d6fcb","choices":[{"index":0,"message":{"role":"assistant","content":"\n\nHello there, how may I assist you today?"},"logprobs":null,"finish_reason":"stop"}],"usage":{"prompt_tokens":9,"completion_tokens":12,"total_tokens":21}}
|
||||
|
||||
|
||||
INFO: 127.0.0.1:56155 - "POST /chat/completions HTTP/1.1" 200 OK
|
||||
|
||||
```
|
||||
|
||||
|
||||
## JSON LOGS
|
||||
|
||||
Set `JSON_LOGS="True"` in your env:
|
||||
|
||||
```bash
|
||||
```bash showLineNumbers
|
||||
export JSON_LOGS="True"
|
||||
```
|
||||
**OR**
|
||||
|
||||
Set `json_logs: true` in your yaml:
|
||||
|
||||
```yaml
|
||||
```yaml showLineNumbers
|
||||
litellm_settings:
|
||||
json_logs: true
|
||||
```
|
||||
|
||||
Start proxy
|
||||
|
||||
```bash
|
||||
```bash showLineNumbers
|
||||
$ litellm
|
||||
```
|
||||
|
||||
|
|
@ -80,7 +118,7 @@ The proxy will now all logs in json format.
|
|||
Turn off fastapi's default 'INFO' logs
|
||||
|
||||
1. Turn on 'json logs'
|
||||
```yaml
|
||||
```yaml showLineNumbers
|
||||
litellm_settings:
|
||||
json_logs: true
|
||||
```
|
||||
|
|
@ -89,20 +127,20 @@ litellm_settings:
|
|||
|
||||
Only get logs if an error occurs.
|
||||
|
||||
```bash
|
||||
```bash showLineNumbers
|
||||
LITELLM_LOG="ERROR"
|
||||
```
|
||||
|
||||
3. Start proxy
|
||||
|
||||
|
||||
```bash
|
||||
```bash showLineNumbers
|
||||
$ litellm
|
||||
```
|
||||
|
||||
Expected Output:
|
||||
|
||||
```bash
|
||||
```bash showLineNumbers
|
||||
# no info statements
|
||||
```
|
||||
|
||||
|
|
@ -119,14 +157,14 @@ This can be caused due to all your models hitting rate limit errors, causing the
|
|||
How to control this?
|
||||
- Adjust the cooldown time
|
||||
|
||||
```yaml
|
||||
```yaml showLineNumbers
|
||||
router_settings:
|
||||
cooldown_time: 0 # 👈 KEY CHANGE
|
||||
```
|
||||
|
||||
- Disable Cooldowns [NOT RECOMMENDED]
|
||||
|
||||
```yaml
|
||||
```yaml showLineNumbers
|
||||
router_settings:
|
||||
disable_cooldowns: True
|
||||
```
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue