scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-07 08:26:10 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 8

docs: add latency overhead troubleshooting guide (#21603)

Browse source 
* add latency overhead troubleshooting doc

* add latency_overhead to troubleshooting sidebar

* docs: add x-litellm-overhead-duration-ms to latency troubleshooting guide
This commit is contained in:
Ishaan Jaff 2026-02-19 12:42:33 -08:00 • committed by GitHub
parent 2c8fcf854a
commit 18f8a2cee3
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 15 additions and 3 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

18
docs/my-website/docs/troubleshoot/latency_overhead.md
View file

@ -4,12 +4,24 @@ Use this guide when you see unexpected latency overhead between LiteLLM proxy an
## Quick Checklist
1. **Is DEBUG logging enabled?** This is the #1 cause of latency with large payloads.
2. **Are you sending large base64 payloads?** (images, PDFs) — see [Large Payload Overhead](#large-payload-overhead).
3. **Enable detailed timing headers** to pinpoint where time is spent.
1. **Collect the `x-litellm-overhead-duration-ms` response header** — this tells you LiteLLM's total overhead on every request. Start here.
2. **Is DEBUG logging enabled?** This is the #1 cause of latency with large payloads.
3. **Are you sending large base64 payloads?** (images, PDFs) — see [Large Payload Overhead](#large-payload-overhead).
4. **Enable detailed timing headers** to pinpoint where time is spent.
## Diagnostic Headers
### `x-litellm-overhead-duration-ms` (always on)
Every response from LiteLLM includes this header. It shows the total latency overhead in milliseconds added by LiteLLM proxy (i.e. total response time minus the LLM API call time). Collect this on every request to understand your baseline overhead.
```bash
curl -s -D - http://localhost:4000/v1/chat/completions \
-H "Authorization: Bearer sk-..." \
-d '{"model": "gpt-4o", "messages": [{"role": "user", "content": "hi"}]}' \
2>&1 | grep x-litellm-overhead-duration-ms
```
### `x-litellm-callback-duration-ms` (always on)
Shows time spent building callback/logging payloads (ms). If this is high (>100ms), your payloads may be too large for efficient logging.