scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-10 22:41:41 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 82

docs: wrap prometheus metric explanation

Browse source
This commit is contained in:
Seongho Bae 2026-02-04 22:02:21 +09:00
parent f56d287042
commit 3c9623ac2e
1 changed files with 9 additions and 4 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

13
docs/my-website/docs/proxy/prometheus.md
View file

@ -182,12 +182,17 @@ Use this for LLM API Error monitoring and tracking remaining rate limits and tok
This metric is computed **inside LiteLLM** as `latency_seconds / output_tokens`:
- **Non-streaming requests**: `latency_seconds` is the time from when LiteLLM starts the deployment request to when it receives the final response.
- **Streaming requests**: `latency_seconds` is the **time-to-first-token (TTFT)** (time until LiteLLM receives the first token from the provider).
- **Non-streaming requests**: `latency_seconds` is the time from when LiteLLM
starts the deployment request to when it receives the final response.
- **Streaming requests**: `latency_seconds` is the **time-to-first-token
(TTFT)** (time until LiteLLM receives the first token from the provider).
Because the timer is taken at the LiteLLM boundary, it reflects **provider latency + any LiteLLM proxy overhead** that occurs during that interval (and can include network overhead).
Because the timer is taken at the LiteLLM boundary, it reflects **provider
latency + any LiteLLM proxy overhead** that occurs during that interval (and can
include network overhead).
If you need a metric that targets *just the provider LLM API call latency*, use `litellm_llm_api_latency_metric`.
If you need a metric that targets *just the provider LLM API call latency*, use
`litellm_llm_api_latency_metric`.
#### Fallback (Failover) Metrics