scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-10 03:28:53 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 85

docs prometheus tracking x-remaining tokens

Browse source
This commit is contained in:
Ishaan Jaff 2024-07-01 22:45:33 -07:00
parent 5f04ef14a6
commit 38770da2f6
1 changed files with 27 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

27
docs/my-website/docs/proxy/prometheus.md
View file

@ -1,3 +1,6 @@
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
# 📈 Prometheus metrics [BETA]
LiteLLM Exposes a `/metrics` endpoint for Prometheus to Poll
@ -77,6 +80,25 @@ litellm_settings:
| `litellm_remaining_tokens` | Track `x-ratelimit-remaining-tokens` return from LLM API Deployment |
Example Metric
<Tabs>
<TabItem value="Remaining Requests" label="Remaining Requests">
```shell
litellm_remaining_requests
{
api_base="https://api.openai.com/v1",
api_provider="openai",
litellm_model_name="gpt-3.5-turbo",
model_group="gpt-3.5-turbo"
}
8998.0
```
</TabItem>
<TabItem value="Requests" label="Remaining Tokens">
```shell
litellm_remaining_tokens
{
@ -87,6 +109,11 @@ litellm_remaining_tokens
}
999981.0
```
</TabItem>
</Tabs>
## Monitor System Health
To monitor the health of litellm adjacent services (redis / postgres), do: