mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-08 22:21:35 +00:00
59118ae5b6
The existing dashboards in this folder chart the litellm_* Prometheus metrics. Nothing charted the gen_ai.* metrics the OpenTelemetry v2 integration emits, and Grafana's own prebuilt GenAI dashboards cannot: twenty of their twenty-two panels filter on telemetry_sdk_name="openlit", a label LiteLLM does not carry and has no setting to add. Ten panels over the six gen_ai instruments: spend, tokens, request count and p95 duration as stats, then request rate, spend per hour, tokens per minute split by input and output, and p95 duration, time to first token, and provider generation time by model. Template variables for data source, service, and model. Verified against a live Grafana Cloud stack with real traffic across three models. The readme documents the attribute filter the panels depend on, since the default attribute set gives nearly every request its own series and makes every rate-based panel read zero. |
||
|---|---|---|
| .. | ||
| dashboard_1 | ||
| dashboard_genai_otel | ||
| dashboard_v2 | ||
| readme.md | ||
Contains LiteLLM maintained grafana dashboard
This folder contains the json for creating Grafana Dashboards
LiteLLM GenAI Dashboard (OpenTelemetry)
Charts the gen_ai.* metrics from the OpenTelemetry v2 integration: spend, tokens, request rate, and latency percentiles by model. Separate from the dashboards below, which chart the litellm_* Prometheus metrics.
LiteLLM v2 Dashboard
Pre-Requisites
- Setup LiteLLM Proxy Prometheus Metrics https://docs.litellm.ai/docs/proxy/prometheus