38f3f8a1a9
* fix(router): carry a 429 status on cooldown and no-deployment errors RouterRateLimitError and RouterRateLimitErrorBasic had no status_code, so the proxy answered an all-deployments-cooled-down request as a generic 500 class error and logged the bug report notice for every cooldown. Both now carry status_code 429, which the proxy maps to a rate_limit_error and which the retry policy already treats as retryable * fix(proxy): start Sentry at config load and never attach request bodies Sentry only came up inside set_callbacks on the first LLM call, so boot and background errors were never reported, and general_settings.alerting: [sentry] always raised Missing SENTRY_DSN because the alerting path read a dead litellm.utils pin instead of the live SDK. The proxy now initializes Sentry once while loading the config whenever sentry is a configured callback or alerting target, the alerting and failure handlers read the live instance, and max_request_body_size is pinned to never so prompts and completions stay out of Sentry events * fix(tracing): log Lens failures at ERROR and export row counters to Prometheus A tracing store that fails to start, or a Lens service that rejects a batch, was only logged at WARNING, so operators running at the default ERROR level never saw tracing silently go dark. Both now log at ERROR and name the lens store. The exporter's rows_written and rows_dropped tallies were never exported, so the two counts are now Prometheus counters litellm_lens_export_rows_written_total and litellm_lens_export_rows_dropped_total, registered lazily on the default registry like the other proxy-level counters and charted on the all-metrics Grafana dashboard * test(proxy): drop the litellm import the alerting tests no longer use * fix(proxy): start Sentry once a DSN is set, from any config source Boot-time init used to lock Sentry in even with no SENTRY_DSN, so a DSN loaded later from DB environment_variables was never picked up. initialize_sentry now skips when no DSN is set and re-inits when the DSN changes. alerting_handler calls it too, so alerting: [sentry] set through the DB works. Drops the litellm_settings.callbacks check since that config fails to load with sentry in it * fix(sentry): keep stack frame locals out of events unless PII is opted in Frame locals carried prompts (messages, request bodies, provider options) into every error event. include_local_variables now follows SENTRY_SEND_DEFAULT_PII * test(router): drop the test that only pinned the 429 class attribute * fix(proxy): start Sentry at boot when it is only an input callback * fix(proxy): keep Sentry init and alerting within the strict ruff and basedpyright ceilings --------- Co-authored-by: yuneng-jiang <yuneng@berri.ai> |
||
|---|---|---|
| .. | ||
| dashboard_all_metrics | ||
| dashboard_genai_otel | ||
| dashboard_v2 | ||
| readme.md | ||
Contains LiteLLM maintained grafana dashboard
This folder contains the json for creating Grafana Dashboards
LiteLLM GenAI Dashboard (OpenTelemetry)
Charts the gen_ai.* metrics from the OpenTelemetry v2 integration: spend, tokens, request rate, and latency percentiles by model. Separate from the dashboards below, which chart the litellm_* Prometheus metrics.
LiteLLM All Prometheus Metrics dashboard
Every litellm_* Prometheus metric family the proxy can emit (134 families, 95 panels) grouped by theme: traffic, latency, spend and tokens, cache, deployments, rate limits, budgets, guardrails, MCP, managed files and batches, users and teams, plus the Redis circuit breaker, spend log cleanup and prometheus_system service metrics. Start here if you want everything on one screen; see its readme for import steps and which panels need a feature enabled before they show data
LiteLLM v2 Dashboard
A compact view of proxy request rate, failures, latency and the top remaining-request / remaining-token gauges per model group
Pre-Requisites
- Setup LiteLLM Proxy Prometheus Metrics https://docs.litellm.ai/docs/proxy/prometheus