mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-19 00:01:29 +00:00
59118ae5b6
The existing dashboards in this folder chart the litellm_* Prometheus metrics. Nothing charted the gen_ai.* metrics the OpenTelemetry v2 integration emits, and Grafana's own prebuilt GenAI dashboards cannot: twenty of their twenty-two panels filter on telemetry_sdk_name="openlit", a label LiteLLM does not carry and has no setting to add. Ten panels over the six gen_ai instruments: spend, tokens, request count and p95 duration as stats, then request rate, spend per hour, tokens per minute split by input and output, and p95 duration, time to first token, and provider generation time by model. Template variables for data source, service, and model. Verified against a live Grafana Cloud stack with real traffic across three models. The readme documents the attribute filter the panels depend on, since the default attribute set gives nearly every request its own series and makes every rate-based panel read zero. |
||
|---|---|---|
| .. | ||
| ai_coding_tool_guides | ||
| anthropic_agent_sdk | ||
| benchmark | ||
| codellama-server | ||
| community-resources | ||
| gollem_go_agent_framework | ||
| litellm-ollama-docker-image | ||
| litellm_proxy_server | ||
| litellm_router | ||
| litellm_router_load_test | ||
| livekit_agent_sdk | ||
| logging_observability | ||
| misc | ||
| mock_guardrail_server | ||
| mock_prompt_management_server | ||
| Benchmarking_LLMs_by_use_case.ipynb | ||
| Claude_(Anthropic)_with_Streaming_liteLLM_Examples.ipynb | ||
| Evaluating_LLMs.ipynb | ||
| google_adk_litellm_tutorial.ipynb | ||
| liteLLM_A121_Jurrasic_example.ipynb | ||
| LiteLLM_Azure_and_OpenAI_example.ipynb | ||
| liteLLM_Baseten.ipynb | ||
| LiteLLM_batch_completion.ipynb | ||
| LiteLLM_Bedrock.ipynb | ||
| liteLLM_clarifai_Demo.ipynb | ||
| LiteLLM_CometAPI.ipynb | ||
| LiteLLM_Comparing_LLMs.ipynb | ||
| LiteLLM_Completion_Cost.ipynb | ||
| liteLLM_function_calling.ipynb | ||
| liteLLM_Getting_Started.ipynb | ||
| LiteLLM_HuggingFace.ipynb | ||
| liteLLM_IBM_Watsonx.ipynb | ||
| liteLLM_Langchain_Demo.ipynb | ||
| litellm_model_fallback.ipynb | ||
| LiteLLM_NovitaAI_Cookbook.ipynb | ||
| liteLLM_Ollama.ipynb | ||
| LiteLLM_OpenRouter.ipynb | ||
| LiteLLM_Petals.ipynb | ||
| LiteLLM_PromptLayer.ipynb | ||
| liteLLM_Replicate_Demo.ipynb | ||
| liteLLM_Streaming_Demo.ipynb | ||
| litellm_test_multiple_llm_demo.ipynb | ||
| litellm_Test_Multiple_Providers.ipynb | ||
| LiteLLM_User_Based_Rate_Limits.ipynb | ||
| liteLLM_VertextAI_Example.ipynb | ||
| Migrating_to_LiteLLM_Proxy_from_OpenAI_Azure_OpenAI.ipynb | ||
| mlflow_langchain_tracing_litellm_proxy.ipynb | ||
| nova_sonic_realtime.py | ||
| Parallel_function_calling.ipynb | ||
| Proxy_Batch_Users.ipynb | ||
| result.html | ||
| TogetherAI_liteLLM.ipynb | ||
| Using_Nemo_Guardrails_with_LiteLLM_Server.ipynb | ||
| veo_video_generation.py | ||
| VLLM_Model_Testing.ipynb | ||