mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-28 01:32:17 +00:00
14c0a2b3e2
* feat(prometheus): emit per-token-type detail metrics (LIT-3220) (#28372) Adds five sparse counter metrics that break out the token detail fields providers already report in `usage.prompt_tokens_details` and `usage.completion_tokens_details`: - litellm_input_cached_tokens_metric (provider prompt-cache reads) - litellm_input_cache_creation_tokens_metric (Anthropic prompt-cache writes) - litellm_input_audio_tokens_metric (audio input tokens) - litellm_output_reasoning_tokens_metric (reasoning tokens) - litellm_output_audio_tokens_metric (audio output tokens) These are additive — existing input/output/total counters are unchanged, so no dashboards break. Each new counter is only incremented when the underlying detail is populated and > 0, keeping scrape output sparse for providers that don't report a given field. Data is read from the canonical Usage dict that `get_standard_logging_object_payload` already attaches at `standard_logging_payload["metadata"]["usage_object"]`, so no new plumbing through the logging pipeline is required. Tests: 10 new unit tests covering registration, label-set parity, all-types increment, zero/None/negative skip behaviour, and the no-metadata/no-usage_object no-op paths. Closes LIT-3220 Co-authored-by: shin-berri <shin-laptop@berri.ai> Co-authored-by: yuneng-jiang <yuneng@berri.ai> Co-authored-by: Krrish Dholakia <krrishdholakia@berri.ai> Co-authored-by: Claude <noreply@anthropic.com> * chore: remove proof folder image --------- Co-authored-by: oss-agent-shin <ext-agent-shin@berri.ai> Co-authored-by: shin-berri <shin-laptop@berri.ai> Co-authored-by: yuneng-jiang <yuneng@berri.ai> Co-authored-by: Krrish Dholakia <krrishdholakia@berri.ai> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Ishaan Jaffer <ishaanjaffer0324@gmail.com> |
||
|---|---|---|
| .. | ||
| containers | ||
| google_genai | ||
| images | ||
| integrations | ||
| interactions | ||
| litellm_core_utils | ||
| llms | ||
| management_endpoints | ||
| mcp_server | ||
| passthrough_endpoints | ||
| prompts | ||
| proxy | ||
| responses | ||
| secret_managers | ||
| videos | ||
| access_group.py | ||
| adapter.py | ||
| agents.py | ||
| caching.py | ||
| completion.py | ||
| compression.py | ||
| embedding.py | ||
| files.py | ||
| fine_tuning.py | ||
| guardrails.py | ||
| mcp.py | ||
| memory_management.py | ||
| policy_engine.py | ||
| rag.py | ||
| realtime.py | ||
| rerank.py | ||
| router.py | ||
| scheduler.py | ||
| search.py | ||
| services.py | ||
| tag_management.py | ||
| tool_management.py | ||
| utils.py | ||
| vector_store_files.py | ||
| vector_stores.py | ||