mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-13 23:11:40 +00:00
45d5153c12
* fix(otel): cap metric attribute cardinality with include/exclude lists OTEL metrics stamped every per-request hidden_params and metadata.* field onto each gen_ai.client.* sample, so near-unique values created one metric time series per request and backends like Splunk Observability Cloud throttled and dropped the data. Add an attributes block under callback_settings.otel with mutually-exclusive include_list (allowlist) and exclude_list (denylist), validated against the known attribute names at startup and applied once to the metric attributes in _record_metrics. Spans are untouched, and with no config every attribute is still emitted so existing setups are unaffected. Resolves LIT-3600 * fix(otel): resolve metric attribute filter from callback_settings The proxy usually constructs the OpenTelemetry logger without forwarding the attributes kwarg, while the filter lives under litellm.callback_settings["otel"]["attributes"]. __init__ only read the kwarg, so the recording instance kept config.attributes=None and shipped metrics at full cardinality even when the filter was configured; a live proxy run exposed this. Fall back to the global at init for the base otel logger, and add a regression test that drives the real success hook through the callback_settings path (the unit tests passed before because they injected the config directly). * fix(otel): reject gen_ai.token.type from metric attribute filter lists gen_ai.token.type was a member of VALID_METRIC_ATTRIBUTE_NAMES, so an operator could list it in include_list or exclude_list and pass startup validation. The attribute is injected into the input/output token series after _filter_metric_attributes runs, so the filter never sees it and the request silently has no effect. Reject it loudly from either list instead, matching the contract that a non-actionable attribute name fails fast rather than falling through to a no-op. It stays a structural discriminator on the token-usage histogram. * fix(otel): resolve metric attribute filter lazily at record time The proxy constructs the OpenTelemetry logger before it populates litellm.callback_settings["otel"]["attributes"], so resolving the filter at __init__ left config.attributes None and shipped metrics at full cardinality. A live proxy run confirmed the leak. Resolve the filter on the first metric record instead, when callback_settings is populated, while still validating an explicit config eagerly so a bad SDK config fails at startup. The regression test now constructs the logger before populating callback_settings to mirror that ordering, so it fails if the filter is resolved too early. * fix(otel): don't cache invalid filter on lazy callback_settings path On the lazy callback_settings resolution path, _ensure_metric_attribute_filter wrote self.config.attributes before validating it. When validation then failed, _metric_attr_filter_resolved stayed False while config.attributes held the bad filter, so the next record skipped the callback_settings re-read and re-raised the stale error indefinitely; fixing the misconfiguration required a restart. Drop the premature write and resolve from the local value. A subsequent record now re-reads callback_settings, so a corrected config takes effect without a restart. The write was dead on the success path anyway, since the resolved frozensets are what the filter reads. * feat(otel-v2): emit the 6 gen_ai.client.* metrics at parity with v1 The v2 OpenTelemetry integration was a span engine: it declared two metric histograms but never created a meter or recorded anything. Bring it to parity with v1 so a v2-default deployment gets bounded metrics. Adds the 4 missing metric names, all 6 histograms, a meter-provider builder that mirrors v1's exporter selection, and a GenAIMetricRecorder that records token usage (split input/output), cost, operation duration, TTFT (streaming), TPOT, and response duration on the success hook. Gated on config.enable_metrics so the default is unchanged. The attribute cardinality filter is reused from v1 by import (no duplication of the valid-name set or validation) and resolved lazily from callback_settings.otel.attributes, matching v1. A misconfigured filter raises out of the recorder; the logger surfaces it once at ERROR and records nothing, rather than silently disabling metrics, and a corrected config recovers without a restart. * test(otel-v2): drop duplicate misconfig logger test (covered in test_otel_v2_logger) |
||
|---|---|---|
| .. | ||
| arize | ||
| azure_storage | ||
| bitbucket | ||
| cloudzero | ||
| compression_interception | ||
| datadog | ||
| dotprompt | ||
| focus | ||
| gcs_bucket | ||
| gcs_pubsub | ||
| gitlab | ||
| langfuse | ||
| levo | ||
| litellm_agent | ||
| newrelic | ||
| open_telemetry | ||
| opik | ||
| otel | ||
| SlackAlerting | ||
| websearch_interception | ||
| rubrik_test_helpers.py | ||
| test_agentops.py | ||
| test_anthropic_cache_control_hook.py | ||
| test_athina.py | ||
| test_azure_sentinel.py | ||
| test_braintrust_logging.py | ||
| test_braintrust_span_name.py | ||
| test_custom_guardrail.py | ||
| test_custom_guardrail_recursion.py | ||
| test_custom_prompt_management.py | ||
| test_deepeval.py | ||
| test_galileo.py | ||
| test_langfuse.py | ||
| test_langfuse_otel.py | ||
| test_langsmith_init.py | ||
| test_mlflow.py | ||
| test_openmeter.py | ||
| test_opentelemetry.py | ||
| test_opentelemetry_dynamic_imports.py | ||
| test_otel_guardrail_violation_spans.py | ||
| test_otel_team_attributes_matrix.py | ||
| test_prometheus_api_promql_escape.py | ||
| test_prometheus_cache_metrics.py | ||
| test_prometheus_client_ip_user_agent.py | ||
| test_prometheus_custom_metadata_label_counts.py | ||
| test_prometheus_end_user_cardinality.py | ||
| test_prometheus_invalid_key_filtering.py | ||
| test_prometheus_labels.py | ||
| test_prometheus_metric_name_consistency.py | ||
| test_prometheus_missing_metrics.py | ||
| test_prometheus_none_metadata.py | ||
| test_prometheus_queue_guardrail_metrics.py | ||
| test_prometheus_rate_limit_labels.py | ||
| test_prometheus_remaining_tokens_router_fallback.py | ||
| test_prometheus_services.py | ||
| test_prometheus_spend_logs_metadata.py | ||
| test_prometheus_stream_label.py | ||
| test_prometheus_token_detail_metrics.py | ||
| test_prometheus_user_team_metrics.py | ||
| test_prompt_manager_ssti.py | ||
| test_responses_background_cost.py | ||
| test_rubrik.py | ||
| test_s3_v2.py | ||
| test_weave_otel.py | ||