mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-12 23:01:41 +00:00
55ec491d03
* fix(otel): bound and shut down credential-scoped tracer providers Each credential-scoped TracerProvider owns a BatchSpanProcessor worker thread that only stops on shutdown, and the v1 cache holding them was an unbounded, unsynchronized dict that never shut anything down. Every distinct team/key credential set therefore added a thread for the life of the process, and concurrent first-requests for the same credential set orphaned duplicate providers outright. Make the cache a lock-guarded bounded LRU that shuts down whatever it drops, matching the v2 TenantTracerCache. Providers wrapping a caller-supplied SpanExporter instance share that exporter with the logger's own provider, so they are dropped without shutdown; those use SimpleSpanProcessor and own no thread. * fix(otel): reclaim dropped providers on a dedicated executor Sustained credential churn queues one blocking shutdown per eviction, so using the shared logging executor let an unreachable tenant endpoint stall unrelated logging work behind the OTLP retry budget. Give provider shutdown its own bounded pool; its threads spawn lazily, so a proxy that never evicts still pays nothing. * fix(otel): decide provider shutdown from the victim, not the evicting request Both dynamic entry points share one provider cache, so it can hold providers of mixed exporter ownership. Reading the ownership flag from the evicting request therefore stopped a shared caller-supplied exporter in one direction, silencing telemetry process-wide, and leaked a BatchSpanProcessor thread in the other. Cache ownership alongside the provider so the drop decision reads the victim's own flag. * fix(otel): honor the widened header mapping type instead of dict only Widening the header parameter to Mapping left the isinstance check on dict, so a non-dict Mapping silently returned no headers at all, which for the OTLP path means an unauthenticated exporter and no traces with nothing raised. The dict branch also returned the caller's own object, and dropping the defensive copy at the call site let that alias reach a long-lived exporter. Match on Mapping and copy. * fix(otel): do not give a provider we may never stop an interpreter-exit hook Every TracerProvider registers an atexit hook by default, and that hook holds a strong reference. Providers wrapping a caller-supplied exporter are dropped without shutdown, so they stayed pinned for the life of the process and then stopped the shared exporter at exit. Tie shutdown_on_exit to ownership: those providers use SimpleSpanProcessor and buffer nothing, so they lose no flush, while providers that own their exporter keep the hook and their exit flush. Also stop the victim the eviction test leaves behind, and trim the added comments. |
||
|---|---|---|
| .. | ||
| arize | ||
| azure_storage | ||
| bitbucket | ||
| cloudzero | ||
| code_interpreter_interception | ||
| compression_interception | ||
| datadog | ||
| dotprompt | ||
| focus | ||
| gcs_bucket | ||
| gcs_pubsub | ||
| gitlab | ||
| langfuse | ||
| levo | ||
| litellm_agent | ||
| mavvrik_focus | ||
| newrelic | ||
| open_telemetry | ||
| opik | ||
| otel | ||
| SlackAlerting | ||
| websearch_interception | ||
| rubrik_test_helpers.py | ||
| test_agentops.py | ||
| test_anthropic_cache_control_hook.py | ||
| test_athina.py | ||
| test_azure_sentinel.py | ||
| test_braintrust_logging.py | ||
| test_braintrust_span_name.py | ||
| test_custom_guardrail.py | ||
| test_custom_guardrail_recursion.py | ||
| test_custom_prompt_management.py | ||
| test_deepeval.py | ||
| test_galileo.py | ||
| test_guardrail_logging_sync.py | ||
| test_helicone.py | ||
| test_langfuse.py | ||
| test_langfuse_otel.py | ||
| test_langsmith_init.py | ||
| test_lunary.py | ||
| test_mlflow.py | ||
| test_openmeter.py | ||
| test_opentelemetry.py | ||
| test_opentelemetry_dynamic_imports.py | ||
| test_opik_utils.py | ||
| test_otel_guardrail_violation_spans.py | ||
| test_otel_team_attributes_matrix.py | ||
| test_prometheus_api_promql_escape.py | ||
| test_prometheus_budget_metric_guard.py | ||
| test_prometheus_budget_metrics_db_lookups.py | ||
| test_prometheus_budget_metrics_timeout.py | ||
| test_prometheus_cache_metrics.py | ||
| test_prometheus_client_ip_user_agent.py | ||
| test_prometheus_custom_metadata_label_counts.py | ||
| test_prometheus_end_user_cardinality.py | ||
| test_prometheus_invalid_key_filtering.py | ||
| test_prometheus_labels.py | ||
| test_prometheus_mcp_tool_metrics.py | ||
| test_prometheus_media_generation_metrics.py | ||
| test_prometheus_metric_name_consistency.py | ||
| test_prometheus_missing_metrics.py | ||
| test_prometheus_none_metadata.py | ||
| test_prometheus_overhead_with_guardrails.py | ||
| test_prometheus_queue_guardrail_metrics.py | ||
| test_prometheus_rate_limit_labels.py | ||
| test_prometheus_remaining_tokens_router_fallback.py | ||
| test_prometheus_service_tier_label.py | ||
| test_prometheus_services.py | ||
| test_prometheus_spend_logs_metadata.py | ||
| test_prometheus_stream_label.py | ||
| test_prometheus_token_detail_metrics.py | ||
| test_prometheus_user_team_metrics.py | ||
| test_prompt_manager_ssti.py | ||
| test_responses_background_cost.py | ||
| test_rubrik.py | ||
| test_s3.py | ||
| test_s3_v2.py | ||
| test_shadow_eval_logger.py | ||
| test_weave_otel.py | ||