mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-29 06:33:28 +00:00
1171273511
* fix(langfuse_otel): emit request metadata under langfuse.observation.metadata and langfuse.trace.metadata keys Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(langfuse_otel): name trace metadata identity keys after their request metadata fields Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(langfuse_otel): type metadata helper and drop null fields so they cannot clobber caller trace metadata Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(langfuse_otel): annotate new request metadata tests Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(langfuse_otel): build rig config and wire batches immutably Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(langfuse_otel): skip empty trace identity values so teamless requests cannot erase attribution Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(langfuse_otel): drain wire batches per poll instead of accumulating Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(langfuse_otel): cover null stripping and empty trace identities Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(langfuse_otel): set redaction flag directly instead of monkeypatching Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(langfuse_otel): add integration audit matrix for langfuse metadata keys Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(langfuse_otel): tighten audit chaos and unauthenticated cells Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(langfuse_otel): restore contracts.json formatting Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(langfuse_otel): close SDK clients, pin worker identity, and normalize response id matching Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(langfuse_otel): gate the slow sink reply instead of sleeping Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(otel v2): emit request metadata and identity under langfuse.observation.metadata and langfuse.trace.metadata Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(otel v2): satisfy type discipline gate for metadata export Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test: drop prohibited covers markers from langfuse otel integration tests Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(otel): put the mutable-ok suppression on the dict construction line the type-discipline gate reports Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(langfuse otel): share request metadata export across callbacks * fix(langfuse otel): drop prompt variables, MCP tool I/O and vector store queries from exported request metadata * fix(langfuse_otel): keep span export alive when nested request metadata has non-string keys The content strip validated prompt_management_metadata, mcp_tool_call_metadata and vector_store_request_metadata items as str-keyed mappings, so an SDK caller passing one with a non-string key raised inside exported_request_metadata: OTEL v2 dropped the LLM span for every vendor and v1 langfuse_otel lost the request metadata and every attribute after it. Validate those nested mappings with object keys instead. Also hoists the langfuse_otel imports. * fix(langfuse_otel): send the request metadata blob to Langfuse destinations only - A requester_metadata that is not a mapping no longer raises. The v2 LLM span (every vendor) and the v1 Langfuse attributes survive it again. - Prompt-quoting routing_decision fields stay out of langfuse.observation.metadata. - An exporter or key/team destination for any other backend drops langfuse.observation.metadata and langfuse.trace.metadata.user_api_key_*, as on main. This covers a collector folded into the Langfuse logger. - The v2 Langfuse key-set pin in the arize family test gains the two new keys. * fix(langfuse_otel): export only caller identity under the Langfuse metadata keys - The Langfuse metadata keys carried the whole request metadata. On OTel v2 that sent request headers, caller metadata, user email, user agent and auth metadata to a Langfuse destination that never received them on main; on v1 it doubled each span's metadata payload, so big caller metadata doubled the export size - Both paths now export only the key alias, user, end user, team, team alias and org. On v2 that is the team plus the identity among the operator's `baggage_metadata_keys`, which v2 spans already carry as `litellm.metadata.*`. Everything else stays in the bare `metadata` attribute, as on main - `output_messages` returns a list again, and the arize v2 b3 key pin is back to main's set * fix(langfuse_otel): drop the caller identity first when a span hits its attribute limit v1 wrote the identity attributes after the OpenInference ones, so a span near the 128-attribute limit (about 33+ tools) evicted openinference.span.kind, metadata and llm.model_name instead. The identity now goes first on a fresh span, so the SDK evicts it before anything else, and a caller's active span that already carries attributes does not get it. v2 sheds the Langfuse identity keys before any indexed message attribute when a span is over budget, so a span keeps every attribute it carried without them. * fix(langfuse_otel): keep other spans and dropped counts as main exports them - v1 writes the caller identity only into the room the span's attribute limit leaves next to the error attributes a failed call adds, so the exported dropped-attribute count matches main - v2 strips the Langfuse request metadata only from LiteLLM's own spans; application, instrumentation and server spans keep every key they set - a stripped v2 span keeps the dropped attribute, event and link counts - a config that lists the Langfuse mapper without the langfuse_otel exporter also keeps the request metadata away from other exporters * fix(langfuse_otel): keep the caller identity off the caller's active span v1 writes LiteLLM's attributes onto the caller's active span when one is current instead of starting its own span. That span goes to the caller's own exporter, so the identity keys reached a non-Langfuse collector and could take attribute slots the caller fills later. The identity keys now go only on the span LiteLLM starts, whatever the caller's span carries. * fix(langfuse otel): keep the identity keys on the OTEL v2 Langfuse mapper only The v1 langfuse_otel callback goes back to main: it sends no langfuse.observation.metadata or langfuse.trace.metadata.user_api_key_* keys. The OTEL v2 Langfuse mapper keeps them. The integration cells now run every endpoint, client, redaction, failure and chaos case on v2. * test(langfuse otel): count only accepted batches in the sink outage cell The wire also records the batches the sink answered 503, so an exporter retry made a span look exported twice. The chaos cells now also check the spans that landed carry the caller identity. * test(langfuse otel): split stacked span comprehensions and drop constant comments * test(langfuse otel): split the metadata integration module into focused files * fix(langfuse otel): send the org id in trace metadata too The trace metadata keys now reuse IDENTITY_METADATA_FIELDS, so a key in an org gets langfuse.trace.metadata.user_api_key_org_id next to the generation's org id. --------- Co-authored-by: yucheng <yucheng@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| _azure_storage_support.py | ||
| _langfuse_otel_metadata_support.py | ||
| _logging_only_scope_support.py | ||
| _openinference_support.py | ||
| _s3_v2_support.py | ||
| _ternary_support.py | ||
| azure_dispatch_support.py | ||
| conftest.py | ||
| logging_worker_drain_canary.py | ||
| test_akto_logging_only.py | ||
| test_apply_guardrail_endpoint.py | ||
| test_arize_otel_v2_openinference_chaos.py | ||
| test_arize_otel_v2_openinference_family.py | ||
| test_arize_otel_v2_openinference_modes.py | ||
| test_arize_otel_v2_openinference_sad_edge.py | ||
| test_arize_otel_v2_openinference_spans.py | ||
| test_azure_content_safety_audit.py | ||
| test_azure_content_safety_dispatch.py | ||
| test_azure_content_safety_dispatch_resilience.py | ||
| test_azure_content_safety_endpoints.py | ||
| test_azure_storage_chaos.py | ||
| test_azure_storage_client_ttl.py | ||
| test_azure_storage_file_names.py | ||
| test_batch_line_items_cold_storage.py | ||
| test_bedrock_converse_guardrail_rewrites.py | ||
| test_bedrock_converse_stream_post_call_guardrail.py | ||
| test_bedrock_error_request_id.py | ||
| test_bedrock_guardrail_attachments.py | ||
| test_bedrock_invoke_stream_post_call_guardrail.py | ||
| test_cache_hit_guardrail_metrics.py | ||
| test_cache_hit_guardrail_metrics_chaos.py | ||
| test_callback_delivery.py | ||
| test_content_filter_logging_only.py | ||
| test_custom_logger_hook_counts.py | ||
| test_custom_logger_hook_counts_chaos.py | ||
| test_custom_logger_hook_counts_modes.py | ||
| test_custom_logger_hook_counts_post_call_block.py | ||
| test_custom_logger_hook_counts_vendors.py | ||
| test_custom_logger_redaction.py | ||
| test_datadog_logging_delivery.py | ||
| test_file_usage_counter_span.py | ||
| test_generic_guardrail_api_wire.py | ||
| test_generic_guardrail_num_retries.py | ||
| test_grayswan_wire.py | ||
| test_grayswan_wire_chaos.py | ||
| test_guardrail_attachment.py | ||
| test_guardrail_effects.py | ||
| test_guardrail_load_balancing.py | ||
| test_guardrail_stream_scope.py | ||
| test_guardrail_stream_scope_chaos.py | ||
| test_guardrail_stream_scope_matrix.py | ||
| test_guardrail_timeout_all_providers.py | ||
| test_langfuse_delivery.py | ||
| test_langfuse_otel_metadata_chaos.py | ||
| test_langfuse_otel_metadata_clients.py | ||
| test_langfuse_otel_metadata_destinations.py | ||
| test_langfuse_otel_metadata_identity.py | ||
| test_langfuse_otel_trace_fields.py | ||
| test_langsmith_logging_delivery.py | ||
| test_langtrace_delivery.py | ||
| test_logging_only_scope_chaos.py | ||
| test_logging_only_scope_config.py | ||
| test_logging_only_scope_runtime.py | ||
| test_logging_worker_drain.py | ||
| test_otel_cache_hit_cost.py | ||
| test_otel_conversation_id.py | ||
| test_otel_excluded_services.py | ||
| test_otel_excluded_services_matrix.py | ||
| test_otel_team_capture_message_content.py | ||
| test_otel_team_sampling_across_pods.py | ||
| test_otel_text_completion_choices.py | ||
| test_otel_trace_response_header.py | ||
| test_otel_v1_request_trace.py | ||
| test_passthrough_guardrail_scan_text.py | ||
| test_passthrough_upstream_error_chaos.py | ||
| test_passthrough_upstream_error_visibility.py | ||
| test_presidio_entity_masking.py | ||
| test_presidio_overlapping_spans.py | ||
| test_presidio_streaming_output.py | ||
| test_prometheus_deployment_failure_api_provider.py | ||
| test_prometheus_project_alias_label.py | ||
| test_prometheus_provider_auth_failure.py | ||
| test_prometheus_provider_auth_failure_chaos.py | ||
| test_prometheus_request_metrics.py | ||
| test_prometheus_series_cap.py | ||
| test_prometheus_series_cap_chaos.py | ||
| test_prometheus_shared_custom_labels.py | ||
| test_realtime_guardrail_translation.py | ||
| test_s3_v2_flush_surfaces.py | ||
| test_s3_v2_partition_granularity.py | ||
| test_s3_v2_streaming_failure_dedupe.py | ||
| test_s3_v2_upload_fanout.py | ||
| test_signoz_delivery.py | ||
| test_straiker_v3_platform.py | ||
| test_stream_error_alert_api_base.py | ||
| test_ternary_focus_export.py | ||
| test_wingback_guardrail.py | ||
| test_xecguard_wire.py | ||