litellm/tests/integration/observability
devin-ai-integration[bot] dc95de5209
feat(otel)!: let a team capture_message_content override the global setting for its OTel v2 destination (#44264)
* feat(otel): let a team strip captured message content from its own OTel v2 destination

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(proxy): split the shared callback option conflict check and sync dashboard API types

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(otel): address review feedback on team capture_message_content

Classify capture_message_content as NotSecret in the credential slot registry, inject test exporters through register_exporter_factory instead of patching _exporter_from_spec, type the test helpers, and stop reassigning processor in build_tracer_provider

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(otel): add integration cells for team capture_message_content

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(otel): cover weave_otel and newrelic team destinations in capture_message_content cells

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(otel): stop rejecting client-sent capture_message_content at the request body boundary

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* feat(otel): let a team capture_message_content override the global setting in both directions

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(otel): keep a team's capture_message_content on its own OTel v2 destination

The setting no longer rides the flattened team callback vars, so a sibling destination and a routed provider follow the global mode. Request bodies get the base 400 back, a failure-only entry rejects the setting, and entries for one backend must agree

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(otel): type the capture_message_content callback validation tests

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(otel): type the remaining capture_message_content test signatures

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(otel): mark the new capture_message_content test locals Final

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(otel): default tenant capture to callback_settings.otel and keep CaptureMessageContent formatting bare

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(otel): keep CaptureMessageContent a plain str like main

The str-to-Enum switch changed the public type: the config field's type,
repr and model_dump, a constructor that now raised on unknown values, and
unknown env values coerced to no_content. CaptureMessageContent is the plain
str subclass main ships again, the field is a str, and a
captures_span_content helper replaces the enum property. A test pins that a
published preset's capture mode stays the default for an unowned
destination, which is what main forwards.

* test(otel): drop the module-level CaptureMessageContent import the rewritten test no longer uses

* fix(lint): keep the merged branch within main's type budgets

Freeze the capture-mode settings model (LIT015) and read stored logging
entries through one typed sequence helper, so the conflicting-capture checks
add no unknown types.

* fix(otel): reject a destination capture_message_content while OTel v2 is off

The legacy OTel callbacks never read the per-destination setting, so a team
could register no_content and still export prompts and replies under the
global capture mode. Registration now refuses the var unless the proxy runs
with LITELLM_OTEL_V2=true, the same rule the Arize transport and New Relic
routing vars follow.

* test(otel): type the OTel v2 fixtures and the v2-off regression test

* fix(otel): accept only no_content and span_only on a team or key destination

OTel v2 writes captured content only to span attributes, so event_only
behaved like no_content and span_and_event like span_only on a
destination. The proxy-wide setting keeps all four values

* docs(otel): say an omitted capture setting follows the proxy's own callback for that backend

* test(otel): describe the general capture fallback in the unowned destination test

---------

Co-authored-by: mrinal <mrinal@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: yucheng-berri <yucheng@berri.ai>
Co-authored-by: yucheng-berri <293926326+yucheng-berri@users.noreply.github.com>
2026-10-10 13:52:32 -07:00
..
_azure_storage_support.py fix(azure_storage): name Data Lake objects without base64 padding or slashes (#43914) 2026-09-30 18:17:13 -07:00
_logging_only_scope_support.py feat(guardrails): split logging_only_scope direction from input-failure policy (#45709) 2026-10-10 01:46:39 -07:00
_openinference_support.py fix(otel): emit OpenInference tool calls and metadata on Arize OTel v2 spans (#43698) 2026-10-08 15:53:07 -07:00
_s3_v2_support.py fix(s3_v2): upload fresh events first, drop terminal failures and hour-old retries by default, opt-in adaptive concurrency (#43022) 2026-09-26 14:58:28 -07:00
azure_dispatch_support.py fix(guardrails): restore Azure guardrail get_user_prompt dispatch and allow logging (#44067) 2026-10-01 23:30:32 -07:00
conftest.py test(observability): wait for warm-up spend rows in capped and count only the post-wipe half in X4 (#45006) 2026-10-07 11:29:05 -07:00
test_akto_logging_only.py feat(guardrails): support logging_only mode for the Akto guardrail (#45461) 2026-10-08 17:28:08 -07:00
test_arize_otel_v2_openinference_chaos.py test: fix stale and flaky tests across CircleCI, GHA and Buildkite (#45530) 2026-10-09 11:40:14 -07:00
test_arize_otel_v2_openinference_family.py fix(otel): emit OpenInference tool calls and metadata on Arize OTel v2 spans (#43698) 2026-10-08 15:53:07 -07:00
test_arize_otel_v2_openinference_modes.py fix(otel): emit OpenInference tool calls and metadata on Arize OTel v2 spans (#43698) 2026-10-08 15:53:07 -07:00
test_arize_otel_v2_openinference_sad_edge.py fix(otel): emit OpenInference tool calls and metadata on Arize OTel v2 spans (#43698) 2026-10-08 15:53:07 -07:00
test_arize_otel_v2_openinference_spans.py fix(proxy): classify /v1/messages pass-through streams as Anthropic on any host (#45602) 2026-10-09 11:57:03 -07:00
test_azure_content_safety_audit.py fix(guardrails): scan Responses API input in Azure Text Moderation (#43965) 2026-09-30 22:58:03 -07:00
test_azure_content_safety_dispatch.py fix(guardrails): restore Azure guardrail get_user_prompt dispatch and allow logging (#44067) 2026-10-01 23:30:32 -07:00
test_azure_content_safety_dispatch_resilience.py fix(guardrails): restore Azure guardrail get_user_prompt dispatch and allow logging (#44067) 2026-10-01 23:30:32 -07:00
test_azure_content_safety_endpoints.py fix(guardrails): scan Responses API input in Azure Prompt Shield (#43786) 2026-09-30 22:48:20 -07:00
test_azure_storage_chaos.py fix(azure_storage): keep the DataLakeServiceClient alive until its TTL elapses (#43082) 2026-09-30 18:50:15 -07:00
test_azure_storage_client_ttl.py fix(azure_storage): keep the DataLakeServiceClient alive until its TTL elapses (#43082) 2026-09-30 18:50:15 -07:00
test_azure_storage_file_names.py fix(azure_storage): keep client call ids from sharing one Data Lake file (#44099) 2026-10-01 23:23:37 -07:00
test_bedrock_error_request_id.py test(integration): cover customer reported cache key, cache_control, bedrock request id, responses schema, scim and tag budget contracts (#42785) 2026-09-23 13:19:38 -07:00
test_cache_hit_guardrail_metrics.py fix(proxy): keep deployment labels on cache-hit post_call guardrail rejections (#42780) 2026-09-23 22:48:04 -07:00
test_cache_hit_guardrail_metrics_chaos.py test(observability): wait for warm-up spend rows in capped and count only the post-wipe half in X4 (#45006) 2026-10-07 11:29:05 -07:00
test_callback_delivery.py test(integration): regression tests for August cost tracking and budgeting bugs (#42622) 2026-09-23 04:13:03 +00:00
test_file_usage_counter_span.py feat(proxy): cap batch file records, daily batch uploads, and per-file downloads (#43632) 2026-10-06 03:06:36 +00:00
test_grayswan_wire.py fix(grayswan): send request conversation and tool calls to post-call monitor (#43770) 2026-09-30 17:52:53 -07:00
test_grayswan_wire_chaos.py fix(grayswan): send request conversation and tool calls to post-call monitor (#43770) 2026-09-30 17:52:53 -07:00
test_guardrail_attachment.py test: replace 61 live logging and otel tests with offline unit and integration coverage (#45362) 2026-10-08 15:55:57 -07:00
test_guardrail_effects.py feat(guardrails): split logging_only_scope direction from input-failure policy (#45709) 2026-10-10 01:46:39 -07:00
test_guardrail_stream_scope.py feat(guardrails): per-mode stream_scope with bedrock stream and pass-through fixes (#43801) 2026-10-08 13:46:53 -07:00
test_guardrail_stream_scope_chaos.py feat(guardrails): per-mode stream_scope with bedrock stream and pass-through fixes (#43801) 2026-10-08 13:46:53 -07:00
test_guardrail_stream_scope_matrix.py feat(guardrails): per-mode stream_scope with bedrock stream and pass-through fixes (#43801) 2026-10-08 13:46:53 -07:00
test_guardrail_timeout_all_providers.py perf(ci): isolate and shard integration suites (#44146) 2026-10-10 11:15:19 -07:00
test_langfuse_delivery.py fix(proxy): traceparent/baggage fallback must not override caller metadata (#43688) 2026-10-07 19:24:50 -07:00
test_langtrace_delivery.py test(integration): turn the model-info refresh off in every integration proxy config (#45104) 2026-10-07 11:46:34 -07:00
test_logging_only_scope_chaos.py feat(guardrails): add logging_only_scope to observe input, output, or both (#43695) 2026-10-07 14:03:33 -07:00
test_logging_only_scope_config.py feat(guardrails): split logging_only_scope direction from input-failure policy (#45709) 2026-10-10 01:46:39 -07:00
test_logging_only_scope_runtime.py feat(guardrails): split logging_only_scope direction from input-failure policy (#45709) 2026-10-10 01:46:39 -07:00
test_otel_conversation_id.py test(integration): turn the model-info refresh off in every integration proxy config (#45104) 2026-10-07 11:46:34 -07:00
test_otel_excluded_services.py fix(caching): key response cache by router model group in litellm_metadata (#44542) 2026-10-05 17:06:21 +00:00
test_otel_excluded_services_matrix.py fix(otel): tolerate non-dict callback_settings.otel and ignore bare EXCLUDED_SERVICES env (#44086) 2026-10-02 12:36:46 -07:00
test_otel_team_capture_message_content.py feat(otel)!: let a team capture_message_content override the global setting for its OTel v2 destination (#44264) 2026-10-10 13:52:32 -07:00
test_otel_text_completion_choices.py fix(otel): keep text completion choice fields beside the synthesized message (#42537) 2026-09-22 14:24:32 -07:00
test_otel_v1_request_trace.py fix(caching): key response cache by router model group in litellm_metadata (#44542) 2026-10-05 17:06:21 +00:00
test_passthrough_upstream_error_chaos.py perf(ci): isolate and shard integration suites (#44146) 2026-10-10 11:15:19 -07:00
test_passthrough_upstream_error_visibility.py test(integration): opt the config pass-through spend-log case into auth (#44265) 2026-10-02 23:12:15 +00:00
test_presidio_entity_masking.py test(integration): move legacy proxy, router and Redis tests into tests/integration (#44128) 2026-10-01 23:00:54 -07:00
test_presidio_streaming_output.py fix(presidio): mask streamed /v1/messages output when the first upstream read is a keepalive, a data-less ping, or a split utf8 character (#43023) 2026-09-25 00:48:42 -07:00
test_prometheus_request_metrics.py test: replace 61 live logging and otel tests with offline unit and integration coverage (#45362) 2026-10-08 15:55:57 -07:00
test_prometheus_series_cap.py feat(prometheus): cap series per metric for every labeled metric (#44420) 2026-10-06 01:28:44 +00:00
test_prometheus_series_cap_chaos.py feat(prometheus): cap series per metric for every labeled metric (#44420) 2026-10-06 01:28:44 +00:00
test_s3_v2_flush_surfaces.py fix(s3_v2): upload fresh events first, drop terminal failures and hour-old retries by default, opt-in adaptive concurrency (#43022) 2026-09-26 14:58:28 -07:00
test_s3_v2_partition_granularity.py feat(s3_v2): add s3_partition_granularity option for hourly S3 folders (#43748) 2026-10-01 11:00:21 -07:00
test_s3_v2_streaming_failure_dedupe.py fix(responses): stop SDK retries nesting under router retries, and unbreak CircleCI integration tests (#44791) 2026-10-06 15:57:33 +00:00
test_s3_v2_upload_fanout.py chore(lint): remove the LIT002 mutable-construction rule (#43971) 2026-10-01 12:24:02 -07:00
test_signoz_delivery.py test(integration): turn the model-info refresh off in every integration proxy config (#45104) 2026-10-07 11:46:34 -07:00
test_straiker_v3_platform.py perf(ci): isolate and shard integration suites (#44146) 2026-10-10 11:15:19 -07:00
test_xecguard_wire.py refactor(types): replace Any with proven types in 7 files (#43704) 2026-09-29 06:12:58 -07:00