litellm/tests/integration/observability
yuneng-jiang 46913be92b
test(integration): wait on the right signal in five flaky integration contracts (#46737)
* test(integration): wait on the right signal in five flaky integration contracts

The Prometheus shared-label tests waited for the latency metric and then read the
request counter, which the logger writes after an await. They now wait for the counter

The auto-router session poll returned on the first 200 while the caller expected two
turns, so it now waits for the expected turn count

The S3 queue cap test let its failing sink recover on a 15s clock that could expire
mid-flush. The sink now stays down until a flush fails after every event is queued

K3 normally takes 168-175s against a 180s limit, so the limit is now 300s

The MCP env var browser spec reused an idle keep-alive socket that the proxy closes
after 5s, so its API calls now send Connection: close

* test(integration): add until_asserted so a wait polls the exact asserts it guards

eventually() takes a predicate separate from the asserts that follow it, which lets a
test wait on one value and then assert on another written later. until_asserted reruns
a block of asserts until they all pass and re-raises the last failure on timeout

The Prometheus shared-label tests and the auto-router session check now use it
2026-10-15 13:48:51 -07:00
..
_azure_storage_support.py
_logging_only_scope_support.py
_openinference_support.py fix(cache): keep cache hits free of unset fields, and follow intentional behavior changes in CircleCI suites (#46359) 2026-10-14 08:56:05 -07:00
_s3_v2_support.py
_ternary_support.py feat(focus): add ternary destination for FOCUS export (#46106) 2026-10-12 22:52:40 -07:00
azure_dispatch_support.py
conftest.py
logging_worker_drain_canary.py
test_akto_logging_only.py
test_apply_guardrail_endpoint.py fix(cache): keep cache hits free of unset fields, and follow intentional behavior changes in CircleCI suites (#46359) 2026-10-14 08:56:05 -07:00
test_arize_otel_v2_openinference_chaos.py
test_arize_otel_v2_openinference_family.py
test_arize_otel_v2_openinference_modes.py
test_arize_otel_v2_openinference_sad_edge.py
test_arize_otel_v2_openinference_spans.py
test_azure_content_safety_audit.py
test_azure_content_safety_dispatch.py
test_azure_content_safety_dispatch_resilience.py
test_azure_content_safety_endpoints.py
test_azure_storage_chaos.py
test_azure_storage_client_ttl.py
test_azure_storage_file_names.py
test_batch_line_items_cold_storage.py feat(batches): store each batch line's request and response in S3 and GCS (#41691) 2026-10-15 11:32:56 -07:00
test_bedrock_error_request_id.py
test_bedrock_guardrail_attachments.py
test_cache_hit_guardrail_metrics.py
test_cache_hit_guardrail_metrics_chaos.py
test_callback_delivery.py
test_custom_logger_redaction.py fix(logging): redact nested callback input and provider response fields (#45923) 2026-10-13 12:25:56 -07:00
test_datadog_logging_delivery.py
test_file_usage_counter_span.py
test_grayswan_wire.py
test_grayswan_wire_chaos.py
test_guardrail_attachment.py
test_guardrail_effects.py test(integration): cover chat_completions endpoint contracts (#44688) 2026-10-14 03:45:38 +00:00
test_guardrail_load_balancing.py
test_guardrail_stream_scope.py
test_guardrail_stream_scope_chaos.py
test_guardrail_stream_scope_matrix.py test(integration): update stale CircleCI integration expectations (#46570) 2026-10-14 18:21:59 -07:00
test_guardrail_timeout_all_providers.py feat(guardrails): add Wingback guardrail integration (#46136) 2026-10-12 22:18:10 -07:00
test_langfuse_delivery.py test(integration): isolate shared proxy state behind six order-dependent flakes (#46172) 2026-10-13 09:55:11 -07:00
test_langsmith_logging_delivery.py
test_langtrace_delivery.py
test_logging_only_scope_chaos.py test(integration): wait on the right signal in five flaky integration contracts (#46737) 2026-10-15 13:48:51 -07:00
test_logging_only_scope_config.py test(integration): update stale CircleCI integration expectations (#46570) 2026-10-14 18:21:59 -07:00
test_logging_only_scope_runtime.py
test_logging_worker_drain.py
test_otel_conversation_id.py
test_otel_excluded_services.py test(integration): isolate shared proxy state behind six order-dependent flakes (#46172) 2026-10-13 09:55:11 -07:00
test_otel_excluded_services_matrix.py
test_otel_team_capture_message_content.py
test_otel_text_completion_choices.py
test_otel_trace_response_header.py feat(proxy): return the W3C traceresponse header on traced responses (#42079) 2026-10-14 18:12:34 -07:00
test_otel_v1_request_trace.py
test_passthrough_upstream_error_chaos.py
test_passthrough_upstream_error_visibility.py test(integration): cover pass_through endpoint contracts (#44690) 2026-10-14 07:38:17 +00:00
test_presidio_entity_masking.py test(integration): cover guardrails_policies endpoint contracts (#44722) 2026-10-14 04:07:26 +00:00
test_presidio_streaming_output.py
test_prometheus_project_alias_label.py fix(prometheus): skip custom metadata labels a metric already carries (#46267) 2026-10-13 18:20:52 -07:00
test_prometheus_request_metrics.py
test_prometheus_series_cap.py
test_prometheus_series_cap_chaos.py
test_prometheus_shared_custom_labels.py test(integration): wait on the right signal in five flaky integration contracts (#46737) 2026-10-15 13:48:51 -07:00
test_realtime_guardrail_translation.py
test_s3_v2_flush_surfaces.py
test_s3_v2_partition_granularity.py
test_s3_v2_streaming_failure_dedupe.py
test_s3_v2_upload_fanout.py test(integration): wait on the right signal in five flaky integration contracts (#46737) 2026-10-15 13:48:51 -07:00
test_signoz_delivery.py test(integration): remove BUG skips for 3 tests that pass on main (#46725) 2026-10-15 18:50:26 +00:00
test_straiker_v3_platform.py
test_ternary_focus_export.py feat(focus): add ternary destination for FOCUS export (#46106) 2026-10-12 22:52:40 -07:00
test_wingback_guardrail.py feat(guardrails): add Wingback guardrail integration (#46136) 2026-10-12 22:18:10 -07:00
test_xecguard_wire.py