litellm/tests/integration/observability
yuneng-jiang eb114e638d
test(integration): look up owned cache keys and clean up sweep models (#47255)
Three integration tests diffed the whole shared Redis keyspace (scan_iter or dbsize before and after), so
a sibling test's cache write could satisfy or break them. Each now finds its own entry by a value only it
wrote: the Gemini reasoning test by a per-run Gemini responseId, the max_messages test by a per-test
answer text, and the S3 cache-hit test by its request marker

The sweep auth tests created models through raw /model/new on the shared gateway and never deleted them.
They now create them through Scenario.model, which deletes each model on exit, including when setup fails
2026-10-18 10:27:53 +00:00
..
_azure_storage_support.py fix(azure_storage): name Data Lake objects without base64 padding or slashes (#43914) 2026-09-30 18:17:13 -07:00
_logging_only_scope_support.py test(integration): stop GET /__observations from draining the upstream log (#47256) 2026-10-18 09:58:05 +00:00
_openinference_support.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
_s3_v2_support.py fix(s3_v2): upload fresh events first, drop terminal failures and hour-old retries by default, opt-in adaptive concurrency (#43022) 2026-09-26 14:58:28 -07:00
_ternary_support.py feat(focus): add ternary destination for FOCUS export (#46106) 2026-10-12 22:52:40 -07:00
azure_dispatch_support.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
conftest.py fix(ci): isolate integration workers and repair test fixtures (#45863) 2026-10-11 01:56:02 -07:00
logging_worker_drain_canary.py test: migrate SDK and proxy-booting legacy CircleCI tests to their tiers (#45758) 2026-10-11 00:17:21 -07:00
test_akto_logging_only.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_apply_guardrail_endpoint.py fix(cache): keep cache hits free of unset fields, and follow intentional behavior changes in CircleCI suites (#46359) 2026-10-14 08:56:05 -07:00
test_arize_otel_v2_openinference_chaos.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_arize_otel_v2_openinference_family.py fix(otel): emit OpenInference tool calls and metadata on Arize OTel v2 spans (#43698) 2026-10-08 15:53:07 -07:00
test_arize_otel_v2_openinference_modes.py fix(ci): isolate integration workers and repair test fixtures (#45863) 2026-10-11 01:56:02 -07:00
test_arize_otel_v2_openinference_sad_edge.py fix(otel): emit OpenInference tool calls and metadata on Arize OTel v2 spans (#43698) 2026-10-08 15:53:07 -07:00
test_arize_otel_v2_openinference_spans.py fix(proxy): classify /v1/messages pass-through streams as Anthropic on any host (#45602) 2026-10-09 11:57:03 -07:00
test_azure_content_safety_audit.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_azure_content_safety_dispatch.py fix(guardrails): restore Azure guardrail get_user_prompt dispatch and allow logging (#44067) 2026-10-01 23:30:32 -07:00
test_azure_content_safety_dispatch_resilience.py test(integration): hand every harness server a port from one per-worker block (#46761) 2026-10-15 16:56:40 -07:00
test_azure_content_safety_endpoints.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_azure_storage_chaos.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_azure_storage_client_ttl.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_azure_storage_file_names.py fix(azure_storage): keep client call ids from sharing one Data Lake file (#44099) 2026-10-01 23:23:37 -07:00
test_batch_line_items_cold_storage.py test(integration): fix seven CircleCI integration tests that race or assume a worker spread (#47001) 2026-10-16 22:45:29 +00:00
test_bedrock_converse_guardrail_rewrites.py fix(guardrails)!: preserve Bedrock Converse redactions and honor tool-only scans (#45874) 2026-10-17 20:38:14 -07:00
test_bedrock_converse_stream_post_call_guardrail.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_bedrock_error_request_id.py fix(responses): preserve provider request IDs in streaming failure logs (#45453) 2026-10-16 00:29:11 -07:00
test_bedrock_guardrail_attachments.py fix(ci): isolate integration workers and repair test fixtures (#45863) 2026-10-11 01:56:02 -07:00
test_bedrock_invoke_stream_post_call_guardrail.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_cache_hit_guardrail_metrics.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_cache_hit_guardrail_metrics_chaos.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_callback_delivery.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_content_filter_logging_only.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_custom_logger_redaction.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_datadog_logging_delivery.py test: migrate legacy unit dirs and unrun folders into tests/unit, tests/integration and tests/e2e (#45940) 2026-10-11 06:24:57 -07:00
test_file_usage_counter_span.py feat(proxy): cap batch file records, daily batch uploads, and per-file downloads (#43632) 2026-10-06 03:06:36 +00:00
test_generic_guardrail_api_wire.py fix(guardrails): accept Gemini tools in generic_guardrail_api, add opt-in forward_messages_as_sent, name the error type on fail-open (#46338) 2026-10-16 14:52:35 -07:00
test_generic_guardrail_num_retries.py feat(guardrails): add num_retries to generic_guardrail_api for checks lost after connect (#46965) 2026-10-17 12:22:41 -07:00
test_grayswan_wire.py fix(grayswan): send request conversation and tool calls to post-call monitor (#43770) 2026-09-30 17:52:53 -07:00
test_grayswan_wire_chaos.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_guardrail_attachment.py test: replace 61 live logging and otel tests with offline unit and integration coverage (#45362) 2026-10-08 15:55:57 -07:00
test_guardrail_effects.py test(integration): stop GET /__observations from draining the upstream log (#47256) 2026-10-18 09:58:05 +00:00
test_guardrail_load_balancing.py test: migrate SDK and proxy-booting legacy CircleCI tests to their tiers (#45758) 2026-10-11 00:17:21 -07:00
test_guardrail_stream_scope.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_guardrail_stream_scope_chaos.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_guardrail_stream_scope_matrix.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_guardrail_timeout_all_providers.py feat(guardrails): add Wingback guardrail integration (#46136) 2026-10-12 22:18:10 -07:00
test_langfuse_delivery.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_langsmith_logging_delivery.py test: migrate legacy unit dirs and unrun folders into tests/unit, tests/integration and tests/e2e (#45940) 2026-10-11 06:24:57 -07:00
test_langtrace_delivery.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_logging_only_scope_chaos.py test(integration): stop GET /__observations from draining the upstream log (#47256) 2026-10-18 09:58:05 +00:00
test_logging_only_scope_config.py test(integration): stop GET /__observations from draining the upstream log (#47256) 2026-10-18 09:58:05 +00:00
test_logging_only_scope_runtime.py test(integration): stop GET /__observations from draining the upstream log (#47256) 2026-10-18 09:58:05 +00:00
test_logging_worker_drain.py test: migrate SDK and proxy-booting legacy CircleCI tests to their tiers (#45758) 2026-10-11 00:17:21 -07:00
test_otel_cache_hit_cost.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_otel_conversation_id.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_otel_excluded_services.py test(integration): isolate shared proxy state behind six order-dependent flakes (#46172) 2026-10-13 09:55:11 -07:00
test_otel_excluded_services_matrix.py test(integration): wait for the gen-AI span before reading the excluded-services operator trace (#46846) 2026-10-15 22:35:23 -07:00
test_otel_team_capture_message_content.py feat(otel)!: let a team capture_message_content override the global setting for its OTel v2 destination (#44264) 2026-10-10 13:52:32 -07:00
test_otel_text_completion_choices.py fix(otel): keep text completion choice fields beside the synthesized message (#42537) 2026-09-22 14:24:32 -07:00
test_otel_trace_response_header.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_otel_v1_request_trace.py fix(caching): key response cache by router model group in litellm_metadata (#44542) 2026-10-05 17:06:21 +00:00
test_passthrough_upstream_error_chaos.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_passthrough_upstream_error_visibility.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_presidio_entity_masking.py test(integration): stop GET /__observations from draining the upstream log (#47256) 2026-10-18 09:58:05 +00:00
test_presidio_overlapping_spans.py fix(presidio): stop overlapping entity spans from corrupting numbered PII tokens (#47024) 2026-10-17 11:21:03 -07:00
test_presidio_streaming_output.py fix(anthropic): carry safeguard_results and stop_details in synthesized message_delta events (#46096) 2026-10-17 16:48:37 +00:00
test_prometheus_project_alias_label.py fix(prometheus): skip custom metadata labels a metric already carries (#46267) 2026-10-13 18:20:52 -07:00
test_prometheus_request_metrics.py test: replace 61 live logging and otel tests with offline unit and integration coverage (#45362) 2026-10-08 15:55:57 -07:00
test_prometheus_series_cap.py feat(prometheus): cap series per metric for every labeled metric (#44420) 2026-10-06 01:28:44 +00:00
test_prometheus_series_cap_chaos.py test(integration): give chaos and timing tests a harness API and gate raw signals (#46031) 2026-10-12 05:27:33 -07:00
test_prometheus_shared_custom_labels.py test(integration): wait on the right signal in five flaky integration contracts (#46737) 2026-10-15 13:48:51 -07:00
test_realtime_guardrail_translation.py test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
test_s3_v2_flush_surfaces.py test(integration): look up owned cache keys and clean up sweep models (#47255) 2026-10-18 10:27:53 +00:00
test_s3_v2_partition_granularity.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_s3_v2_streaming_failure_dedupe.py test(integration): cut CircleCI proxy boot time with concurrent worker stacks and single-worker test proxies (#46037) 2026-10-12 09:48:42 -07:00
test_s3_v2_upload_fanout.py test(integration): fix seven CircleCI integration tests that race or assume a worker spread (#47001) 2026-10-16 22:45:29 +00:00
test_signoz_delivery.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_straiker_v3_platform.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_stream_error_alert_api_base.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_ternary_focus_export.py feat(focus): add ternary destination for FOCUS export (#46106) 2026-10-12 22:52:40 -07:00
test_wingback_guardrail.py test(integration): give each owned proxy a private database and Redis by default (#47213) 2026-10-18 03:37:41 +00:00
test_xecguard_wire.py refactor(types): replace Any with proven types in 7 files (#43704) 2026-09-29 06:12:58 -07:00