litellm/tests/e2e/logging
yucheng-berri ab6d7578ce
test(e2e): read datadog log delivery back from the real datadog api (#33604)
* test(e2e): read datadog log delivery back from the real datadog api

* test(e2e): compare datadog-read cost with math.isclose, not bit-equality

The response_cost now round-trips through DataDog's attribute indexing
pipeline, whose float serialization is not guaranteed to preserve the
exact bit pattern the proxy shipped. rel_tol=1e-9 (equal to 9 significant
digits) still fails on any real cost discrepancy while tolerating
representation drift. Addresses the Greptile P2 on this PR.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(e2e): widen the duplicate-settle window to 30s for real DataDog

Against the local sink one poll interval (5s) after the first hit was
enough to catch a same-call duplicate, because both events arrived in the
same flush batch. Against real DataDog, ingestion jitter can make one
call's two events searchable tens of seconds apart, so a 5s settle could
let the LIT-4447 duplicate slip past the exactly-one assertion. The reader
now keeps re-reading for DD_SETTLE_SECONDS (default 30s, env-overridable
via E2E_DD_SETTLE_SECONDS) after the first event appears, returning early
only when a duplicate is already visible - more waiting cannot clear it.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 16:06:35 -07:00
..
conftest.py test(e2e): read datadog log delivery back from the real datadog api (#33604) 2026-07-16 16:06:35 -07:00
datadog_reader.py test(e2e): read datadog log delivery back from the real datadog api (#33604) 2026-07-16 16:06:35 -07:00
logging_client.py test(e2e): datadog log delivery for successful chat, messages, and responses (LIT-4447) (#33415) 2026-07-16 09:54:07 -07:00
otel_client.py test(e2e): otel trace completeness on /chat/completions (#33132) 2026-07-13 19:06:12 -07:00
test_datadog_log_e2e.py test(e2e): read datadog log delivery back from the real datadog api (#33604) 2026-07-16 16:06:35 -07:00
test_otel_trace_e2e.py test(e2e): datadog log delivery for successful chat, messages, and responses (LIT-4447) (#33415) 2026-07-16 09:54:07 -07:00
test_prometheus_cardinality_e2e.py test(e2e): close coverage gaps across chat/responses, provider features, batches, prometheus, and langfuse eviction (#32165) 2026-07-04 18:56:52 -07:00