mirror of
https://github.com/BerriAI/litellm.git
synced 2026-08-28 05:25:59 +00:00
0223383d94
* fix(e2e): make the datadog read-back find what DataDog actually indexes Live verification of the merged #33604 against real DataDog (us5) exposed three read-back defects that the local-sink tests could never see; all three fixes are verified against the real API: - Marker search: DataDog consumes the shipped JSON message into the event's attributes and leaves the indexed message EMPTY, so the full-text '"marker"' query matched nothing and every test failed with zero events. The query is now '*:*marker*', which scans all attributes (the marker sits in messages.content); verified to return exactly the event for the call. - Rate limit: the Logs Search API budget is 2 requests per 10s org-wide (x-ratelimit-name logs_public_search_api). Polling at POLL_INTERVAL=5s sat exactly at the limit and the reader hard-failed on the first 429. Searches now pace at DD_SEARCH_INTERVAL (10s default) and a 429 backs off and retries up to 5 times; only non-429 failures stay hard fails. - Envelope status: DataDog re-derives the indexed event status from the parsed payload's status attribute ('success') and normalizes it to its OK severity, so the assertion expects 'ok', not the shipped 'info'. Live run: chat_completions and responses pass every assertion including the exact response-cost cross-check; messages red-pins the LIT-4447 duplicate for real (one call -> two sync-sweep copies + one async batch copy, same request id, confirmed in proxy debug logs). The duplicate is race-dependent, so the pin flickers until #33589 lands. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(e2e): datadog log delivery for streamed chat, messages, and responses Rewritten from the dd-sink version (original #33566) to judge delivery on what real DataDog ingested, matching the merged #33604 conversion: the dd_logs reader searches events back through the Logs Search API and the assertions validate the indexed envelope (source:litellm tag, ok status) and the StandardLoggingPayload fields under the event's attributes. Each streamed test drives one STREAMED call per route, asserts the stream actually streamed (event-stream content type, >0 chunks, no upstream error event), then pins exactly one DataDog event whose payload records stream=true, the aggregated token count, and a response_cost equal to the /spend/logs row for the call - a stream's headers ship before its cost exists, so the spend row is the cross-check anchor, and the spend row and DataDog event must also agree on total_tokens. Coverage registry: adds logging.datadog.stream.exports_metric exercised on chat_completions, messages, and responses. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Update test_datadog_log_e2e.py --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|---|---|---|
| .. | ||
| conftest.py | ||
| datadog_reader.py | ||
| logging_client.py | ||
| otel_client.py | ||
| test_datadog_log_e2e.py | ||
| test_otel_trace_e2e.py | ||
| test_prometheus_cardinality_e2e.py | ||