litellm/tests/e2e/logging
paveln83 05518975a5
test(e2e): move audio, realtime, and Langfuse tests off model ids with shutdown dates (#51816)
* test(e2e): move audio and realtime tests off OpenAI model ids with shutdown dates

* test(e2e): move langfuse completion, image, and vertex realtime ids off dated models

* test(e2e): register the weather tool in the first realtime session update and read Langfuse v2 flattened attributes

* test(e2e): expect the OCR upload's file summary in the Langfuse generation input

The OCR upload route has sent the uploaded file, not a data URL, since the native OCR route, and the generation input summarizes that form as the mime type and file name. The broken Langfuse lookup hid the stale data URL expectation

* test(e2e): move the image generation test off gpt-image-1-mini

gpt-image-1-mini has a 2026-12-01 deprecation_date, so the image generation e2e would start failing that day. gpt-image-2.5-flare has no date and already serves the Langfuse image tests

* test(e2e): use gpt-image-2.6-flare and check every n=2 choice reaches Langfuse

OpenAI names gpt-image-2.6-flare as the successor of gpt-image-1-mini, so the image tests use it. The n=2 completions test now needs one logged message per choice, since both choices repeat the same text and a single logged message used to pass

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-11-14 09:51:53 -05:00
..
conftest.py test(e2e): add logging e2e coverage (s3_v2, gcs_bucket, team langfuse callback, datadog failure) (#38552) 2026-08-29 09:43:44 -07:00
datadog_reader.py test(e2e): tag guardrails and logging tests with Subject metadata and record client steps (#44963) 2026-10-07 10:32:25 -07:00
gcs_reader.py test(e2e): tag guardrails and logging tests with Subject metadata and record client steps (#44963) 2026-10-07 10:32:25 -07:00
logging_client.py test(e2e): move audio, realtime, and Langfuse tests off model ids with shutdown dates (#51816) 2026-11-14 09:51:53 -05:00
s3_reader.py test(e2e): tag guardrails and logging tests with Subject metadata and record client steps (#44963) 2026-10-07 10:32:25 -07:00
test_datadog_log_e2e.py test(e2e): tag guardrails and logging tests with Subject metadata and record client steps (#44963) 2026-10-07 10:32:25 -07:00
test_gcs_log_e2e.py test(e2e): tag guardrails and logging tests with Subject metadata and record client steps (#44963) 2026-10-07 10:32:25 -07:00
test_langsmith_batch_serialization_e2e.py fix(e2e): restore main qualification across providers and UI (#49763) 2026-11-03 20:38:47 +00:00
test_otel_trace_e2e.py fix(e2e): restore main qualification across providers and UI (#49763) 2026-11-03 20:38:47 +00:00
test_otel_v2_langfuse_generation_output_e2e.py test(e2e): move audio, realtime, and Langfuse tests off model ids with shutdown dates (#51816) 2026-11-14 09:51:53 -05:00
test_prometheus_cardinality_e2e.py test(e2e): tag guardrails and logging tests with Subject metadata and record client steps (#44963) 2026-10-07 10:32:25 -07:00
test_prometheus_queue_time_e2e.py test(e2e): tag guardrails and logging tests with Subject metadata and record client steps (#44963) 2026-10-07 10:32:25 -07:00
test_responses_langfuse_usage_e2e.py fix(langfuse): log Responses API token usage instead of zeros (#42212) 2026-10-10 16:42:59 -07:00
test_s3_log_e2e.py test(e2e): tag guardrails and logging tests with Subject metadata and record client steps (#44963) 2026-10-07 10:32:25 -07:00
test_team_langfuse_callback_e2e.py test(e2e): tag guardrails and logging tests with Subject metadata and record client steps (#44963) 2026-10-07 10:32:25 -07:00