litellm/tests/integration/observability
devin-ai-integration[bot] 849f3037b4
fix(langtrace): deliver spans to app.langtrace.ai/api/trace with x-api-key (#43322)
* test(langtrace): integration test for the built-in callback wire (path, x-api-key)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(langtrace): deliver spans to app.langtrace.ai/api/trace with x-api-key

The built-in langtrace callback posted to the dead host langtrace.ai, sent
the key as api_key instead of x-api-key, and let the OTLP endpoint
normalizer append /v1/traces to the complete /api/trace path, so every
export returned 404. Default the host to https://app.langtrace.ai, honor
LANGTRACE_API_HOST for self-hosted servers, pass the key as an exporter
header instead of a process-wide env var, and keep the /api/trace path
unchanged for traces on the langtrace callback only

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(langtrace): keep LANGTRACE_API_HOST that already ends in /api/trace

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(langtrace): deterministic audit inventory for the built-in callback (surfaces, failures, endpoints, chaos)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(langtrace): build the repeated-request body once

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(langtrace): outage cell asserts at-most-once delivery, not span loss

The OTLP HTTP exporter reposts once on ConnectionError and the batch
processor may still be flushing the previous burst when the sink closes,
so whether the outage burst is lost or delivered after revival depends
on timing. The invariant is no duplicate and recovery on the same port

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(langtrace): delivered spans must carry the prompt in the gen_ai.content.prompt event

The upstream echoes the marker into the completion, so a whole-span
match alone would still pass if the prompt event disappeared

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(langtrace): disable model info refresh so the scripted upstream only sees completion requests

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-26 16:31:52 -07:00
..
_s3_v2_support.py fix(s3_v2): upload fresh events first, drop terminal failures and hour-old retries by default, opt-in adaptive concurrency (#43022) 2026-09-26 14:58:28 -07:00
test_bedrock_error_request_id.py test(integration): cover customer reported cache key, cache_control, bedrock request id, responses schema, scim and tag budget contracts (#42785) 2026-09-23 13:19:38 -07:00
test_cache_hit_guardrail_metrics.py fix(proxy): keep deployment labels on cache-hit post_call guardrail rejections (#42780) 2026-09-23 22:48:04 -07:00
test_cache_hit_guardrail_metrics_chaos.py test(guardrails): run the cache-hit redis outage test on the shared owned_redis helper (#42925) 2026-09-24 12:00:10 -07:00
test_callback_delivery.py test(integration): regression tests for August cost tracking and budgeting bugs (#42622) 2026-09-23 04:13:03 +00:00
test_guardrail_effects.py test(integration): regression tests for July cost tracking, budgeting and spend bugs (#42694) 2026-09-23 09:51:56 -07:00
test_langfuse_delivery.py test(integration): run the Langfuse DB-callback test on its own scratch database (#43288) 2026-09-26 00:15:56 -07:00
test_langtrace_delivery.py fix(langtrace): deliver spans to app.langtrace.ai/api/trace with x-api-key (#43322) 2026-09-26 16:31:52 -07:00
test_otel_conversation_id.py feat(otel): emit gen_ai.conversation.id from the caller's session id on v2 LLM spans (#42486) 2026-09-23 00:49:29 -07:00
test_otel_text_completion_choices.py fix(otel): keep text completion choice fields beside the synthesized message (#42537) 2026-09-22 14:24:32 -07:00
test_passthrough_upstream_error_chaos.py fix(ci): stop stale CI reds, keep unit tests off the host env, retry CyberArk policy conflicts (#43294) 2026-09-26 09:25:13 -07:00
test_passthrough_upstream_error_visibility.py fix(passthrough): log upstream 4xx/5xx error bodies and carry them into the failure hook (#42695) 2026-09-23 23:42:24 -07:00
test_presidio_streaming_output.py fix(presidio): mask streamed /v1/messages output when the first upstream read is a keepalive, a data-less ping, or a split utf8 character (#43023) 2026-09-25 00:48:42 -07:00
test_s3_v2_flush_surfaces.py fix(s3_v2): upload fresh events first, drop terminal failures and hour-old retries by default, opt-in adaptive concurrency (#43022) 2026-09-26 14:58:28 -07:00
test_s3_v2_upload_fanout.py fix(s3_v2): upload fresh events first, drop terminal failures and hour-old retries by default, opt-in adaptive concurrency (#43022) 2026-09-26 14:58:28 -07:00
test_straiker_v3_platform.py test(straiker): deterministic integration audit of the v3 platform relay (#42781) 2026-09-23 15:57:51 -07:00