Max Azatian 5145250d83 fix(langfuse_otel)!: send the completion start time (#45597) ... * fix(langfuse_otel): send the completion start time langfuse_otel never set langfuse.observation.completion_start_time, so Langfuse showed no time to first token for any call, streamed or not. The legacy langfuse callback already sends kwargs["completion_start_time"]. Send the same value as an ISO timestamp with an explicit offset. * fix(langfuse_otel): type kwargs and test against a real span Type the kwargs of _set_langfuse_specific_attributes as dict[str, Any], the declared type of model_call_details, so the new completion start time read adds no unknown-type errors. The tests now check a real SDK span instead of patching safe_set_attribute, and the parameter is typed. * fix(otel): send the completion start time from the v2 Langfuse mapper With LITELLM_OTEL_V2 the langfuse_otel spans come from LangfuseMapper, which had no completion start time either. Carry the payload's completionStartTime on LLMCallSpanData for streamed calls, the same calls that get time_to_first_chunk_seconds, and map it to langfuse.observation.completion_start_time. * fix(langfuse_otel): take the logging kwargs as a read-only Mapping The new kwargs annotation tripped LIT001. Both helpers only read the logging kwargs, so type them Mapping[str, Any] instead of dict. * fix(langfuse_otel): time /v1/messages streams from the call start and send the V2 completion start time like the default emitter * fix(langfuse_otel): time Gemini, Interactions and A2A streams from their first chunk and send the start time as JSON The native Gemini, Interactions and A2A streams never recorded a first chunk, so with completion_start_time now sent both emitters showed TTFT equal to latency on them. They now record it, and Gemini and Interactions take the call's start time like /v1/messages. Both emitters send the completion start time as a JSON string, the form the Langfuse SDK and the langfuse callback send, through one serializer. * fix(langfuse_otel): time A2A streams from the call start and record a cached replay's first event A2A streams took their start time when the iterator was built, so their latency left out the work before it. They now use the call's start time like every other route. Cached /v1/messages stream replays recorded no first event, so their time to first token fell back to the end time. The replay now records it on the first event it yields. The tests use the real Interactions config and typed logger hooks. * fix(a2a): time streamed A2A calls from before the agent card fetch and make the cached replay test clock-free * test(a2a): type the card fetch test and stop rebinding the seed helper's recorder * test(otel): expect completion_start_time and cache_hit in the B3 span keys --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>		2026-10-20 13:52:26 +00:00
..
_azure_storage_support.py	fix(azure_storage): name Data Lake objects without base64 padding or slashes (#43914)	2026-09-30 18:17:13 -07:00
_logging_only_scope_support.py	test(integration): stop GET /__observations from draining the upstream log (#47256)	2026-10-18 09:58:05 +00:00
_openinference_support.py	test(integration): expect the requested model in streamed Responses events on the arize v2 rig (#47407)	2026-10-19 22:06:17 +00:00
_s3_v2_support.py	fix(s3_v2): upload fresh events first, drop terminal failures and hour-old retries by default, opt-in adaptive concurrency (#43022)	2026-09-26 14:58:28 -07:00
_ternary_support.py	feat(focus): add ternary destination for FOCUS export (#46106)	2026-10-12 22:52:40 -07:00
azure_dispatch_support.py	test(integration): give each owned proxy a private database and Redis by default (#47213)	2026-10-18 03:37:41 +00:00
conftest.py	fix(ci): isolate integration workers and repair test fixtures (#45863)	2026-10-11 01:56:02 -07:00
logging_worker_drain_canary.py	test: migrate SDK and proxy-booting legacy CircleCI tests to their tiers (#45758)	2026-10-11 00:17:21 -07:00
test_akto_logging_only.py	test(integration): give each owned proxy a private database and Redis by default (#47213)	2026-10-18 03:37:41 +00:00
test_apply_guardrail_endpoint.py	fix(cache): keep cache hits free of unset fields, and follow intentional behavior changes in CircleCI suites (#46359)	2026-10-14 08:56:05 -07:00
test_arize_otel_v2_openinference_chaos.py	test(integration): give chaos and timing tests a harness API and gate raw signals (#46031)	2026-10-12 05:27:33 -07:00
test_arize_otel_v2_openinference_family.py	fix(langfuse_otel)!: send the completion start time (#45597)	2026-10-20 13:52:26 +00:00
test_arize_otel_v2_openinference_modes.py	fix(ci): isolate integration workers and repair test fixtures (#45863)	2026-10-11 01:56:02 -07:00
test_arize_otel_v2_openinference_sad_edge.py	fix(otel): emit OpenInference tool calls and metadata on Arize OTel v2 spans (#43698)	2026-10-08 15:53:07 -07:00
test_arize_otel_v2_openinference_spans.py	test(integration): remove BUG skips for 3 tests that pass on main (#47364)	2026-10-19 21:39:35 -07:00
test_azure_content_safety_audit.py	test(integration): give each owned proxy a private database and Redis by default (#47213)	2026-10-18 03:37:41 +00:00
test_azure_content_safety_dispatch.py	fix(guardrails): restore Azure guardrail get_user_prompt dispatch and allow logging (#44067)	2026-10-01 23:30:32 -07:00
test_azure_content_safety_dispatch_resilience.py	test(integration): hand every harness server a port from one per-worker block (#46761)	2026-10-15 16:56:40 -07:00
test_azure_content_safety_endpoints.py	test(integration): give each owned proxy a private database and Redis by default (#47213)	2026-10-18 03:37:41 +00:00
test_azure_storage_chaos.py	test(integration): give chaos and timing tests a harness API and gate raw signals (#46031)	2026-10-12 05:27:33 -07:00
test_azure_storage_client_ttl.py	test(integration): give each owned proxy a private database and Redis by default (#47213)	2026-10-18 03:37:41 +00:00
test_azure_storage_file_names.py	fix(azure_storage): keep client call ids from sharing one Data Lake file (#44099)	2026-10-01 23:23:37 -07:00
test_batch_line_items_cold_storage.py	test(integration): fix seven CircleCI integration tests that race or assume a worker spread (#47001)	2026-10-16 22:45:29 +00:00
test_bedrock_converse_guardrail_rewrites.py	fix(guardrails)!: preserve Bedrock Converse redactions and honor tool-only scans (#45874)	2026-10-17 20:38:14 -07:00
test_bedrock_converse_stream_post_call_guardrail.py	test(integration): give each owned proxy a private database and Redis by default (#47213)	2026-10-18 03:37:41 +00:00
test_bedrock_error_request_id.py	fix(responses): preserve provider request IDs in streaming failure logs (#45453)	2026-10-16 00:29:11 -07:00
test_bedrock_guardrail_attachments.py	fix(ci): isolate integration workers and repair test fixtures (#45863)	2026-10-11 01:56:02 -07:00
test_bedrock_invoke_stream_post_call_guardrail.py	test(integration): give each owned proxy a private database and Redis by default (#47213)	2026-10-18 03:37:41 +00:00
test_cache_hit_guardrail_metrics.py	test(integration): give each owned proxy a private database and Redis by default (#47213)	2026-10-18 03:37:41 +00:00
test_cache_hit_guardrail_metrics_chaos.py	test(integration): give each owned proxy a private database and Redis by default (#47213)	2026-10-18 03:37:41 +00:00
test_callback_delivery.py	test(integration): give each owned proxy a private database and Redis by default (#47213)	2026-10-18 03:37:41 +00:00
test_content_filter_logging_only.py	test(integration): give each owned proxy a private database and Redis by default (#47213)	2026-10-18 03:37:41 +00:00
test_custom_logger_redaction.py	test(integration): give each owned proxy a private database and Redis by default (#47213)	2026-10-18 03:37:41 +00:00
test_datadog_logging_delivery.py	test: migrate legacy unit dirs and unrun folders into tests/unit, tests/integration and tests/e2e (#45940)	2026-10-11 06:24:57 -07:00
test_file_usage_counter_span.py	feat(proxy): cap batch file records, daily batch uploads, and per-file downloads (#43632)	2026-10-06 03:06:36 +00:00
test_generic_guardrail_api_wire.py	fix(guardrails): accept Gemini tools in generic_guardrail_api, add opt-in forward_messages_as_sent, name the error type on fail-open (#46338)	2026-10-16 14:52:35 -07:00
test_generic_guardrail_num_retries.py	feat(guardrails): add num_retries to generic_guardrail_api for checks lost after connect (#46965)	2026-10-17 12:22:41 -07:00
test_grayswan_wire.py	fix(grayswan): send request conversation and tool calls to post-call monitor (#43770)	2026-09-30 17:52:53 -07:00
test_grayswan_wire_chaos.py	test(integration): give chaos and timing tests a harness API and gate raw signals (#46031)	2026-10-12 05:27:33 -07:00
test_guardrail_attachment.py	test: replace 61 live logging and otel tests with offline unit and integration coverage (#45362)	2026-10-08 15:55:57 -07:00
test_guardrail_effects.py	test(integration): hold the upstream until the disconnect is settled before reading its spend row (#47303)	2026-10-19 13:20:50 -07:00
test_guardrail_load_balancing.py	test: migrate SDK and proxy-booting legacy CircleCI tests to their tiers (#45758)	2026-10-11 00:17:21 -07:00
test_guardrail_stream_scope.py	test(integration): give each owned proxy a private database and Redis by default (#47213)	2026-10-18 03:37:41 +00:00
test_guardrail_stream_scope_chaos.py	test(integration): remove BUG skips for 3 tests that pass on main (#47364)	2026-10-19 21:39:35 -07:00
test_guardrail_stream_scope_matrix.py	test(integration): give each owned proxy a private database and Redis by default (#47213)	2026-10-18 03:37:41 +00:00
test_guardrail_timeout_all_providers.py	feat(guardrails): add Wingback guardrail integration (#46136)	2026-10-12 22:18:10 -07:00
test_langfuse_delivery.py	test(integration): give each owned proxy a private database and Redis by default (#47213)	2026-10-18 03:37:41 +00:00
test_langsmith_logging_delivery.py	test: migrate legacy unit dirs and unrun folders into tests/unit, tests/integration and tests/e2e (#45940)	2026-10-11 06:24:57 -07:00
test_langtrace_delivery.py	test(integration): give each owned proxy a private database and Redis by default (#47213)	2026-10-18 03:37:41 +00:00
test_logging_only_scope_chaos.py	test(integration): stop GET /__observations from draining the upstream log (#47256)	2026-10-18 09:58:05 +00:00
test_logging_only_scope_config.py	test(integration): stop GET /__observations from draining the upstream log (#47256)	2026-10-18 09:58:05 +00:00
test_logging_only_scope_runtime.py	test(integration): stop GET /__observations from draining the upstream log (#47256)	2026-10-18 09:58:05 +00:00
test_logging_worker_drain.py	test: migrate SDK and proxy-booting legacy CircleCI tests to their tiers (#45758)	2026-10-11 00:17:21 -07:00
test_otel_cache_hit_cost.py	test(integration): give each owned proxy a private database and Redis by default (#47213)	2026-10-18 03:37:41 +00:00
test_otel_conversation_id.py	test(integration): give each owned proxy a private database and Redis by default (#47213)	2026-10-18 03:37:41 +00:00
test_otel_excluded_services.py	test(integration): isolate shared proxy state behind six order-dependent flakes (#46172)	2026-10-13 09:55:11 -07:00
test_otel_excluded_services_matrix.py	test(integration): wait for the gen-AI span before reading the excluded-services operator trace (#46846)	2026-10-15 22:35:23 -07:00
test_otel_team_capture_message_content.py	feat(otel)!: let a team capture_message_content override the global setting for its OTel v2 destination (#44264)	2026-10-10 13:52:32 -07:00
test_otel_text_completion_choices.py	fix(otel): keep text completion choice fields beside the synthesized message (#42537)	2026-09-22 14:24:32 -07:00
test_otel_trace_response_header.py	test(integration): give each owned proxy a private database and Redis by default (#47213)	2026-10-18 03:37:41 +00:00
test_otel_v1_request_trace.py	fix(caching): key response cache by router model group in litellm_metadata (#44542)	2026-10-05 17:06:21 +00:00
test_passthrough_upstream_error_chaos.py	test(integration): give each owned proxy a private database and Redis by default (#47213)	2026-10-18 03:37:41 +00:00
test_passthrough_upstream_error_visibility.py	test(integration): give each owned proxy a private database and Redis by default (#47213)	2026-10-18 03:37:41 +00:00
test_presidio_entity_masking.py	test(integration): stop GET /__observations from draining the upstream log (#47256)	2026-10-18 09:58:05 +00:00
test_presidio_overlapping_spans.py	fix(presidio): stop overlapping entity spans from corrupting numbered PII tokens (#47024)	2026-10-17 11:21:03 -07:00
test_presidio_streaming_output.py	fix(anthropic): carry safeguard_results and stop_details in synthesized message_delta events (#46096)	2026-10-17 16:48:37 +00:00
test_prometheus_deployment_failure_api_provider.py	fix(prometheus): label deployment failures with the deployment's provider instead of None (#47384)	2026-10-19 13:38:26 -07:00
test_prometheus_project_alias_label.py	fix(prometheus): skip custom metadata labels a metric already carries (#46267)	2026-10-13 18:20:52 -07:00
test_prometheus_request_metrics.py	test: replace 61 live logging and otel tests with offline unit and integration coverage (#45362)	2026-10-08 15:55:57 -07:00
test_prometheus_series_cap.py	feat(prometheus): cap series per metric for every labeled metric (#44420)	2026-10-06 01:28:44 +00:00
test_prometheus_series_cap_chaos.py	test(integration): give chaos and timing tests a harness API and gate raw signals (#46031)	2026-10-12 05:27:33 -07:00
test_prometheus_shared_custom_labels.py	test(integration): wait on the right signal in five flaky integration contracts (#46737)	2026-10-15 13:48:51 -07:00
test_realtime_guardrail_translation.py	test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759)	2026-10-11 07:58:53 +00:00
test_s3_v2_flush_surfaces.py	test(integration): look up owned cache keys and clean up sweep models (#47255)	2026-10-18 10:27:53 +00:00
test_s3_v2_partition_granularity.py	test(integration): give each owned proxy a private database and Redis by default (#47213)	2026-10-18 03:37:41 +00:00
test_s3_v2_streaming_failure_dedupe.py	test(integration): cut CircleCI proxy boot time with concurrent worker stacks and single-worker test proxies (#46037)	2026-10-12 09:48:42 -07:00
test_s3_v2_upload_fanout.py	test(integration): fix seven CircleCI integration tests that race or assume a worker spread (#47001)	2026-10-16 22:45:29 +00:00
test_signoz_delivery.py	test(integration): give each owned proxy a private database and Redis by default (#47213)	2026-10-18 03:37:41 +00:00
test_straiker_v3_platform.py	test(integration): give each owned proxy a private database and Redis by default (#47213)	2026-10-18 03:37:41 +00:00
test_stream_error_alert_api_base.py	test(integration): give each owned proxy a private database and Redis by default (#47213)	2026-10-18 03:37:41 +00:00
test_ternary_focus_export.py	feat(focus): add ternary destination for FOCUS export (#46106)	2026-10-12 22:52:40 -07:00
test_wingback_guardrail.py	test(integration): give each owned proxy a private database and Redis by default (#47213)	2026-10-18 03:37:41 +00:00
test_xecguard_wire.py	refactor(types): replace Any with proven types in 7 files (#43704)	2026-09-29 06:12:58 -07:00