Commit graph

2 commits

Author SHA1 Message Date
mateo-berri
df60e36d07 fix(responses): end stream cleanly on transport error after terminal event 2026-07-15 08:33:50 -07:00
yucheng-berri
8449ecee6a
fix(streaming): stamp completion_start_time on first chunk for /v1/messages and /v1/responses (#32284)
Streaming pass-through for native Anthropic /v1/messages and the /v1/responses
streaming iterator never set logging_obj.completion_start_time, so
_success_handler_helper_fn fell back to completion_start_time = end_time.
Downstream TTFT consumers (Prometheus, OTEL, Langfuse, Admin UI, spend logs
completionStartTime) then reported time-to-first-token equal to total request
duration.

Stamp completion_start_time on the first chunk in PassThroughStreamingHandler.
chunk_processor and BaseResponsesAPIStreamingIterator._process_chunk, mirroring
CustomStreamWrapper for /chat/completions.

Resolves LIT-4185

Co-authored-by: yucheng <yucheng@yuchengs-MBP.localdomain>
2026-07-06 19:30:31 -07:00