mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-30 06:43:27 +00:00
38f318e736
* fix(otel): decide per-team Arize sampling from the trace id so every pod agrees * test(otel): cover per-team sampling across pods, workers, endpoints and restarts * fix(otel): remember only a failed tree's sampling verdict so a later failed call answers to the error rate A successful tree's verdict follows from the trace id alone, so remembering it only made a later failed call of the same trace follow the success rate on the pod that served both, while other pods judged it at the error rate * test(otel): flush every worker before the restart cell kills it and cover a failed call after a dropped one * fix(otel): decide team sampling per request, not per trace Calls sharing a caller's trace id are separate requests, keyed by their root span. Each is decided by the trace hash at the rate for its own outcome, and only its own late spans follow its verdict, so no pod applies an earlier call's verdict to a later call of the trace * fix(otel): forget a span's request root when it ends, not when newer spans start The root of each open span was kept in a first-in-first-out table of 16384 entries, so a span two levels below the root that stayed open while the worker started more spans than that lost its entry and ended under its parent's id. Its request then never closed and was held from every sampled destination until shutdown. Each open span now keeps its entry until it ends, and a span starting under a parent that has already ended reads the root from the parent span itself * fix(otel): decide a request as its spans end once it falls out of the open span table A request whose span was pushed out of the table of open spans, because the worker held more than 16384 open at once, ended that span under another key and was held from every sampled destination until shutdown. Pushing a span out now drops its request's open count too, so the request is decided as each of its spans ends, the way one pushed out of the open request table already is * test(otel): annotate the team sampling test locals as Final * test(otel): build the caller's remote parent in one helper instead of rebinding it per trace * fix(otel): keep each open span's request until it ends, bounded by the open requests The span table had its own bound of 16384, so a worker holding more spans open than that pushed out the oldest, and a failure in such a span no longer reached its request. Each request now holds the ids of its open spans, and a span's entry in the lookup goes when the span ends or when its request is pushed out of the open request table, the same bound main already applies. A failure anywhere in a tracked request reaches its verdict however many spans are open --------- Co-authored-by: yucheng-berri <293926326+yucheng-berri@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| __init__.py | ||
| test_db_endpoint.py | ||
| test_langfuse_logger.py | ||
| test_otel_v2_baggage.py | ||
| test_otel_v2_components.py | ||
| test_otel_v2_config_baggage_parenting_guardrails.py | ||
| test_otel_v2_destinations.py | ||
| test_otel_v2_dynamic.py | ||
| test_otel_v2_emitter.py | ||
| test_otel_v2_logger.py | ||
| test_otel_v2_metrics.py | ||
| test_otel_v2_mount.py | ||
| test_otel_v2_multibackend.py | ||
| test_otel_v2_presets.py | ||
| test_otel_v2_sources_of_truth.py | ||
| test_otel_v2_vendor_mappers.py | ||
| test_runtime.py | ||