From e319bf270cb65e1324a57aa6f971e7d664367c62 Mon Sep 17 00:00:00 2001 From: yucheng-berri Date: Thu, 24 Sep 2026 23:22:56 -0700 Subject: [PATCH] feat(langfuse): migrate the sdk callback to langfuse v4 (#36741) * feat(langfuse): migrate the sdk callback to langfuse v4 Replace the v2 trace()/generation()/span() calls with SDK v4 observations exported over OpenTelemetry, with one isolated tracer provider per Langfuse credential set, a discarding exporter for mock mode, and v4 trace and observation id normalization. Keeps the session-header trace provenance logic from main so each call under a session alias still gets its own trace Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(langfuse): drop the always-true prompt client check now that v4 get_prompt is non-optional Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(langfuse): isolate the e2e sync test from cached clients and log the real sdk major Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(langfuse): type the slack trace-url lookup and drop dead v2 test shims Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(slack): cover the langfuse trace url built from the logger host Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * build(docker): pin langfuse to the locked 4.15.2 in the pip image Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(langfuse): hash all-zero trace and observation ids instead of passing them through Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * feat(langfuse): honour caller generation ids and assert v4 OTLP exports in legacy tests v2 accepted generation(id=...). v4 derives the observation id from the OTel span id, so the isolated tracer provider now carries an id generator that hands out the id start_generation asked for through a context variable, and the callback passes the resolved generation_id metadata into it. The legacy e2e suite patched httpx.Client.post and compared v2 ingestion batches; it now patches requests.Session.post, decodes the OTLP protobuf and compares the exported generation against regenerated fixtures. The local readback test replaces the removed get_generations() with api.observations.get_many() and polls Langfuse Cloud instead of sleeping. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(langfuse): read the sdk version header from package metadata Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(langfuse): propagate trace_metadata as trace-level attributes in v4 v2 wrote trace(metadata=...) onto the trace object. In v4 the trace only carries what the observations propagate, so a continuation request with update_trace_keys=["trace_metadata"] updated the generation's metadata while the trace kept its stale values. Coerce each entry to the SDK's string limit and hand it to propagate_attributes(metadata=...). Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(langfuse): propagate interrupts raised during deferred client teardown Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(langfuse): honor ssl_verify=False and SSL_VERIFY on the v4 OTLP exporter Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(langfuse): fall back to the default CA when the configured bundle path is missing Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(langfuse): renew the client when eviction lands before the callback lease The cache can evict a logger between handing it to the callback and the callback taking its lease. Such a lease now hands back a fresh client acquired through the same parameters, so that callback exports through a live tracer provider instead of one teardown already shut down. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(langfuse): emit litellm_call_id and response_id as generation metadata v2 put the provider response id inside the generation id. v4 observation ids are 16 hex chars derived from that string, so the ids move to generation metadata to keep generations searchable by response id Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(langfuse): read the response id through a typed protocol Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(langfuse): do not claim trace root when continuing an existing trace Langfuse derives a trace's name and I/O from any observation flagged langfuse.internal.as_root, so a request carrying existing_trace_id renamed the trace to the generation name and replaced the trace input and output on every continuation. v2 only updated the keys listed in update_trace_keys. Continuations now export as plain children of the remote parent and keep the explicit langfuse.trace.* attributes for the fields they do want changed. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(langfuse): iterate lease renewal instead of recursing, monkeypatch update_trace_keys flag in tests The recursive lease fallback tripped tests/code_coverage_tests/recursive_detector.py; the renewal candidates are now walked with itertools.chain. The six update_trace_keys tests set the litellm global through pytest monkeypatch so the TQ008 budget stays within its ceiling Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(langfuse): retry raised OTLP exports and honor LANGFUSE_TIMEOUT The OTLP http exporter only retries 429 and 5xx; a connect or read timeout propagates and BatchSpanProcessor drops the batch. Wrap the exporter in RetryingSpanExporter (three backoff retries, as the v2 consumer did) and build it on every path so the default and private-CA deployments share the same channel, timeout and retry behaviour Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(langfuse): sample on a hash of the full trace id and tolerate bad LANGFUSE_SAMPLE_RATE TraceIdRatioBased reads the low 64 bits of the trace id. litellm trace ids are UUIDs, whose variant bits sit at the top of that word, so every fractional rate up to 0.5 dropped all traces. A SHA-256 of the full id gives an unbiased, deterministic decision. Values outside [0, 1] or non numeric now warn and export everything instead of raising during callback construction, which surfaced as a 500 on the first request of each worker Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(langfuse): put the Langfuse trace link back into Slack alerts The proxy registers LangfusePromptManagement for callbacks: ["langfuse"], so the alert helper never saw the literal "langfuse" string and returned before looking up the trace id, and the prompt management logger never stored the trace id it got back from log_event_on_langfuse. Recognize LangFuseLogger instances in the callback list, record the returned trace id in the shared service trace id cache, and skip the link when no trace id arrives Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * chore(deps): relock langfuse 4.15.2 and opentelemetry 1.33.1 on current main Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * chore(langfuse): mark the deliberate blind except in client teardown for the strict ruff gate Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(langfuse): pass the resource attributes mapping straight to Resource.create Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(langfuse): warn about ignored UPSTREAM_LANGFUSE_* on the shared client init path too Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(langfuse): normalise the OTLP export path so a trailing host slash never yields a double slash Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(langfuse): nest guardrail and grounding spans under the generation Langfuse v4 derives the trace name and I/O from every observation marked as_root, and the one with the latest start time wins. Guardrail and grounding spans used to claim root next to the generation, so a post_call guardrail could replace the model's request and response on the trace with its own. Only the generation claims root now; the sibling spans become its children Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(langfuse): rebuild the cached bundle when mock mode or sample rate changes The SDK keys resource bundles on the public key alone, so a bundle built with the discarding exporter for LANGFUSE_MOCK, or with an earlier LANGFUSE_SAMPLE_RATE, was handed back to a client that asked for a live exporter or a different rate. Compare both when deciding whether the cached bundle is still valid Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(langfuse): keep trace_public true when a guardrail span is exported Langfuse folds langfuse.trace.public across every observation in the trace and reads a missing attribute as false, so a guardrail child span without the flag turned a trace_public: true request private on Langfuse Cloud. Child spans now repeat the generation's value Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(langfuse): emit observations as plain OTel spans, keep the SDK for prompts and auth The callback now owns an isolated TracerProvider and OTLP exporter and builds generation and child spans with public OpenTelemetry APIs plus the LangfuseOtelSpanAttributes constants. Caller trace ids, generation ids, parent observation ids and historical start and end times are honoured through the OTel id generator, remote SpanContext and explicit span timestamps, so no private Langfuse SDK tracing handle is used any more. The Langfuse client stays only for get_prompt and auth_check This also resolves the gauntlet findings on the previous draft: fresh traces start from an empty context so caller application spans are never stamped, the Slack trace link is read from the request logging state instead of constructing a logger per alert, a truthy non-mapping trace_metadata is serialized instead of raising, trace_input and trace_output land on the root generation, discarding a cached client is done under the lock, and the prompt cache no longer leaks a task manager because the client cache no longer tears down shared providers Fixtures under tests/logging_callback_tests lose the SDK-private langfuse.internal.as_root marker; every other exported attribute is unchanged Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(langfuse): hand the SDK client a validated sample rate so an unusable LANGFUSE_SAMPLE_RATE no longer breaks the callback Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(langfuse): gate the SDK version before importing the OTel module in prompt management Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(langfuse): flush every export channel on proxy shutdown and use the callback's host in Slack trace links The shutdown hook imported litellm.utils.langFuseLogger, a global the callback registry never assigns, so a graceful restart dropped the spans still queued in the batch processors. Shutdown now calls flush_langfuse_tracing, which force-flushes every acquired channel. The Slack alert link falls back to the registered LangFuseLogger's langfuse_host when the request carries no dynamic host, and the export endpoint tests pin that scheme-relative or absolute LANGFUSE_OTEL_TRACES_EXPORT_PATH values stay on the configured host Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(langfuse): store resolved credentials on LangfusePromptManagement The Slack alert trace link reads langfuse_host from every registered LangFuseLogger. Prompt management subclasses it without calling the parent constructor, so it never set the attribute and the alerting handler crashed before posting Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(langfuse): flush every export channel concurrently under one shutdown deadline Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(langfuse): flush export channels on daemon threads so a stuck channel cannot hold up interpreter exit Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(langfuse): own the tracer config and drop the SDK client for prompts and auth The callback's TracerProvider now sets its sampler, span limits and id generator explicitly so unrelated OTEL_* variables no longer change what Langfuse receives, and trace metadata is written once on the trace instead of folded into the generation, which kept input and output under the attribute cap. Spans are emitted under the langfuse-sdk scope so Langfuse renders them natively, the batch processor queues 100k spans and honors LANGFUSE_FLUSH_AT, and the proxy shutdown flush runs off the event loop with a 10s deadline and logs a miss. Prompts, auth_check and the project id now go through LangfuseAPI directly with a litellm-owned TTL cache, so no Langfuse() client is built and a host application's client on the same public key is left alone. Dead attributes, the unreachable exporter branch and the export list are cleaned up, and the client-budget eviction behavior is documented. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(langfuse): export OTLP spans and fetch prompts through litellm's HTTPHandler instead of a private requests session Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(langfuse): gate the SDK version before importing the tracing module and retire unheld export channels An installed v2 SDK used to fail inside the langfuse_sdk import and surface as "Langfuse not installed"; the version check now runs first so v2 users get the upgrade message, and only PackageNotFoundError means the package is missing Export channels are now leased per credential set: acquire adds a holder, LangFuseLogger.stop (called by DynamicLoggingCache on expiry) releases one, and a channel with no holders is flushed and shut down after a 60 s grace, so rotating key or team credentials no longer grows one batch thread per credential set for the life of the process Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(langfuse): end the generation when a child span fails, take the client slot last, keep prompt cache keys structured Generation spans now end in a finally block so a bad guardrail or provider entry cannot strand the trace. The logger acquires its export channel and REST client before counting a client slot and releases the channel synchronously if the REST client fails to build, so retries after a bad config do not exhaust the budget. LANGFUSE_TIMEOUT accepts decimals for the REST client like it already did for OTLP export. The prompt cache keys on (name, version, label) so a missing label and the literal label None stay apart Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(langfuse): claim the cache entry before releasing its slot and channel hold on eviction Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(langfuse): coerce generation names, keep v2 release, timeout and retry defaults, refresh stale prompts off the loop A non-string metadata generation_name reached the OTLP encoder and took the whole batch down; it is now exported as its text and the exporter drops only the span the encoder rejects. LANGFUSE_RELEASE falls back to the deploy platform's commit variable again, the export deadline is back to the v2 default of 20 s and LANGFUSE_MAX_RETRIES sizes the retry ladder. An expired prompt is served at once while one background thread refreshes it, a re-acquired export channel cancels the pending retire timer, and flush reports delivery rather than a drained queue Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(proxy): assert the current Langfuse shutdown flush warning Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(langfuse): keep host OTel resource out, carry big metadata ints, tolerate bad flush and TTL env, stamp trace I/O under a parent Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(langfuse): name a malformed prompt cache TTL before the SDK import, keep metadata ints JSON safe, retry every 5xx export Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(langfuse): name the auth check failure, split a 413 export, wire LANGFUSE_DEBUG, stamp error output under a parent, send the ingestion version header Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(langfuse): honor LANGFUSE_DEBUG on the callbacks path, cap retry backoff, name the auth failure status and body Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(langfuse): cap LANGFUSE_MAX_RETRIES at 1000 so an absurd value cannot stall callback init Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(langfuse): fold 413 halving into bounded rounds instead of recursion The code-quality recursive-function gate flagged LangfuseSpanExporter.export. A batch of n spans settles within n.bit_length() halving rounds, so the split is a reduce over a frozen round state with the same posts, logs and results. The TTL gate test now asserts the gate returns without raising Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(langfuse): truncate a single oversized span like v2 instead of dropping it, no retries on REST auth and project lookups Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(langfuse): write the metadata truncation marker under a flattened key so Langfuse keeps it Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(langfuse): patch the HTTPHandler export path and sync the metadata fixture and lease registry with the v4 callback Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(langfuse): give the 413 split helpers a single explicit return path Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(langfuse): url-encode prompt names and fetch cold prompts without client retries A cold get_prompt runs inline on the event loop; the generated v4 client's default two retries slept through Retry-After (up to 60 s per attempt) and held the loop. The wrapper also passed the raw name into api/public/v2/prompts/{name}, so 'what?' fetched prompt 'what' and folder names left the route. Quote the name with safe='' like the v4 SDK's own get_prompt and pass max_retries=0 like the projects.get calls Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(langfuse): retry a cold prompt miss once and drop upstream headers from prompt errors A cold prompt fetch makes one immediate second attempt after a 5xx or a transport failure, as the v2 client did, still with the generated client's sleeping retries and Retry-After handling off so the event loop never stalls. A failed fetch raises LangfusePromptError carrying only the status and body, so the proxy no longer forwards Langfuse's response headers to its client Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(langfuse): stub the logger in the health auth_check test instead of dialing a closed port Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(langfuse): integration test for OTLP v4 delivery and prompt fetch through a real proxy Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * build(docker): keep the pip image's langfuse and otel pins on the v2 line its litellm 1.83.0 wheel expects The image validates the published PyPI artifact, whose langfuse callback still reads langfuse.version, so the 4.15.2 pin broke that callback. The pins move together with the next LITELLM_VERSION bump. Also rewords the trace_version precedence test docstring: v2 carried two version fields, v4 has one per span Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --- litellm/constants.py | 1 + litellm/integrations/SlackAlerting/utils.py | 38 +- litellm/integrations/langfuse/langfuse.py | 543 +++-- .../langfuse/langfuse_prompt_management.py | 115 +- litellm/integrations/langfuse/langfuse_sdk.py | 1213 ++++++++++++ litellm/litellm_core_utils/litellm_logging.py | 52 +- .../specialty_caches/dynamic_logging_cache.py | 51 +- .../service_trace_id_cache.py | 20 + .../health_endpoints/_health_endpoints.py | 4 +- litellm/proxy/proxy_server.py | 23 +- litellm/types/integrations/langfuse.py | 5 + pyproject.toml | 30 +- .../observability/test_langfuse_delivery.py | 270 +++ tests/litellm_utils_tests/test_utils.py | 25 +- tests/local_testing/test_alangfuse.py | 29 +- .../completion.json | 131 +- .../completion_with_bedrock_call.json | 108 +- .../completion_with_complex_metadata.json | 168 +- .../completion_with_langfuse_metadata.json | 170 +- .../completion_with_no_choices.json | 108 +- .../completion_with_router.json | 120 +- .../completion_with_tags.json | 140 +- .../completion_with_tags_stream.json | 140 +- .../completion_with_vertex_call.json | 104 +- .../complex_metadata.json | 143 +- .../complex_metadata_2.json | 135 +- .../empty_metadata.json | 129 +- .../metadata_with_function.json | 129 +- .../metadata_with_lock.json | 129 +- .../nested_metadata.json | 135 +- .../simple_metadata.json | 135 +- .../simple_metadata2.json | 139 +- .../simple_metadata3.json | 143 +- .../test_langfuse_dynamic_credentials.py | 39 +- .../test_langfuse_e2e_test.py | 340 ++-- .../test_langfuse_unit_tests.py | 62 +- .../test_slack_alerting_utils.py | 100 +- .../test_langfuse_prompt_management.py | 223 ++- .../langfuse/test_langfuse_sdk.py | 1759 +++++++++++++++++ .../integrations/test_langfuse.py | 1328 ++++++++++--- .../test_dynamic_logging_cache.py | 90 +- .../health_endpoints/test_health_endpoints.py | 19 + .../test_callback_management_endpoints.py | 42 +- .../proxy/proxy_server/test_lifecycle.py | 56 + uv.lock | 301 +-- 45 files changed, 6159 insertions(+), 3025 deletions(-) create mode 100644 litellm/integrations/langfuse/langfuse_sdk.py create mode 100644 litellm/litellm_core_utils/specialty_caches/service_trace_id_cache.py create mode 100644 tests/integration/observability/test_langfuse_delivery.py create mode 100644 tests/test_litellm/integrations/langfuse/test_langfuse_sdk.py diff --git a/litellm/constants.py b/litellm/constants.py index 79929b0bf6e..e7ba1f6b07f 100644 --- a/litellm/constants.py +++ b/litellm/constants.py @@ -598,6 +598,7 @@ FIREWORKS_AI_DEFAULT_CACHE_READ_RATE_RATIO: Final = 0.5 #### Logging callback constants #### REDACTED_BY_LITELM_STRING: Final = "REDACTED_BY_LITELM" MAX_LANGFUSE_INITIALIZED_CLIENTS: Final = int(os.getenv("MAX_LANGFUSE_INITIALIZED_CLIENTS", 50)) +LANGFUSE_SHUTDOWN_FLUSH_TIMEOUT_MILLIS: Final = 10_000 # Backpressure + lifetime bounds for the /v1/messages streaming relay (see # BaseAnthropicMessagesStreamingIterator.async_sse_wrapper). The relay queue is # bounded so a slow client throttles the upstream pump instead of letting it diff --git a/litellm/integrations/SlackAlerting/utils.py b/litellm/integrations/SlackAlerting/utils.py index 297d069a868..eb3a7f80f72 100644 --- a/litellm/integrations/SlackAlerting/utils.py +++ b/litellm/integrations/SlackAlerting/utils.py @@ -3,9 +3,11 @@ Utils used for slack alerting """ import asyncio +from collections.abc import Callable from typing import TYPE_CHECKING, Any, Final import litellm +from litellm.integrations.custom_logger import CustomLogger from litellm.proxy._types import AlertType from litellm.secret_managers.main import get_secret @@ -66,25 +68,27 @@ async def _add_langfuse_trace_id_to_alert( -> trace_id -> litellm_call_id """ - if "langfuse" not in litellm.logging_callback_manager._get_all_callbacks(): + from litellm.integrations.langfuse.langfuse import LangFuseLogger, resolve_langfuse_host + + callbacks: Final[list[CustomLogger | Callable[..., object] | str]] = ( + litellm.logging_callback_manager._get_all_callbacks() + ) + if not any(callback == "langfuse" or isinstance(callback, LangFuseLogger) for callback in callbacks): return None - ######################################################### - # Only run if langfuse is added as a callback - ######################################################### - if request_data is not None and request_data.get("litellm_logging_obj", None) is not None: - trace_id: str | None = None - litellm_logging_obj: Final[Logging] = request_data["litellm_logging_obj"] + if request_data is None or request_data.get("litellm_logging_obj", None) is None: + return None - for _ in range(3): - trace_id = litellm_logging_obj._get_trace_id(service_name="langfuse") - if trace_id is not None: - break - await asyncio.sleep(3) # wait 3s before retrying for trace id - ######################################################### - langfuse_object: Final = litellm_logging_obj._get_callback_object(service_name="langfuse") - if langfuse_object is not None: - base_url: Final = langfuse_object.Langfuse.base_url - return f"{base_url}/trace/{trace_id}" + litellm_logging_obj: Final[Logging] = request_data["litellm_logging_obj"] + instance_host: Final = next( + (callback.langfuse_host for callback in callbacks if isinstance(callback, LangFuseLogger)), None + ) + host: Final = resolve_langfuse_host( + litellm_logging_obj.standard_callback_dynamic_params.get("langfuse_host") or instance_host + ) + for _ in range(3): + if (trace_id := litellm_logging_obj._get_trace_id(service_name="langfuse")) is not None: + return f"{host}/trace/{trace_id}" + await asyncio.sleep(3) # wait 3s before retrying for trace id return None diff --git a/litellm/integrations/langfuse/langfuse.py b/litellm/integrations/langfuse/langfuse.py index 96d711337fb..9b860840e69 100644 --- a/litellm/integrations/langfuse/langfuse.py +++ b/litellm/integrations/langfuse/langfuse.py @@ -1,14 +1,14 @@ #### What this does #### # On success, logs events to Langfuse -import inspect import os import re import traceback from collections.abc import Callable, Iterable, Mapping, Sequence from datetime import datetime from functools import lru_cache +from importlib.metadata import PackageNotFoundError, version from types import MappingProxyType -from typing import TYPE_CHECKING, Any, Final, Literal, Protocol, cast +from typing import TYPE_CHECKING, Any, Final, Literal, Protocol, cast, runtime_checkable from packaging.version import Version @@ -45,13 +45,13 @@ from litellm.types.utils import ( ) if TYPE_CHECKING: - from langfuse.client import Langfuse, StatefulTraceClient - + from litellm.integrations.langfuse.langfuse_sdk import LangfuseApiClient, LangfuseObservation, LangfuseTracing from litellm.litellm_core_utils.litellm_logging import DynamicLoggingCache else: DynamicLoggingCache = Any - StatefulTraceClient = Any - Langfuse = Any + LangfuseApiClient = Any + LangfuseObservation = Any + LangfuseTracing = Any _DENIED_STEERING_KEYS: Final = frozenset({"headers", "endpoint", "caching_groups", "previous_models"}) @@ -142,6 +142,20 @@ def _logging_id(start_time: datetime | None, response_obj: object) -> str | None return litellm.utils.get_logging_id(start_time, response_obj) +@runtime_checkable +class _ResponseWithId(Protocol): + """Response payloads (ModelResponse and friends, or a plain dict) expose their provider id via ``get``.""" + + def get(self, key: Literal["id"], default: None = None, /) -> object: ... + + +def _lookup_ids(litellm_call_id: str | None, response_obj: object) -> Mapping[str, str]: + """v2 carried the response id inside the generation id; v4 hashes ids to 16 hex chars, so they ride in metadata.""" + response_id: Final[object] = response_obj.get("id") if isinstance(response_obj, _ResponseWithId) else None + ids: Final[tuple[tuple[str, object], ...]] = (("litellm_call_id", litellm_call_id), ("response_id", response_id)) + return MappingProxyType({key: str(value) for key, value in ids if value is not None}) + + def _as_steering_flag(value: object) -> bool: """A string ``str_to_bool`` does not recognise falls back to its truthiness.""" if isinstance(value, str): @@ -158,6 +172,68 @@ def _as_steering_key_sequence(value: object) -> tuple[str, ...]: return () +MINIMUM_LANGFUSE_VERSION: Final = "4.7" +UNSUPPORTED_LANGFUSE_VERSION: Final = "5" +PROMPT_CACHE_TTL_ENV: Final = "LANGFUSE_PROMPT_CACHE_DEFAULT_TTL_SECONDS" + + +def installed_langfuse_version() -> str: + """Only ``importlib.metadata`` reads correctly on every major. + + ``langfuse.version`` was removed in v4, ``langfuse.__version__`` does not + exist in v3, and in v2 it reports a different value from the distribution + that is actually installed. + """ + return version("langfuse") + + +def raise_if_unsupported_langfuse_version(installed_version: str) -> None: + """Fail at logger construction rather than dropping every event at request time. + + v4 moved the callback onto OpenTelemetry, so on an older SDK the import of + `LangfuseOtelSpanAttributes` raises inside the per-request handler and the + broad except there turns it into silent total data loss. + """ + installed: Final = Version(installed_version) + # compare majors, not versions: "5.0.0rc1" sorts below "5" but is just as unsupported + if Version(MINIMUM_LANGFUSE_VERSION) <= installed and installed.major < Version(UNSUPPORTED_LANGFUSE_VERSION).major: + return + raise ImportError( + f"\033[91mlitellm requires langfuse>={MINIMUM_LANGFUSE_VERSION},<{UNSUPPORTED_LANGFUSE_VERSION} for the " + f"'langfuse' callback, but {installed_version} is installed. Run " + f"'pip install \"langfuse>={MINIMUM_LANGFUSE_VERSION},<{UNSUPPORTED_LANGFUSE_VERSION}\"' to upgrade, or use " + f"the 'langfuse_otel' callback, which does not depend on the langfuse SDK\033[0m" + ) + + +def whole_number(raw: str) -> int | None: + try: + return int(raw) + except ValueError: + return None + + +def raise_if_unusable_prompt_cache_ttl() -> None: + """The v4 SDK runs ``int()`` on this variable while it is being imported, so a value that is not a whole + number has to be named here, before that import fails with a bare ``ValueError`` on every request.""" + raw: Final = os.environ.get(PROMPT_CACHE_TTL_ENV) + if raw is None or whole_number(raw) is not None: + return + raise ValueError(f"\033[91m{PROMPT_CACHE_TTL_ENV}={raw!r} must be a whole number of seconds\033[0m") + + +def _optional_str(value: object) -> str | None: + """v4 sets attribute values raw; a non-string version would be dropped by the server.""" + return str(value) if value is not None else None + + +def _trace_public_flag(value: object) -> bool | None: + """``trace_public`` reaches here as a bool from metadata or a string from a ``langfuse_*`` header.""" + if value is None: + return None + return _as_steering_flag(value) + + def resolve_langfuse_credentials( langfuse_public_key=None, langfuse_secret=None, @@ -172,9 +248,29 @@ def resolve_langfuse_credentials( secret_key = langfuse_secret or langfuse_secret_key or os.getenv("LANGFUSE_SECRET_KEY") public_key = langfuse_public_key or os.getenv("LANGFUSE_PUBLIC_KEY") - resolved_host: Final = langfuse_host or os.getenv("LANGFUSE_HOST", "https://cloud.langfuse.com") + return public_key, secret_key, resolve_langfuse_host(langfuse_host) - return public_key, secret_key, resolved_host + +def resolve_langfuse_host(langfuse_host: object = None) -> str: + """The Langfuse base URL for ``langfuse_host`` with the env fallbacks, always carrying a scheme.""" + resolved: Final = str( + langfuse_host or os.getenv("LANGFUSE_HOST") or os.getenv("LANGFUSE_BASE_URL") or "https://cloud.langfuse.com" + ) + return resolved if resolved.startswith(("http://", "https://")) else f"http://{resolved}" + + +def warn_if_upstream_langfuse_configured() -> None: + if os.getenv("UPSTREAM_LANGFUSE_SECRET_KEY") is None: + return + verbose_logger.warning( + "UPSTREAM_LANGFUSE_* is no longer supported: the langfuse callback moved to SDK v4, " + "which has no second ingestion client. The values are ignored." + ) + + +def parse_langfuse_debug(raw_value: str | None) -> bool: + """Parse the LANGFUSE_DEBUG value into the boolean flag the langfuse client expects.""" + return raw_value is not None and raw_value.strip().lower() in ("true", "1") @lru_cache(maxsize=8) @@ -199,29 +295,29 @@ class LangFuseLogger: allow_env_credentials: bool = True, ): try: - import langfuse - from langfuse import Langfuse - except Exception as e: + self.langfuse_sdk_version: str = installed_langfuse_version() + except PackageNotFoundError as e: raise Exception( - f"\033[91mLangfuse not installed, try running 'pip install langfuse' to fix this error: {e}\n{traceback.format_exc()}\033[0m" - ) + f"\033[91mLangfuse not installed, try running 'pip install langfuse' to fix this error: {e}\033[0m" + ) from e + raise_if_unsupported_langfuse_version(self.langfuse_sdk_version) + raise_if_unusable_prompt_cache_ttl() + from litellm.integrations.langfuse.langfuse_sdk import configured_release + self.public_key, self.secret_key, self.langfuse_host = resolve_langfuse_credentials( langfuse_public_key=langfuse_public_key, langfuse_secret=langfuse_secret, langfuse_host=langfuse_host, allow_env_credentials=allow_env_credentials, ) - if not (self.langfuse_host.startswith("http://") or self.langfuse_host.startswith("https://")): - # add http:// if unset, assume communicating over private network - e.g. render - self.langfuse_host = "http://" + self.langfuse_host _env_override: Final = str(langfuse_environment).strip() if langfuse_environment is not None else None if _env_override: validate_langfuse_environment_value(_env_override) self.langfuse_environment: str | None = _env_override else: self.langfuse_environment = self.resolve_deployment_environment() - self.langfuse_release = os.getenv("LANGFUSE_RELEASE") - self.langfuse_debug = os.getenv("LANGFUSE_DEBUG") + self.langfuse_release = configured_release() + self.langfuse_debug = parse_langfuse_debug(os.getenv("LANGFUSE_DEBUG")) self.langfuse_flush_interval = LangFuseLogger._get_langfuse_flush_interval(flush_interval) if should_use_langfuse_mock(): @@ -232,22 +328,9 @@ class LangFuseLogger: self.langfuse_client = self._http_handler.client self.is_mock_mode = False - parameters: Final = { - "public_key": self.public_key, - "secret_key": self.secret_key, - "host": self.langfuse_host, - "release": self.langfuse_release, - "debug": self.langfuse_debug, - "flush_interval": self.langfuse_flush_interval, # flush interval in seconds - "httpx_client": self.langfuse_client, - } - self.langfuse_sdk_version: str = langfuse.version.__version__ - - if "environment" in inspect.signature(Langfuse.__init__).parameters: - parameters["environment"] = self.langfuse_environment - if Version(self.langfuse_sdk_version) >= Version("2.6.0"): - parameters["sdk_integration"] = "litellm" - self.Langfuse: Langfuse = self.safe_init_langfuse_client(parameters) + self.api_client: LangfuseApiClient + self.tracing: LangfuseTracing + self.api_client, self.tracing = self.safe_init_langfuse_client() # set the current langfuse project id in the environ # this is used by Alerting to link to the correct project @@ -256,49 +339,62 @@ class LangFuseLogger: verbose_logger.debug("Langfuse Mock: Using mock project ID") else: try: - project_id = self.Langfuse.client.projects.get().data[0].id - os.environ["LANGFUSE_PROJECT_ID"] = project_id + project_id: Final = self.api_client.project_id() + if project_id is not None: + os.environ["LANGFUSE_PROJECT_ID"] = project_id except Exception: - project_id = None + verbose_logger.debug("Langfuse project id unavailable, alerting links will omit it") - if os.getenv("UPSTREAM_LANGFUSE_SECRET_KEY") is not None: - upstream_langfuse_debug_env: Final = os.getenv("UPSTREAM_LANGFUSE_DEBUG") - upstream_langfuse_debug: Final = ( - str_to_bool(upstream_langfuse_debug_env) if upstream_langfuse_debug_env is not None else None - ) - self.upstream_langfuse_secret_key = os.getenv("UPSTREAM_LANGFUSE_SECRET_KEY") - self.upstream_langfuse_public_key = os.getenv("UPSTREAM_LANGFUSE_PUBLIC_KEY") - self.upstream_langfuse_host = os.getenv("UPSTREAM_LANGFUSE_HOST") - self.upstream_langfuse_release = os.getenv("UPSTREAM_LANGFUSE_RELEASE") - self.upstream_langfuse_debug = upstream_langfuse_debug_env - self.upstream_langfuse = Langfuse( - public_key=self.upstream_langfuse_public_key, - secret_key=self.upstream_langfuse_secret_key, - host=self.upstream_langfuse_host, - release=self.upstream_langfuse_release, - debug=(upstream_langfuse_debug if upstream_langfuse_debug is not None else False), - ) - else: - self.upstream_langfuse = None + warn_if_upstream_langfuse_configured() - def safe_init_langfuse_client(self, parameters: dict) -> Langfuse: + def safe_init_langfuse_client(self) -> "tuple[LangfuseApiClient, LangfuseTracing]": + """Build the REST client and export channel while the process is under its logger budget. + + The budget dates from the SDK client, which started a consumer thread per instance and once + pinned a CPU at 100% when many were built; it still bounds the number of per-key loggers. """ - Safely init a langfuse client if the number of initialized clients is less than the max - - Note: - - Langfuse initializes 1 thread everytime a client is initialized. - - We've had an incident in the past where we reached 100% cpu utilization because Langfuse was initialized several times. - """ - from langfuse import Langfuse - if litellm.initialized_langfuse_clients >= MAX_LANGFUSE_INITIALIZED_CLIENTS: raise Exception( f"Max langfuse clients reached: {litellm.initialized_langfuse_clients} is greater than {MAX_LANGFUSE_INITIALIZED_CLIENTS}" ) - langfuse_client: Final = Langfuse(**parameters) + from litellm.integrations.langfuse.langfuse_sdk import ( + acquire_langfuse_tracing, + build_langfuse_client, + release_langfuse_tracing, + ) + + tracing: Final = acquire_langfuse_tracing( + public_key=str(self.public_key), + secret_key=str(self.secret_key), + base_url=self.langfuse_host, + environment=self.langfuse_environment, + release=self.langfuse_release, + flush_interval=self.langfuse_flush_interval, + mock_mode=self.is_mock_mode, + ) + try: + api_client: Final = build_langfuse_client( + public_key=self.public_key, + secret_key=self.secret_key, + base_url=self.langfuse_host, + httpx_client=self.langfuse_client, + ) + except Exception: + release_langfuse_tracing(tracing, grace_seconds=0.0) + raise litellm.initialized_langfuse_clients += 1 verbose_logger.debug("Created langfuse client number %s", litellm.initialized_langfuse_clients) - return langfuse_client + return api_client, tracing + + def flush(self) -> None: + """Push every queued observation to Langfuse before the process goes away.""" + self.tracing.flush() + + def stop(self) -> None: + """Give the export channel back; ``DynamicLoggingCache`` calls this when a per-key logger expires.""" + from litellm.integrations.langfuse.langfuse_sdk import release_langfuse_tracing + + release_langfuse_tracing(self.tracing) @staticmethod def add_metadata_from_header(litellm_params: dict, metadata: dict) -> dict[str, object]: @@ -349,7 +445,7 @@ class LangFuseLogger: user_id: str | None = None, level: str = "DEFAULT", status_message: str | None = None, - ) -> dict: + ) -> LangfuseLoggedEvent: """ Logs a success or error event on Langfuse """ @@ -411,10 +507,10 @@ class LangFuseLogger: verbose_logger.debug("Langfuse Layer Logging - final response object: %s", response_obj) verbose_logger.info("Langfuse Layer Logging - logging success") - return {"trace_id": trace_id, "generation_id": generation_id} + return LangfuseLoggedEvent(trace_id=trace_id, generation_id=generation_id) except Exception as e: verbose_logger.exception("Langfuse Layer Error(): Exception occured - %s", e) - return {"trace_id": None, "generation_id": None} + return LangfuseLoggedEvent(trace_id=None, generation_id=None) def _get_langfuse_input_output_content( self, @@ -518,18 +614,14 @@ class LangFuseLogger: level: str, litellm_call_id: str | None, ) -> tuple: - verbose_logger.debug("Langfuse Layer Logging - logging to langfuse v2") + verbose_logger.debug("Langfuse Layer Logging - logging to langfuse via sdk v%s", self.langfuse_sdk_version) try: standard_logging_object: Final[StandardLoggingPayload | None] = cast( StandardLoggingPayload | None, kwargs.get("standard_logging_object", None), ) - tags = ( - self._get_langfuse_tags(standard_logging_object=standard_logging_object) - if self._supports_tags() - else [] - ) + tags = self._get_langfuse_tags(standard_logging_object=standard_logging_object) allowlisted_metadata: Final[StandardLoggingMetadata | Mapping[str, object]] = ( standard_logging_object["metadata"] if standard_logging_object is not None else _NO_METADATA @@ -581,17 +673,17 @@ class LangFuseLogger: # This allows continuing an existing trace while still returning the correct trace_id if existing_trace_id is not None: trace_id = existing_trace_id - resolved_trace_id: Final = ( + call_trace_id: Final = ( litellm_call_id or trace_id if existing_trace_id is None and _is_session_header_trace(trace_id, session_id, litellm_params.get("proxy_server_request")) else trace_id ) - if resolved_trace_id != trace_id: + if call_trace_id != trace_id: verbose_logger.debug( "Langfuse: trace_id %s came from a session header; using call id %s so each call gets its own trace", trace_id, - resolved_trace_id, + call_trace_id, ) requested_trace_keys: Final = _as_steering_key_sequence(clean_metadata.pop("update_trace_keys", ())) update_trace_keys: Final = ( @@ -647,7 +739,7 @@ class LangFuseLogger: trace_params["output"] = masked_output if not mask_output else "redacted-by-litellm" else: # don't overwrite an existing trace trace_params = { - "id": resolved_trace_id, + "id": call_trace_id, "name": trace_name, "session_id": session_id, "input": masked_input if not mask_input else "redacted-by-litellm", @@ -659,10 +751,7 @@ class LangFuseLogger: for key in list(filter(lambda key: key.startswith("trace_"), clean_metadata.keys())): trace_params[key.replace("trace_", "")] = clean_metadata.pop(key, None) - if level == "ERROR": - trace_params["status_message"] = masked_output - else: - trace_params["output"] = masked_output if not mask_output else "redacted-by-litellm" + trace_params["output"] = masked_output if not mask_output else "redacted-by-litellm" if debug is True or (isinstance(debug, str) and debug.lower() == "true"): debug_metadata: Final = { @@ -697,17 +786,16 @@ class LangFuseLogger: ("api_base", api_base, bool(api_base)), ("vertex_location", vertex_location, bool(vertex_location)), ("aws_region_name", aws_region_name, bool(aws_region_name)), - ("cache_hit", kwargs.get("cache_hit") or False, self._supports_tags() and "cache_hit" in kwargs), + ("cache_hit", kwargs.get("cache_hit") or False, "cache_hit" in kwargs), ) enrichments: Final[Mapping[str, object]] = { key: value for key, value, include in candidate_enrichments if include } - if self._supports_tags(): - if "cache_hit" in kwargs and kwargs["cache_hit"] is None: - kwargs["cache_hit"] = False # rebind-ok: pre-existing normalization other integrations rely on - if existing_trace_id is None: - trace_params.update({"tags": tags}) + if "cache_hit" in kwargs and kwargs["cache_hit"] is None: + kwargs["cache_hit"] = False # rebind-ok: pre-existing normalization other integrations rely on + if existing_trace_id is None: + trace_params.update({"tags": tags}) proxy_server_request: Final = litellm_params.get("proxy_server_request", None) if proxy_server_request: @@ -721,17 +809,6 @@ class LangFuseLogger: if key.lower() not in _REDACTED_PROXY_HEADERS: clean_headers[key] = value - trace: Final[StatefulTraceClient] = self.Langfuse.trace(**trace_params) - - # Log provider specific information as a span - log_provider_specific_information_as_span(trace, enrichments) - - # Log guardrail information as a span - self._log_guardrail_information_as_span( - trace=trace, - standard_logging_object=standard_logging_object, - ) - generation_id = None usage = None usage_details = None @@ -753,7 +830,7 @@ class LangFuseLogger: usage = { "prompt_tokens": prompt_tokens, "completion_tokens": completion_tokens, - "total_cost": cost if self._supports_costs() else None, + "total_cost": cost, } # According to langfuse documentation: "the input value must be reduced by the number of cache_read_input_tokens" input_tokens: Final = prompt_tokens - cache_read_input_tokens @@ -765,15 +842,15 @@ class LangFuseLogger: cache_read_input_tokens=cache_read_input_tokens, ) - generation_name = clean_metadata.pop("generation_name", None) - if generation_name is None: - # if `generation_name` is None, use sensible default values - # If using litellm proxy user `key_alias` if not None - # If `key_alias` is None, just log `litellm-{call_type}` as the generation name - _user_api_key_alias: Final = cast(str | None, clean_metadata.get("user_api_key_alias", None)) - generation_name = f"litellm-{cast(str, kwargs.get('call_type', 'completion'))}" - if _user_api_key_alias is not None: - generation_name = f"litellm:{_user_api_key_alias}" + requested_generation_name: Final = clean_metadata.pop("generation_name", None) + _user_api_key_alias: Final = cast(str | None, clean_metadata.get("user_api_key_alias", None)) + generation_name: Final = ( + str(requested_generation_name) + if requested_generation_name is not None + else f"litellm:{_user_api_key_alias}" + if _user_api_key_alias is not None + else f"litellm-{cast(str, kwargs.get('call_type', 'completion'))}" + ) if response_obj is not None: system_fingerprint = getattr(response_obj, "system_fingerprint", None) @@ -789,53 +866,97 @@ class LangFuseLogger: generation_params = { "name": generation_name, "id": clean_metadata.pop("generation_id", generation_id), - "start_time": start_time, - "end_time": end_time, - "model": model_name, - "model_parameters": optional_params, "input": masked_input if not mask_input else "redacted-by-litellm", "output": masked_output if not mask_output else "redacted-by-litellm", - "usage": usage, - "usage_details": usage_details, - "metadata": { - **log_requester_metadata(redact_user_api_key_info(metadata=allowlisted_metadata)), + "cost_details": {"total": cost} # mutable-ok: langfuse serializes this payload + if usage is not None and isinstance(cost, (int, float)) + else None, + "metadata": { # mutable-ok: langfuse serializes this payload, a proxy is not json-encodable + **log_requester_metadata(redact_user_api_key_info(metadata=allowlisted_metadata)), # pyright: ignore[reportArgumentType] # TypedDict in, plain metadata dict out **enrichments, + **_lookup_ids(litellm_call_id, response_obj), }, - "level": level, - "version": clean_metadata.pop("version", None), + "version": _optional_str(clean_metadata.pop("version", None)), } parent_observation_id: Final = metadata.get("parent_observation_id", None) - if parent_observation_id is not None: - generation_params["parent_observation_id"] = parent_observation_id - - if self._supports_prompt(): - generation_params = _add_prompt_to_generation_params( - generation_params=generation_params, - clean_metadata=clean_metadata, - prompt_management_metadata=prompt_management_metadata, - langfuse_client=self.Langfuse, - ) + generation_params = _add_prompt_to_generation_params( + generation_params=generation_params, + clean_metadata=clean_metadata, + prompt_management_metadata=prompt_management_metadata, + langfuse_client=self.api_client, + ) if masked_output is not None and isinstance(masked_output, str) and level == "ERROR": generation_params["status_message"] = masked_output - if self._supports_completion_start_time(): - generation_params["completion_start_time"] = kwargs.get("completion_start_time", None) + # langfuse ships in the proxy-runtime extra, so this module must import cleanly without it + from litellm.integrations.langfuse.langfuse_sdk import ( + observation_attributes, + resolve_observation_id, + resolve_trace_id, + start_generation, + trace_attributes, + ) - generation_client: Final = trace.generation(**generation_params) + resolved_trace_id: Final = resolve_trace_id(call_trace_id) # pyright: ignore[reportArgumentType] # metadata value, str or None at runtime + continued_trace: Final = existing_trace_id is not None + generation_is_trace_root: Final = not continued_trace and parent_observation_id is None + trace_public: Final = _trace_public_flag(trace_params.get("public")) + trace_input: Final = trace_params.get("input") + trace_output: Final = trace_params.get("output") + trace_level_attributes: Final = trace_attributes( + name=trace_params.get("name"), + user_id=trace_params.get("user_id"), + session_id=trace_params.get("session_id"), + version=trace_params.get("version"), + release=trace_params.get("release"), + tags=trace_params.get("tags"), + metadata=trace_params.get("metadata"), + public=trace_public, + input=None if generation_is_trace_root and trace_input == generation_params["input"] else trace_input, + output=None + if generation_is_trace_root and trace_output == generation_params["output"] + else trace_output, + ) + generation_attributes: Final = observation_attributes( + observation_type="generation", + input=generation_params["input"], + output=generation_params["output"], + metadata=generation_params["metadata"], + level=level, + status_message=generation_params.get("status_message"), + version=generation_params["version"], + model=model_name, + model_parameters=optional_params, + usage_details=usage_details, + cost_details=generation_params["cost_details"], + completion_start_time=kwargs.get("completion_start_time", None), + prompt=generation_params.get("prompt"), + ) + generation: Final = start_generation( + tracing=self.tracing, + trace_id=resolved_trace_id, + parent_observation_id=resolve_observation_id(parent_observation_id), # pyright: ignore[reportArgumentType] # metadata value, str or None at runtime + existing_trace=continued_trace, + observation_id=resolve_observation_id(generation_params["id"]), + name=generation_params["name"], # pyright: ignore[reportArgumentType] # always the str set a few lines up + start_time=start_time, + public=trace_public, + attributes=MappingProxyType({**generation_attributes, **trace_level_attributes}), + ) + try: + log_provider_specific_information_as_span( + tracing=self.tracing, parent=generation, enrichments=enrichments + ) + self._log_guardrail_information_as_span( + tracing=self.tracing, parent=generation, standard_logging_object=standard_logging_object + ) + finally: + generation.end(end_time) - # Return the trace_id we set (which should be litellm_call_id when no explicit trace_id provided) - # We explicitly set trace_id in trace_params["id"], so langfuse should use it - # Verify langfuse accepted our trace_id; if it differs, log a warning but still return our intended value - # to match expected test behavior - if hasattr(generation_client, "trace_id") and generation_client.trace_id: - if generation_client.trace_id != resolved_trace_id: - verbose_logger.warning( - "Langfuse trace_id mismatch: set %s, but langfuse returned %s. Using our intended trace_id for consistency.", - resolved_trace_id, - generation_client.trace_id, - ) - return resolved_trace_id, generation_id + # log_event_on_langfuse tuple-unpacks this and re-wraps it in the dict callers cache. + # The observation id is the requested generation_id after resolve_observation_id. + return resolved_trace_id, generation.id except Exception: verbose_logger.error("Langfuse Layer Error - %s", traceback.format_exc()) return None, None @@ -904,27 +1025,11 @@ class LangFuseLogger: _cache_key = _hidden_params.get("cache_key", None) if _cache_key is None and litellm.cache is not None: # fallback to using "preset_cache_key" - _preset_cache_key: Final = litellm.cache._get_preset_cache_key_from_kwargs(**kwargs) + _preset_cache_key: Final = litellm.cache._get_preset_cache_key_from_kwargs(**kwargs) # pyright: ignore[reportPrivateUsage] # kwargs-ok: no public preset-cache-key accessor _cache_key = _preset_cache_key tags.append(f"cache_key:{_cache_key}") return tags - def _supports_tags(self): - """Check if current langfuse version supports tags""" - return Version(self.langfuse_sdk_version) >= Version("2.6.3") - - def _supports_prompt(self): - """Check if current langfuse version supports prompt""" - return Version(self.langfuse_sdk_version) >= Version("2.7.3") - - def _supports_costs(self): - """Check if current langfuse version supports costs""" - return Version(self.langfuse_sdk_version) >= Version("2.7.3") - - def _supports_completion_start_time(self): - """Check if current langfuse version supports completion start time""" - return Version(self.langfuse_sdk_version) >= Version("2.7.3") - @staticmethod def _apply_masking_function(data: object, masking_function: Callable[[object], object]) -> object: """ @@ -973,23 +1078,24 @@ class LangFuseLogger: @staticmethod def _get_langfuse_flush_interval(flush_interval: int) -> int: - """ - Get the langfuse flush interval to initialize the Langfuse client - - Reads `LANGFUSE_FLUSH_INTERVAL` from the environment variable. - If not set, uses the flush interval passed in as an argument. - - Args: - flush_interval: The flush interval to use if LANGFUSE_FLUSH_INTERVAL is not set - - Returns: - [int] The flush interval to use to initialize the Langfuse client - """ - return int(os.getenv("LANGFUSE_FLUSH_INTERVAL") or flush_interval) + """``LANGFUSE_FLUSH_INTERVAL`` in whole seconds above 0 (the export scheduler's delay), else ``flush_interval``.""" + raw: Final = os.getenv("LANGFUSE_FLUSH_INTERVAL") + if not raw: + return flush_interval + parsed: Final = int(raw) if raw.strip().isdigit() else None + if parsed is None or parsed <= 0: + verbose_logger.warning( + "LANGFUSE_FLUSH_INTERVAL=%r is not a whole number of seconds above 0; flushing every %d s", + raw, + flush_interval, + ) + return flush_interval + return parsed def _log_guardrail_information_as_span( self, - trace: StatefulTraceClient, + tracing: "LangfuseTracing", + parent: "LangfuseObservation", standard_logging_object: StandardLoggingPayload | None, ): """ @@ -1011,6 +1117,8 @@ class LangFuseLogger: ) return + from litellm.integrations.langfuse.langfuse_sdk import observation_attributes, start_child_span + for guardrail_entry in guardrail_information: if not isinstance(guardrail_entry, dict): verbose_logger.debug( @@ -1019,30 +1127,35 @@ class LangFuseLogger: ) continue - span = trace.span( + span = start_child_span( + tracing=tracing, + parent=parent, name="guardrail", - input=guardrail_entry.get("guardrail_request", None), - output=guardrail_entry.get("guardrail_response", None), - metadata={ - "guardrail_name": guardrail_entry.get("guardrail_name", None), - "guardrail_mode": guardrail_entry.get("guardrail_mode", None), - "guardrail_masked_entity_count": guardrail_entry.get("masked_entity_count", None), - }, start_time=guardrail_entry.get("start_time", None), - end_time=guardrail_entry.get("end_time", None), + attributes=observation_attributes( + observation_type="span", + input=guardrail_entry.get("guardrail_request", None), + output=guardrail_entry.get("guardrail_response", None), + metadata=MappingProxyType( + { + "guardrail_name": guardrail_entry.get("guardrail_name", None), + "guardrail_mode": guardrail_entry.get("guardrail_mode", None), + "guardrail_masked_entity_count": guardrail_entry.get("masked_entity_count", None), + } + ), + ), ) verbose_logger.debug("Logged guardrail information as span: %s", span) - span.end() + span.end(guardrail_entry.get("end_time", None)) def _add_prompt_to_generation_params( generation_params: dict, clean_metadata: dict, prompt_management_metadata: StandardLoggingPromptManagementMetadata | None, - langfuse_client: object, + langfuse_client: "LangfuseApiClient", ) -> dict: - from langfuse import Langfuse from langfuse.model import ( ChatPromptClient, Prompt_Chat, @@ -1050,8 +1163,6 @@ def _add_prompt_to_generation_params( TextPromptClient, ) - langfuse_client = cast(Langfuse, langfuse_client) - user_prompt: Final = clean_metadata.pop("prompt", None) if user_prompt is None and prompt_management_metadata is None: pass @@ -1075,7 +1186,7 @@ def _add_prompt_to_generation_params( if "labels" in prompt_text_params and "tags" in prompt_text_params: _data["labels"] = user_prompt.get("labels", []) or [] _data["tags"] = user_prompt.get("tags", []) or [] - _prompt_obj = Prompt_Text(**_data) + _prompt_obj = Prompt_Text(**_data) # pyright: ignore[reportArgumentType] # kwargs-ok: shape mirrors the pydantic model, values from the user's prompt dict generation_params["prompt"] = TextPromptClient(prompt=_prompt_obj) elif isinstance(user_prompt["prompt"], list): @@ -1090,7 +1201,7 @@ def _add_prompt_to_generation_params( _data["labels"] = user_prompt.get("labels", []) or [] _data["tags"] = user_prompt.get("tags", []) or [] - _prompt_obj = Prompt_Chat(**_data) + _prompt_obj = Prompt_Chat(**_data) # pyright: ignore[reportArgumentType] # kwargs-ok: shape mirrors the pydantic model, values from the user's prompt dict generation_params["prompt"] = ChatPromptClient(prompt=_prompt_obj) else: @@ -1110,21 +1221,14 @@ def _add_prompt_to_generation_params( def log_provider_specific_information_as_span( - trace, - clean_metadata: Mapping[str, Any], + *, + tracing: "LangfuseTracing", + parent: "LangfuseObservation", + enrichments: Mapping[str, Any], ): - """ - Logs provider-specific information as spans. + """Logs provider-specific information as spans under the generation.""" - Parameters: - trace: The tracing object used to log spans. - clean_metadata: A dictionary containing metadata to be logged. - - Returns: - None - """ - - _hidden_params: Final[Mapping[str, object] | None] = clean_metadata.get("hidden_params", None) + _hidden_params: Final[Mapping[str, object] | None] = enrichments.get("hidden_params", None) if _hidden_params is None: return @@ -1135,22 +1239,27 @@ def log_provider_specific_information_as_span( for elem in vertex_ai_grounding_metadata: if isinstance(elem, dict): for key, value in elem.items(): - trace.span( - name=key, - input=value, - ) + _end_grounding_span(tracing=tracing, parent=parent, name=key, value=value) else: - trace.span( - name="vertex_ai_grounding_metadata", - input=elem, - ) + _end_grounding_span(tracing=tracing, parent=parent, name="vertex_ai_grounding_metadata", value=elem) else: - trace.span( - name="vertex_ai_grounding_metadata", - input=vertex_ai_grounding_metadata, + _end_grounding_span( + tracing=tracing, parent=parent, name="vertex_ai_grounding_metadata", value=vertex_ai_grounding_metadata ) +def _end_grounding_span(*, tracing: "LangfuseTracing", parent: "LangfuseObservation", name: str, value: object) -> None: + from litellm.integrations.langfuse.langfuse_sdk import observation_attributes, start_child_span + + start_child_span( + tracing=tracing, + parent=parent, + name=name, + start_time=None, + attributes=observation_attributes(observation_type="span", input=value), + ).end() + + def log_requester_metadata(clean_metadata: Mapping[str, Any]): returned_metadata: Final = {} requester_metadata: Final = clean_metadata.get("requester_metadata") or {} diff --git a/litellm/integrations/langfuse/langfuse_prompt_management.py b/litellm/integrations/langfuse/langfuse_prompt_management.py index 90db0626e23..3786087ba91 100644 --- a/litellm/integrations/langfuse/langfuse_prompt_management.py +++ b/litellm/integrations/langfuse/langfuse_prompt_management.py @@ -2,16 +2,14 @@ Call Hook for LiteLLM Proxy which allows Langfuse prompt management. """ -import inspect -import os from functools import lru_cache from typing import TYPE_CHECKING, Any, Final, Literal, TypeAlias, cast -from packaging.version import Version - from litellm.integrations.custom_logger import CustomLogger from litellm.integrations.prompt_management_base import PromptManagementClient from litellm.litellm_core_utils.asyncify import run_async_function +from litellm.llms.custom_httpx.http_handler import HTTPHandler +from litellm.types.integrations.langfuse import LangfuseLoggedEvent from litellm.types.llms.openai import AllMessageValues, ChatCompletionSystemMessage from litellm.types.prompts.init_prompts import PromptSpec from litellm.types.utils import StandardCallbackDynamicParams, StandardLoggingPayload @@ -19,17 +17,27 @@ from litellm.types.utils import StandardCallbackDynamicParams, StandardLoggingPa from ...litellm_core_utils.specialty_caches.dynamic_logging_cache import ( DynamicLoggingCache, ) +from ...litellm_core_utils.specialty_caches.service_trace_id_cache import in_memory_trace_id_cache from ..prompt_management_base import PromptManagementBase -from .langfuse import LangFuseLogger, resolve_langfuse_credentials +from .langfuse import ( + LangFuseLogger, + installed_langfuse_version, + raise_if_unsupported_langfuse_version, + raise_if_unusable_prompt_cache_ttl, + resolve_langfuse_credentials, + warn_if_upstream_langfuse_configured, +) from .langfuse_handler import LangFuseHandler +from .langfuse_mock_client import create_mock_langfuse_client, should_use_langfuse_mock if TYPE_CHECKING: - from langfuse import Langfuse - from langfuse.client import ChatPromptClient, TextPromptClient + from langfuse.model import ChatPromptClient, TextPromptClient from litellm.litellm_core_utils.litellm_logging import Logging as LiteLLMLoggingObj - LangfuseClass: TypeAlias = Langfuse + from .langfuse_sdk import LangfuseApiClient + + LangfuseClass: TypeAlias = LangfuseApiClient PROMPT_CLIENT = TextPromptClient | ChatPromptClient else: @@ -49,23 +57,24 @@ def langfuse_client_init( allow_env_credentials: bool = True, ) -> LangfuseClass: """ - Initialize Langfuse client with caching to prevent multiple initializations. + Initialize the Langfuse REST client with caching to prevent multiple initializations. Args: langfuse_public_key (str, optional): Public key for Langfuse. Defaults to None. langfuse_secret (str, optional): Secret key for Langfuse. Defaults to None. langfuse_host (str, optional): Host URL for Langfuse. Defaults to None. - flush_interval (int, optional): Flush interval in seconds. Defaults to 1. + flush_interval (int, optional): Kept in the signature so cached callers keep their cache key. Returns: - Langfuse: Initialized Langfuse client instance + LangfuseApiClient: prompt, auth and project lookups for one credential set Raises: Exception: If langfuse package is not installed """ + raise_if_unsupported_langfuse_version(installed_langfuse_version()) + raise_if_unusable_prompt_cache_ttl() try: - import langfuse - from langfuse import Langfuse + from .langfuse_sdk import build_langfuse_client except Exception as e: raise Exception( f"\033[91mLangfuse not installed, try running 'pip install langfuse' to fix this error: {e}\n\033[0m" @@ -83,39 +92,22 @@ def langfuse_client_init( # add http:// if unset, assume communicating over private network - e.g. render langfuse_host = "http://" + langfuse_host - langfuse_release: Final = os.getenv("LANGFUSE_RELEASE") - langfuse_debug: Final = os.getenv("LANGFUSE_DEBUG") + warn_if_upstream_langfuse_configured() - parameters: Final = { - "public_key": public_key, - "secret_key": secret_key, - "host": langfuse_host, - "release": langfuse_release, - "debug": langfuse_debug, - "flush_interval": LangFuseLogger._get_langfuse_flush_interval(flush_interval), # flush interval in seconds - } + httpx_client: Final = create_mock_langfuse_client() if should_use_langfuse_mock() else HTTPHandler().client + return build_langfuse_client( + public_key=public_key, + secret_key=secret_key, + base_url=langfuse_host, + httpx_client=httpx_client, + ) - if Version(langfuse.version.__version__) >= Version("2.6.0"): - parameters["sdk_integration"] = "litellm" - if Version(langfuse.version.__version__) >= Version("2.7.3"): - import httpx - - import litellm - - from ...llms.custom_httpx.http_handler import get_ssl_configuration - - parameters["httpx_client"] = httpx.Client( - verify=get_ssl_configuration(), - cert=os.getenv("SSL_CERTIFICATE", litellm.ssl_certificate), - ) - - if "environment" in inspect.signature(Langfuse.__init__).parameters: - parameters["environment"] = LangFuseLogger.resolve_deployment_environment() - - client: Final = Langfuse(**parameters) - - return client +def _remember_trace_id(litellm_call_id: object, logged: LangfuseLoggedEvent) -> None: + trace_id: Final = logged["trace_id"] + if not isinstance(litellm_call_id, str) or trace_id is None: + return + in_memory_trace_id_cache.set_cache(litellm_call_id=litellm_call_id, service_name="langfuse", trace_id=trace_id) class LangfusePromptManagement(LangFuseLogger, PromptManagementBase, CustomLogger): @@ -126,15 +118,33 @@ class LangfusePromptManagement(LangFuseLogger, PromptManagementBase, CustomLogge langfuse_host=None, flush_interval=1, ): - import langfuse - self.langfuse_sdk_version = langfuse.version.__version__ - self.Langfuse = langfuse_client_init( + self.langfuse_sdk_version = installed_langfuse_version() + raise_if_unsupported_langfuse_version(self.langfuse_sdk_version) + raise_if_unusable_prompt_cache_ttl() + + from .langfuse_sdk import acquire_langfuse_tracing, configured_release + + self.api_client = langfuse_client_init( langfuse_public_key=langfuse_public_key, langfuse_secret=langfuse_secret, langfuse_host=langfuse_host, flush_interval=flush_interval, ) + self.public_key, self.secret_key, self.langfuse_host = resolve_langfuse_credentials( + langfuse_public_key=langfuse_public_key, + langfuse_secret=langfuse_secret, + langfuse_host=langfuse_host, + ) + self.tracing = acquire_langfuse_tracing( + public_key=str(self.public_key), + secret_key=str(self.secret_key), + base_url=self.langfuse_host, + environment=LangFuseLogger.resolve_deployment_environment(), + release=configured_release(), + flush_interval=LangFuseLogger._get_langfuse_flush_interval(flush_interval), # pyright: ignore[reportPrivateUsage] # shared env-fallback helper, not part of the logger's API + mock_mode=should_use_langfuse_mock(), + ) @property def integration_name(self): @@ -228,11 +238,8 @@ class LangfusePromptManagement(LangFuseLogger, PromptManagementBase, CustomLogge langfuse_host=dynamic_callback_params.get("langfuse_host"), allow_env_credentials=dynamic_callback_params.get("langfuse_host") is None, ) - langfuse_prompt_client: Final = self._get_prompt_from_id( - langfuse_prompt_id=prompt_id, - langfuse_client=langfuse_client, - ) - return langfuse_prompt_client is not None + self._get_prompt_from_id(langfuse_prompt_id=prompt_id, langfuse_client=langfuse_client) + return True def _compile_prompt_helper( self, @@ -311,13 +318,14 @@ class LangfusePromptManagement(LangFuseLogger, PromptManagementBase, CustomLogge standard_callback_dynamic_params=standard_callback_dynamic_params, in_memory_dynamic_logger_cache=in_memory_dynamic_logger_cache, ) - langfuse_logger_to_use.log_event_on_langfuse( + logged: Final = langfuse_logger_to_use.log_event_on_langfuse( kwargs=kwargs, response_obj=response_obj, start_time=start_time, end_time=end_time, user_id=kwargs.get("user", None), ) + _remember_trace_id(litellm_call_id=kwargs.get("litellm_call_id"), logged=logged) except Exception as e: from litellm._logging import verbose_logger @@ -339,7 +347,7 @@ class LangfusePromptManagement(LangFuseLogger, PromptManagementBase, CustomLogge status_message = str(kwargs.get("exception", "Unknown error")) if standard_logging_object is not None: status_message = standard_logging_object.get("error_str", None) or status_message - langfuse_logger_to_use.log_event_on_langfuse( + logged: Final = langfuse_logger_to_use.log_event_on_langfuse( start_time=start_time, end_time=end_time, response_obj=None, @@ -348,6 +356,7 @@ class LangfusePromptManagement(LangFuseLogger, PromptManagementBase, CustomLogge level="ERROR", kwargs=kwargs, ) + _remember_trace_id(litellm_call_id=kwargs.get("litellm_call_id"), logged=logged) except Exception as e: from litellm._logging import verbose_logger diff --git a/litellm/integrations/langfuse/langfuse_sdk.py b/litellm/integrations/langfuse/langfuse_sdk.py new file mode 100644 index 00000000000..66819c95ebf --- /dev/null +++ b/litellm/integrations/langfuse/langfuse_sdk.py @@ -0,0 +1,1213 @@ +from __future__ import annotations + +import logging +import os +import re +import threading +from base64 import b64encode +from collections.abc import Iterable, Mapping, Sequence +from contextvars import ContextVar +from dataclasses import dataclass, replace +from datetime import datetime +from functools import partial, reduce +from hashlib import sha256 +from importlib.metadata import version +from itertools import chain +from time import monotonic, sleep +from types import MappingProxyType +from typing import Final, Literal +from urllib.parse import quote + +import httpx +import opentelemetry.trace as otel_trace +from langfuse import LangfuseOtelSpanAttributes +from langfuse.api import LangfuseAPI, Prompt, Prompt_Chat +from langfuse.api.core.api_error import ApiError +from langfuse.api.core.request_options import RequestOptions +from langfuse.model import BasePromptClient, ChatPromptClient, PromptClient, TextPromptClient +from opentelemetry.context import Context +from opentelemetry.exporter.otlp.proto.common.trace_encoder import encode_spans +from opentelemetry.sdk.resources import Resource +from opentelemetry.sdk.trace import ReadableSpan, SpanLimits, TracerProvider +from opentelemetry.sdk.trace.export import BatchSpanProcessor, SpanExporter, SpanExportResult +from opentelemetry.sdk.trace.id_generator import RandomIdGenerator +from opentelemetry.sdk.trace.sampling import ALWAYS_ON, Decision, Sampler, SamplingResult +from opentelemetry.trace import Link, NonRecordingSpan, Span, SpanContext, SpanKind, TraceFlags, Tracer, TraceState +from opentelemetry.util.types import Attributes, AttributeValue +from pydantic import BaseModel, ConfigDict + +import litellm +from litellm._logging import verbose_logger +from litellm.integrations.langfuse.langfuse import PROMPT_CACHE_TTL_ENV, parse_langfuse_debug, whole_number +from litellm.litellm_core_utils.safe_json_dumps import safe_dumps +from litellm.llms.custom_httpx.http_handler import HTTPHandler, _get_httpx_client + +__all__ = ( + "AuthCheckFailure", + "DiscardingSpanExporter", + "LangfuseApiClient", + "LangfuseObservation", + "LangfusePromptError", + "LangfuseSpanExporter", + "LangfuseTracing", + "TraceIdHashSampler", + "acquire_langfuse_tracing", + "build_langfuse_client", + "build_langfuse_tracing", + "configured_flush_at", + "configured_max_retries", + "configured_release", + "configured_sample_rate", + "configured_timeout", + "enable_langfuse_debug_logging", + "flush_langfuse_tracing", + "observation_attributes", + "release_langfuse_tracing", + "resolve_observation_id", + "resolve_trace_id", + "start_child_span", + "start_generation", + "to_unix_nanos", + "trace_attributes", +) + +_TRACE_ID_PATTERN: Final = re.compile(r"^(?=.*[1-9a-f])[0-9a-f]{32}$") +_OBSERVATION_ID_PATTERN: Final = re.compile(r"^(?=.*[1-9a-f])[0-9a-f]{16}$") +_TRACER_NAME: Final = "langfuse-sdk" +_LANGFUSE_INGESTION_VERSION_HEADER: Final = "x-langfuse-ingestion-version" +_LANGFUSE_INGESTION_VERSION: Final = "4" +_NO_REST_RETRIES: Final = RequestOptions(max_retries=0) +_TRUNCATION_MARKER: Final = "" +_METADATA_PREFIXES: Final = (LangfuseOtelSpanAttributes.OBSERVATION_METADATA, LangfuseOtelSpanAttributes.TRACE_METADATA) +_TRUNCATION_GROUPS: Final = ( + (LangfuseOtelSpanAttributes.OBSERVATION_INPUT, LangfuseOtelSpanAttributes.TRACE_INPUT), + (LangfuseOtelSpanAttributes.OBSERVATION_OUTPUT, LangfuseOtelSpanAttributes.TRACE_OUTPUT), + _METADATA_PREFIXES, +) +_SERVER_FLOOR_HINT: Final = ( + "; the OTLP traces route needs a self-hosted Langfuse server on 3.63.0 or newer " + "(https://langfuse.com/self-hosting/upgrade/versioning#sdk-server)" +) +_langfuse_logger: Final = logging.getLogger("langfuse") +_MAX_QUEUE_SIZE: Final = 100_000 +_DEFAULT_FLUSH_AT: Final = 512 +_CHANNEL_RETIRE_GRACE_SECONDS: Final = 60.0 +_DEFAULT_TIMEOUT_SECONDS: Final = 20.0 +_DEFAULT_MAX_RETRIES: Final = 3 +_MAX_RETRIES: Final = 1_000 +_MAX_BACKOFF_EXPONENT: Final = 6 +_DEFAULT_PROMPT_CACHE_TTL_SECONDS: Final = 60.0 +_JSON_SAFE_INT: Final = 2**53 - 1 +_COMMON_RELEASE_ENVS: Final = ( + "RENDER_GIT_COMMIT", + "CI_COMMIT_SHA", + "CIRCLE_SHA1", + "SOURCE_VERSION", + "TRAVIS_COMMIT", + "GIT_COMMIT", + "GITHUB_SHA", + "BITBUCKET_COMMIT", + "BUILD_SOURCEVERSION", + "DRONE_COMMIT_SHA", +) +_SPAN_LIMITS: Final = SpanLimits( + max_attributes=SpanLimits.UNSET, + max_events=128, + max_links=128, + max_span_attributes=SpanLimits.UNSET, + max_event_attributes=128, + max_link_attributes=128, + max_attribute_length=SpanLimits.UNSET, + max_span_attribute_length=SpanLimits.UNSET, +) + + +def to_unix_nanos(value: datetime | float | None) -> int | None: + """Langfuse v4 takes OTel timestamps, which are integer nanoseconds since the epoch. + + Guardrail entries carry unix seconds as floats rather than datetimes, so both + shapes have to convert; the v2 SDK accepted either through a pydantic model. + """ + if value is None: + return None + seconds: Final = value.timestamp() if isinstance(value, datetime) else float(value) + return int(seconds * 1_000_000_000) + + +def resolve_trace_id(trace_id: object | None) -> str: + """Map a caller's trace id onto the 32 lowercase hex characters v4 requires.""" + serialized: Final = "" if trace_id is None else str(trace_id) + normalized: Final = serialized.lower().replace("-", "") + if _TRACE_ID_PATTERN.fullmatch(normalized): + return normalized + if not serialized: + return format(RandomIdGenerator().generate_trace_id(), "032x") + return sha256(serialized.encode("utf-8")).digest()[:16].hex() + + +def resolve_observation_id(observation_id: object | None) -> str | None: + """Map a caller's parent observation id onto v4's 16 lowercase hex characters.""" + serialized: Final = "" if observation_id is None else str(observation_id) + normalized: Final = serialized.lower().replace("-", "") + if _OBSERVATION_ID_PATTERN.fullmatch(normalized): + return normalized + if not serialized: + return None + return sha256(serialized.encode("utf-8")).digest()[:8].hex() + + +def _serialize(value: object) -> str | None: + return value if value is None or isinstance(value, str) else safe_dumps(value) + + +def _string_or_none(value: object) -> str | None: + return None if value is None else str(value) + + +def _serialize_datetime(value: object) -> str | None: + """A datetime the way the SDK's ``EventSerializer`` sends one: a JSON string, naive values read as local time.""" + if isinstance(value, datetime): + return safe_dumps(value.astimezone().isoformat()) + return _serialize(value) + + +def _strings(items: Iterable[object]) -> tuple[str, ...]: + return tuple(str(item) for item in items) + + +def _string_sequence(value: object) -> Sequence[str] | None: + if value is None: + return None + if isinstance(value, (list, tuple, set, frozenset)): + return _strings(value) or None + return (str(value),) + + +def _present(entries: Iterable[tuple[str, AttributeValue | None]]) -> Mapping[str, AttributeValue]: + return MappingProxyType({key: value for key, value in entries if value is not None}) + + +def _metadata_value(value: object) -> AttributeValue | None: + """A metadata value as it survives the trip: OTLP drops ints past int64 and a JSON reader rounds ints past + 2**53, so those go as strings, which is how v2's readback showed them.""" + if isinstance(value, (str, bool)): + return value + if isinstance(value, int) and -_JSON_SAFE_INT <= value <= _JSON_SAFE_INT: + return value + return _serialize(value) + + +def _flattened_metadata(prefix: str, metadata: object) -> Mapping[str, AttributeValue]: + """Mirror the SDK's wire shape: one ``.`` attribute per key, or ```` for a non-dict.""" + if metadata is None: + return _present(()) + if not isinstance(metadata, Mapping): + return _present(((prefix, _serialize(metadata)),)) + return _present((f"{prefix}.{key}", _metadata_value(value)) for key, value in metadata.items()) + + +def trace_attributes( + *, + name: object = None, + user_id: object = None, + session_id: object = None, + version: object = None, + release: object = None, + tags: object = None, + metadata: object = None, + public: bool | None = None, + input: object = None, + output: object = None, +) -> Mapping[str, AttributeValue]: + """Trace-level fields ride on an observation's span as ``langfuse.trace.*`` style attributes in v4. + + On the root observation they define the trace; on a continuation they update it, which is + how v2's ``trace(...)`` and ``update_trace_keys`` contracts map onto the OTLP ingestion. + """ + scalar: Final[tuple[tuple[str, str | bool | None], ...]] = ( + (LangfuseOtelSpanAttributes.TRACE_NAME, _string_or_none(name)), + (LangfuseOtelSpanAttributes.TRACE_USER_ID, _string_or_none(user_id)), + (LangfuseOtelSpanAttributes.TRACE_SESSION_ID, _string_or_none(session_id)), + (LangfuseOtelSpanAttributes.VERSION, _string_or_none(version)), + (LangfuseOtelSpanAttributes.RELEASE, _string_or_none(release)), + (LangfuseOtelSpanAttributes.TRACE_PUBLIC, public), + (LangfuseOtelSpanAttributes.TRACE_INPUT, _serialize(input)), + (LangfuseOtelSpanAttributes.TRACE_OUTPUT, _serialize(output)), + ) + tags_entry: Final[tuple[str, Sequence[str] | None]] = ( + LangfuseOtelSpanAttributes.TRACE_TAGS, + _string_sequence(tags), + ) + return _present( + chain(scalar, (tags_entry,), _flattened_metadata(LangfuseOtelSpanAttributes.TRACE_METADATA, metadata).items()) + ) + + +def observation_attributes( + *, + observation_type: Literal["generation", "span"], + input: object = None, + output: object = None, + metadata: object = None, + level: object = None, + status_message: object = None, + version: object = None, + model: object = None, + model_parameters: object = None, + usage_details: object = None, + cost_details: object = None, + completion_start_time: object = None, + prompt: object = None, +) -> Mapping[str, AttributeValue]: + """The observation's own fields, serialized the way the SDK's ``create_generation_attributes`` does. + + ``prompt`` links the generation to a managed prompt only when it is a real prompt client; + v2 dropped anything else, and a fallback prompt has no server-side version to link. + """ + linked_prompt: Final = prompt if isinstance(prompt, BasePromptClient) and not prompt.is_fallback else None + scalar: Final[tuple[tuple[str, str | int | None], ...]] = ( + (LangfuseOtelSpanAttributes.OBSERVATION_TYPE, observation_type), + (LangfuseOtelSpanAttributes.OBSERVATION_LEVEL, _string_or_none(level)), + (LangfuseOtelSpanAttributes.OBSERVATION_STATUS_MESSAGE, _string_or_none(status_message)), + (LangfuseOtelSpanAttributes.VERSION, _string_or_none(version)), + (LangfuseOtelSpanAttributes.OBSERVATION_INPUT, _serialize(input)), + (LangfuseOtelSpanAttributes.OBSERVATION_OUTPUT, _serialize(output)), + (LangfuseOtelSpanAttributes.OBSERVATION_MODEL, _string_or_none(model)), + (LangfuseOtelSpanAttributes.OBSERVATION_MODEL_PARAMETERS, _serialize(model_parameters)), + (LangfuseOtelSpanAttributes.OBSERVATION_USAGE_DETAILS, _serialize(usage_details)), + (LangfuseOtelSpanAttributes.OBSERVATION_COST_DETAILS, _serialize(cost_details)), + (LangfuseOtelSpanAttributes.OBSERVATION_COMPLETION_START_TIME, _serialize_datetime(completion_start_time)), + (LangfuseOtelSpanAttributes.OBSERVATION_PROMPT_NAME, linked_prompt.name if linked_prompt else None), + (LangfuseOtelSpanAttributes.OBSERVATION_PROMPT_VERSION, linked_prompt.version if linked_prompt else None), + ) + return _present( + chain(scalar, _flattened_metadata(LangfuseOtelSpanAttributes.OBSERVATION_METADATA, metadata).items()) + ) + + +@dataclass(frozen=True, slots=True) +class LangfuseObservation: + """A Langfuse observation as the OTel span litellm exports for it.""" + + span: Span + public: bool | None + + @property + def id(self) -> str: + return format(self.span.get_span_context().span_id, "016x") + + @property + def trace_id(self) -> str: + return format(self.span.get_span_context().trace_id, "032x") + + def end(self, end_time: datetime | float | None = None) -> None: + self.span.end(end_time=to_unix_nanos(end_time)) + + +_requested_trace_id: Final[ContextVar[int | None]] = ContextVar("litellm_langfuse_requested_trace_id", default=None) +_requested_span_id: Final[ContextVar[int | None]] = ContextVar("litellm_langfuse_requested_span_id", default=None) + + +class _RequestedIdGenerator(RandomIdGenerator): + """Hand out the ids the calling context asked for, random otherwise. + + v2 took caller trace and generation ids as plain fields; OTel derives both from + the tracer's id generator, so the request rides on a context variable instead. + """ + + def generate_trace_id(self) -> int: + requested: Final = _requested_trace_id.get() + return super().generate_trace_id() if requested is None else requested + + def generate_span_id(self) -> int: + requested: Final = _requested_span_id.get() + return super().generate_span_id() if requested is None else requested + + +def _parent_context(*, trace_id: str, parent_observation_id: str | None, existing_trace: bool) -> Context: + """Where a new observation hangs: nowhere for a fresh trace, under a remote parent when continuing one. + + ``existing_trace`` is the v2 ``existing_trace_id`` contract: the trace is appended to, never + rewritten. The server takes a root observation's name and I/O as the trace's, so a continuation + without a known parent hangs under a parent id that is never exported instead of claiming root. + An explicitly empty context also keeps the caller's own active span out of the picture. + """ + if parent_observation_id is None and not existing_trace: + return Context() + parent_span_id: Final = ( + int(parent_observation_id, 16) if parent_observation_id is not None else RandomIdGenerator().generate_span_id() + ) + remote_parent: Final = NonRecordingSpan( + SpanContext( + trace_id=int(trace_id, 16), + span_id=parent_span_id, + is_remote=True, + trace_flags=TraceFlags(TraceFlags.SAMPLED), + ) + ) + return otel_trace.set_span_in_context(remote_parent) + + +def _start_span( + tracer: Tracer, + *, + name: str, + context: Context, + start_time: datetime | float | None, + trace_id: str | None, + observation_id: str | None, + attributes: Mapping[str, AttributeValue], +) -> Span: + trace_token: Final = _requested_trace_id.set(int(trace_id, 16) if trace_id is not None else None) + span_token: Final = _requested_span_id.set(int(observation_id, 16) if observation_id is not None else None) + try: + return tracer.start_span( + name=name, context=context, start_time=to_unix_nanos(start_time), attributes=attributes + ) + finally: + _requested_span_id.reset(span_token) + _requested_trace_id.reset(trace_token) + + +def start_generation( + *, + tracing: LangfuseTracing, + trace_id: str, + parent_observation_id: str | None, + existing_trace: bool, + observation_id: str | None, + name: str, + start_time: datetime | float | None, + public: bool | None, + attributes: Mapping[str, AttributeValue], +) -> LangfuseObservation: + """Create the generation for one model call, timed from when that call began. + + ``trace_id``, ``parent_observation_id`` and ``observation_id`` are the v2 ``trace(id=...)``, + ``generation(parent_observation_id=...)`` and ``generation(id=...)`` arguments, already + normalized by ``resolve_trace_id`` and ``resolve_observation_id``. + """ + span: Final = _start_span( + tracing.tracer, + name=name, + context=_parent_context( + trace_id=trace_id, parent_observation_id=parent_observation_id, existing_trace=existing_trace + ), + start_time=start_time, + trace_id=trace_id, + observation_id=observation_id, + attributes=attributes, + ) + return LangfuseObservation(span=span, public=public) + + +def start_child_span( + *, + tracing: LangfuseTracing, + parent: LangfuseObservation, + name: str, + start_time: datetime | float | None, + attributes: Mapping[str, AttributeValue], +) -> LangfuseObservation: + """Create an observation under the generation, keeping its own time window. + + The server folds the trace's ``public`` flag across every observation, with a missing + attribute read as ``False``, so the child repeats the generation's value. + """ + public_entry: Final[tuple[str, bool | None]] = (LangfuseOtelSpanAttributes.TRACE_PUBLIC, parent.public) + span: Final = _start_span( + tracing.tracer, + name=name, + context=otel_trace.set_span_in_context(parent.span), + start_time=start_time, + trace_id=None, + observation_id=None, + attributes=_present(chain((public_entry,), attributes.items())), + ) + return LangfuseObservation(span=span, public=parent.public) + + +@dataclass(frozen=True, slots=True) +class TraceIdHashSampler(Sampler): + """Sample on a SHA-256 of the trace id rather than its low 64 bits. + + litellm trace ids are UUIDs, whose variant bits pin the top of that low word, so + ``TraceIdRatioBased`` drops every trace at rates up to 0.5 and skews above it. + """ + + rate: float + + def should_sample( + self, + parent_context: Context | None, + trace_id: int, + name: str, + kind: SpanKind | None = None, + attributes: Attributes = None, + links: Sequence[Link] | None = None, + trace_state: TraceState | None = None, + ) -> SamplingResult: + digest: Final = sha256(trace_id.to_bytes(16, "big")).digest() + sampled: Final = int.from_bytes(digest[:8], "big") < round(self.rate * 2**64) + parent: Final = otel_trace.get_current_span(parent_context).get_span_context() + return SamplingResult( + Decision.RECORD_AND_SAMPLE if sampled else Decision.DROP, + attributes if sampled else None, + parent.trace_state if parent.is_valid else None, + ) + + def get_description(self) -> str: + return f"TraceIdHashSampler{{{self.rate}}}" + + +def _parse_float(raw: str) -> float | None: + try: + return float(raw) + except ValueError: + return None + + +def _parse_sample_rate(raw: str) -> float | None: + rate: Final = _parse_float(raw) + return rate if rate is not None and 0.0 <= rate <= 1.0 else None + + +def configured_sample_rate() -> float: + """``LANGFUSE_SAMPLE_RATE`` as a fraction, exporting everything when it is unset or unusable.""" + raw: Final = os.environ.get("LANGFUSE_SAMPLE_RATE") + if raw is None: + return 1.0 + parsed: Final = _parse_sample_rate(raw) + if parsed is None: + verbose_logger.warning( + "LANGFUSE_SAMPLE_RATE=%r is not a number between 0.0 and 1.0; ignoring it and exporting every trace", raw + ) + return 1.0 + return parsed + + +def configured_timeout() -> float: + """``LANGFUSE_TIMEOUT`` in seconds for every export and REST call, the v2 SDK's 20 s when unset. + + A value that is not a number raises, as the v2 client did at construction, so a typo is not silently ignored. + """ + return float(os.environ.get("LANGFUSE_TIMEOUT", _DEFAULT_TIMEOUT_SECONDS)) + + +def configured_max_retries() -> int: + """``LANGFUSE_MAX_RETRIES`` as the number of re-sends after a failed export, the v2 SDK's knob and default. + + Capped at ``_MAX_RETRIES``: with the backoff ceiling that is already hours per batch, and the exporter holds + one delay per re-send. + """ + raw: Final = os.environ.get("LANGFUSE_MAX_RETRIES") + if raw is None: + return _DEFAULT_MAX_RETRIES + if not raw.strip().isdigit(): + verbose_logger.warning( + "LANGFUSE_MAX_RETRIES=%r is not a whole number; retrying %d times", raw, _DEFAULT_MAX_RETRIES + ) + return _DEFAULT_MAX_RETRIES + requested: Final = int(raw) + if requested > _MAX_RETRIES: + verbose_logger.warning( + "LANGFUSE_MAX_RETRIES=%d is above the ceiling; retrying %d times", requested, _MAX_RETRIES + ) + return min(requested, _MAX_RETRIES) + + +def configured_release() -> str | None: + """``LANGFUSE_RELEASE``, else the commit variable of the CI or deploy platform, as both SDK generations resolve it.""" + return os.environ.get("LANGFUSE_RELEASE") or next( + (os.environ[name] for name in _COMMON_RELEASE_ENVS if name in os.environ), None + ) + + +def configured_prompt_cache_ttl() -> float: + """``LANGFUSE_PROMPT_CACHE_DEFAULT_TTL_SECONDS`` in whole seconds as the SDK reads it, its 60 s default when unset + or unusable; ``raise_if_unusable_prompt_cache_ttl`` has already named a value that is not a whole number.""" + raw: Final = os.environ.get(PROMPT_CACHE_TTL_ENV) + if raw is None: + return _DEFAULT_PROMPT_CACHE_TTL_SECONDS + parsed: Final = whole_number(raw) + if parsed is None or parsed < 0: + verbose_logger.warning( + "%s=%r is not a whole number of seconds at or above 0; caching prompts for %.0f s", + PROMPT_CACHE_TTL_ENV, + raw, + _DEFAULT_PROMPT_CACHE_TTL_SECONDS, + ) + return _DEFAULT_PROMPT_CACHE_TTL_SECONDS + return float(parsed) + + +def configured_flush_at() -> int: + """``LANGFUSE_FLUSH_AT`` as the export batch size, the SDK's own knob, with its default when unset or unusable.""" + raw: Final = os.environ.get("LANGFUSE_FLUSH_AT") + if raw is None: + return _DEFAULT_FLUSH_AT + parsed: Final = int(raw) if raw.strip().isdigit() else None + if parsed is None or not 0 < parsed <= _MAX_QUEUE_SIZE: + verbose_logger.warning( + "LANGFUSE_FLUSH_AT=%r is not a whole number between 1 and %d; exporting batches of %d", + raw, + _MAX_QUEUE_SIZE, + _DEFAULT_FLUSH_AT, + ) + return _DEFAULT_FLUSH_AT + return parsed + + +class DiscardingSpanExporter(SpanExporter): + """Accept and drop every span, for mock mode. + + The mock intercepts the httpx client behind the REST API, but observations + travel over OTLP, so without this the "no network calls" contract silently sends + real traces to the configured host. + """ + + def export(self, spans: object) -> SpanExportResult: + return SpanExportResult.SUCCESS + + def shutdown(self) -> None: + return None + + def force_flush(self, timeout_millis: int = 30_000) -> bool: + return True + + +_ExportOutcome = Literal["delivered", "retry", "rejected", "too_large"] +_Batch = tuple[ReadableSpan, ...] + + +@dataclass(frozen=True, slots=True) +class _Halving: + """One round of a 413 split: the batches still to send and the results of the ones already settled.""" + + pending: tuple[_Batch, ...] + settled: tuple[SpanExportResult, ...] = () + + +def _smaller(batch: _Batch) -> tuple[_Batch, ...]: + """What to send after a 413: the two halves of a batch, or a single span with its largest field truncated.""" + if len(batch) != 1: + return batch[: len(batch) // 2], batch[len(batch) // 2 :] + (only,) = batch + truncated: Final = _truncated(only) + return () if truncated is None else ((truncated,),) + + +def _in_group(key: str, group: tuple[str, ...]) -> bool: + return any(key == prefix or key.startswith(prefix + ".") for prefix in group) + + +def _group_size(attributes: Mapping[str, AttributeValue], group: tuple[str, ...]) -> int: + return sum( + len(str(value)) for key, value in attributes.items() if _in_group(key, group) and value != _TRUNCATION_MARKER + ) + + +def _marker_key(prefix: str) -> str: + """Langfuse reads input and output as one string but metadata only as flattened keys, so the marker gets one.""" + return f"{prefix}.truncated" if prefix in _METADATA_PREFIXES else prefix + + +def _truncated(span: ReadableSpan) -> ReadableSpan | None: + """The span with its largest remaining input, output or metadata replaced by the marker the v2 consumer wrote + when an event went over ``LANGFUSE_MAX_EVENT_SIZE_BYTES``, or ``None`` once all three are gone.""" + attributes: Final = span.attributes or MappingProxyType({}) + largest: Final = max(_TRUNCATION_GROUPS, key=lambda group: _group_size(attributes, group)) + if _group_size(attributes, largest) == 0: + return None + kept: Final = {key: value for key, value in attributes.items() if not _in_group(key, largest)} + marked: Final = { + _marker_key(prefix): _TRUNCATION_MARKER + for prefix in largest + if any(_in_group(key, (prefix,)) for key in attributes) + } + return ReadableSpan( + name=span.name, + context=span.context, + parent=span.parent, + resource=span.resource, + attributes=MappingProxyType({**kept, **marked}), + events=span.events, + links=span.links, + kind=span.kind, + status=span.status, + start_time=span.start_time, + end_time=span.end_time, + instrumentation_scope=span.instrumentation_scope, + ) + + +def enable_langfuse_debug_logging() -> None: + """What ``Langfuse(debug=True)`` does: a root handler if none exists, and the ``langfuse`` logger at DEBUG.""" + logging.basicConfig(format="%(asctime)s - %(name)s - %(levelname)s - %(message)s") + _langfuse_logger.setLevel(logging.DEBUG) + + +def _retryable_status(status: int) -> bool: + """Any 5xx, a timeout or a rate limit: what the v2 consumer re-sent, plus the 408 the OTLP exporter retries.""" + return status in (408, 429) or 500 <= status <= 599 + + +@dataclass(frozen=True, slots=True) +class LangfuseSpanExporter(SpanExporter): + """OTLP/HTTP protobuf export through litellm's own HTTP handler. + + The handler carries litellm's TLS material (``ssl_verify``, CA bundle, client certificate) exactly + as v2's injected httpx client did. A connect or read failure and a retryable status are re-sent after + each delay, matching the v2 ingestion consumer; ``BatchSpanProcessor`` would otherwise drop the whole + batch on the first exception. A 413 splits the batch in halves until each body fits or a single span + is left; that span is re-sent with its input, output and metadata replaced by the v2 consumer's + truncation marker, largest first, and dropped only when the fully truncated span is still refused. + """ + + handler: HTTPHandler + endpoint: str + headers: Mapping[str, str] + timeout: float + delays: Sequence[float] = (1.0, 2.0, 4.0) + + def export(self, spans: Sequence[ReadableSpan]) -> SpanExportResult: + """Halving a batch of n spans settles every span within ``n.bit_length()`` rounds plus one per truncation + step, so the rounds are a fixed fold rather than a recursion.""" + rounds: Final = range(len(spans).bit_length() + 1 + len(_TRUNCATION_GROUPS)) + final: Final = reduce(lambda halving, _: self._round(halving), rounds, _Halving(pending=(tuple(spans),))) + return ( + SpanExportResult.SUCCESS + if all(result is SpanExportResult.SUCCESS for result in final.settled) + else SpanExportResult.FAILURE + ) + + def _round(self, halving: _Halving) -> _Halving: + sent: Final = tuple((batch, self._send_batch(batch)) for batch in halving.pending) + return _Halving( + pending=tuple(part for batch, outcome in sent if outcome == "too_large" for part in _smaller(batch)), + settled=halving.settled + + tuple( + SpanExportResult.SUCCESS if outcome == "delivered" else SpanExportResult.FAILURE + for _, outcome in sent + if outcome != "too_large" + ), + ) + + def _send_batch(self, batch: _Batch) -> _ExportOutcome: + """A 413 on more than one span asks for halves; on a single span it asks for a truncation, and the span is + dropped and reported once nothing is left to truncate.""" + body: Final = _encode(batch) + if body is None: + return "rejected" + outcome: Final = self._send(body) + if outcome != "too_large": + return outcome + match batch: + case (only,) if _truncated(only) is None: + verbose_logger.error( + "Langfuse rejected a single %d byte span export to %s as too large, dropping it", + len(body), + self.endpoint, + ) + return "rejected" + case (_,): + verbose_logger.warning( + "Langfuse rejected a single %d byte span export to %s as too large, resending it with its " + "largest field replaced by %r", + len(body), + self.endpoint, + _TRUNCATION_MARKER, + ) + case _: + verbose_logger.warning( + "Langfuse rejected a %d byte export of %d spans as too large, resending in halves", + len(body), + len(batch), + ) + return "too_large" + + def _send(self, body: bytes) -> _ExportOutcome: + for delay in self.delays: + outcome: _ExportOutcome = self._post(body) + if outcome != "retry": + return outcome + verbose_logger.warning("Langfuse export to %s failed, retrying in %ss", self.endpoint, delay) + sleep(delay) + last: Final = self._post(body) + if last == "retry": + verbose_logger.error("Langfuse export to %s failed after %d retries", self.endpoint, len(self.delays)) + return last + + def _post(self, body: bytes) -> _ExportOutcome: + try: + self.handler.post(self.endpoint, data=body, headers=dict(self.headers), timeout=self.timeout) + except httpx.HTTPStatusError as error: + status: Final = error.response.status_code + if _retryable_status(status): + return "retry" + if status == 413: + return "too_large" + verbose_logger.error( + "Langfuse rejected an export to %s with HTTP %d%s", + self.endpoint, + status, + _SERVER_FLOOR_HINT if status == 404 else "", + ) + return "rejected" + except (httpx.TransportError, litellm.Timeout) as error: + verbose_logger.warning("Langfuse export to %s raised %s", self.endpoint, error) + return "retry" + _langfuse_logger.debug("Exported %d bytes of spans to %s", len(body), self.endpoint) + return "delivered" + + def shutdown(self) -> None: + return None + + def force_flush(self, timeout_millis: int = 30_000) -> bool: + return True + + +def _encode(spans: Sequence[ReadableSpan]) -> bytes | None: + """The OTLP body, or ``None`` when nothing survived: a span the encoder rejects is dropped, not the whole batch.""" + try: + return encode_spans(spans).SerializeToString() + except Exception: # noqa: BLE001 # protobuf raises TypeError or ValueError depending on the field + kept: Final = tuple(span for span in spans if _encodes(span)) + verbose_logger.error("Langfuse export dropped %d span(s) the OTLP encoder rejected", len(spans) - len(kept)) + return encode_spans(kept).SerializeToString() if kept else None + + +def _encodes(span: ReadableSpan) -> bool: + try: + encode_spans((span,)) + except Exception: # noqa: BLE001 # same encoder failure modes as above + return False + return True + + +def _build_span_exporter(*, public_key: str, secret_key: str, base_url: str) -> LangfuseSpanExporter: + """Endpoint, headers and export path are the v4 SDK span processor's, so the server treats the spans as SDK + traffic; the 20 s timeout and the retry count are what the v2 consumer used. The ingestion-version header is + the one Langfuse's compatibility matrix asks a v4 producer to send.""" + export_path: Final = os.getenv("LANGFUSE_OTEL_TRACES_EXPORT_PATH") or "/api/public/otel/v1/traces" + encoded_auth: Final = b64encode(f"{public_key}:{secret_key}".encode()).decode("ascii") + return LangfuseSpanExporter( + handler=_get_httpx_client(), + endpoint=f"{base_url.rstrip('/')}/{export_path.lstrip('/')}", + headers=MappingProxyType( + { + "Authorization": "Basic " + encoded_auth, + "Content-Type": "application/x-protobuf", + "x-langfuse-sdk-name": "python", + "x-langfuse-sdk-version": version("langfuse"), + "x-langfuse-public-key": public_key, + _LANGFUSE_INGESTION_VERSION_HEADER: _LANGFUSE_INGESTION_VERSION, + } + ), + timeout=configured_timeout(), + delays=tuple(2.0 ** min(attempt, _MAX_BACKOFF_EXPONENT) for attempt in range(configured_max_retries())), + ) + + +def _resource(*, environment: str | None, release: str | None) -> Resource: + """Only litellm's own attributes: ``Resource.create`` would merge the host's ``OTEL_RESOURCE_ATTRIBUTES``.""" + return Resource( + _present( + ( + (LangfuseOtelSpanAttributes.ENVIRONMENT, environment), + (LangfuseOtelSpanAttributes.RELEASE, release), + ) + ) + ) + + +class _ExportLedger(SpanExporter): + """Counts the batches the exporter gave up on, so a flush can report delivery rather than a drained queue.""" + + def __init__(self, exporter: SpanExporter) -> None: + self.exporter: Final = exporter + self.failed_batches = 0 + + def export(self, spans: Sequence[ReadableSpan]) -> SpanExportResult: + result: Final = self.exporter.export(spans) + if result is not SpanExportResult.SUCCESS: + self.failed_batches += 1 + return result + + def shutdown(self) -> None: + self.exporter.shutdown() + + def force_flush(self, timeout_millis: int = 30_000) -> bool: + return self.exporter.force_flush(timeout_millis) + + +@dataclass(frozen=True, slots=True) +class LangfuseTracing: + """litellm's own export channel to one Langfuse project: a provider, its tracer and the exporter behind them. + + The channel is litellm's rather than the SDK's so that the process-global OTel provider stays + untouched, historical timestamps and caller ids are honoured, and no SDK internals are needed. + """ + + provider: TracerProvider + tracer: Tracer + ledger: _ExportLedger + + def flush(self, timeout_millis: int = 30_000) -> bool: + """``True`` only when the queue drained in time and every batch it held was accepted by the destination.""" + failed_before: Final = self.ledger.failed_batches + return self.provider.force_flush(timeout_millis) and self.ledger.failed_batches == failed_before + + def shutdown(self) -> None: + self.provider.shutdown() + + +@dataclass(frozen=True, slots=True) +class _TracingKey: + public_key: str + secret_key: str + base_url: str + environment: str | None + release: str | None + sample_rate: float + flush_at: int + flush_interval_millis: int + mock_mode: bool + + +@dataclass(frozen=True, slots=True) +class _Lease: + tracing: LangfuseTracing + holders: int + retire: threading.Timer | None = None + + +_TRACING_LOCK: Final = threading.Lock() +_TRACING: Final[dict[_TracingKey, _Lease]] = {} # mutable-ok: process-wide channel cache, guarded by _TRACING_LOCK + + +def acquire_langfuse_tracing( + *, + public_key: str, + secret_key: str, + base_url: str, + environment: str | None, + release: str | None, + flush_interval: float, + mock_mode: bool, +) -> LangfuseTracing: + """One export channel per credential set, shared by every logger built for it. + + A provider owns a batch export thread, so a channel lives while any logger holds it and is + retired through ``release_langfuse_tracing`` once the last holder lets go. + """ + if parse_langfuse_debug(os.getenv("LANGFUSE_DEBUG")): + enable_langfuse_debug_logging() + key: Final = _TracingKey( + public_key=public_key, + secret_key=secret_key, + base_url=base_url, + environment=environment, + release=release, + sample_rate=configured_sample_rate(), + flush_at=configured_flush_at(), + flush_interval_millis=int(flush_interval * 1000), + mock_mode=mock_mode, + ) + with _TRACING_LOCK: + cached: Final = _TRACING.get(key) + if cached is not None: + if cached.retire is not None: + cached.retire.cancel() + _TRACING[key] = replace(cached, holders=cached.holders + 1, retire=None) + return cached.tracing + created: Final = build_langfuse_tracing( + exporter=DiscardingSpanExporter() + if mock_mode + else _build_span_exporter(public_key=public_key, secret_key=secret_key, base_url=base_url), + environment=environment, + release=release, + sample_rate=key.sample_rate, + flush_at=key.flush_at, + flush_interval_millis=key.flush_interval_millis, + ) + _TRACING[key] = _Lease(tracing=created, holders=1) + return created + + +def release_langfuse_tracing(tracing: LangfuseTracing, *, grace_seconds: float = _CHANNEL_RETIRE_GRACE_SECONDS) -> None: + """Let go of one logger's hold on its channel; a channel nobody holds is retired ``grace_seconds`` later. + + The grace covers a callback that fetched its logger from the cache just before the entry expired, + and a logger rebuilt for the same credentials in the meantime picks the channel back up instead. + """ + with _TRACING_LOCK: + held: Final = next(((key, lease) for key, lease in _TRACING.items() if lease.tracing is tracing), None) + if held is None: + return + key, lease = held + if lease.holders <= 0: + return + if lease.holders > 1: + _TRACING[key] = replace(lease, holders=lease.holders - 1) + return + if grace_seconds > 0: + retire: Final = threading.Timer(grace_seconds, lambda: _retire_unless_reacquired(key, retire)) + retire.name = "langfuse-retire" + retire.daemon = True + _TRACING[key] = _Lease(tracing=tracing, holders=0, retire=retire) + retire.start() + return + del _TRACING[key] + tracing.shutdown() + + +def _retire_unless_reacquired(key: _TracingKey, timer: threading.Timer) -> None: + """Only the timer the lease still points at may retire it; a re-acquire cancels and clears the pending one.""" + with _TRACING_LOCK: + lease: Final = _TRACING.get(key) + if lease is None or lease.retire is not timer: + return + del _TRACING[key] + lease.tracing.shutdown() + + +class _FlushWorker(threading.Thread): + """Daemon, so a channel still blocked at the deadline cannot hold up interpreter exit.""" + + def __init__(self, channel: LangfuseTracing, timeout_millis: int) -> None: + super().__init__(name="langfuse-flush", daemon=True) + self.channel: Final = channel + self.timeout_millis: Final = timeout_millis + self.flushed = False + + def run(self) -> None: + self.flushed = self.channel.flush(self.timeout_millis) + + +def flush_langfuse_tracing(timeout_millis: int = 30_000) -> bool: + """Force-flush every export channel this process acquired, all within one ``timeout_millis`` deadline. + + ``True`` only when every channel flushed in time; a channel still blocked at the deadline is left to + finish in the background rather than pushing the deadline out for the channels after it. + """ + with _TRACING_LOCK: + channels: Final = tuple(lease.tracing for lease in _TRACING.values()) + workers: Final = tuple(_FlushWorker(channel, timeout_millis) for channel in channels) + deadline: Final = monotonic() + timeout_millis / 1000 + for worker in workers: + worker.start() + for worker in workers: + worker.join(max(0.0, deadline - monotonic())) + return all(not worker.is_alive() and worker.flushed for worker in workers) + + +def build_langfuse_tracing( + *, + exporter: SpanExporter, + environment: str | None, + release: str | None, + sample_rate: float, + flush_interval_millis: int, + flush_at: int = _DEFAULT_FLUSH_AT, +) -> LangfuseTracing: + """Wire the provider from litellm's own settings so a host's ``OTEL_*`` variables do not steer it. + + An unset sampler or span limit falls back to ``OTEL_TRACES_SAMPLER`` and + ``OTEL_SPAN_ATTRIBUTE_COUNT_LIMIT`` style variables, which are meant for the + host application's own tracing. ``OTEL_SDK_DISABLED`` still applies, as it does to the SDK. + + The tracer carries the SDK's scope name because Langfuse keys on it: spans from any other + scope are treated as foreign OTel traffic and get their raw attributes echoed into metadata. + """ + if os.environ.get("OTEL_SDK_DISABLED", "").strip().lower() == "true": + verbose_logger.warning("OTEL_SDK_DISABLED=true also disables the langfuse callback's export channel") + provider: Final = TracerProvider( + resource=_resource(environment=environment, release=release), + sampler=ALWAYS_ON if sample_rate >= 1 else TraceIdHashSampler(sample_rate), + id_generator=_RequestedIdGenerator(), + span_limits=_SPAN_LIMITS, + ) + ledger: Final = _ExportLedger(exporter) + provider.add_span_processor( + BatchSpanProcessor( + ledger, + max_queue_size=_MAX_QUEUE_SIZE, + max_export_batch_size=flush_at, + schedule_delay_millis=flush_interval_millis, + ) + ) + return LangfuseTracing(provider=provider, tracer=provider.get_tracer(_TRACER_NAME), ledger=ledger) + + +@dataclass(frozen=True, slots=True) +class _CachedPrompt: + prompt: PromptClient + fetched_at: float + + +_PromptKey = tuple[str, int | None, str | None] + + +def _prompt_client(prompt: Prompt) -> PromptClient: + return ChatPromptClient(prompt) if isinstance(prompt, Prompt_Chat) else TextPromptClient(prompt) + + +@dataclass(frozen=True, slots=True) +class AuthCheckFailure: + reason: str + + +def _auth_check_failure(reason: str) -> AuthCheckFailure: + verbose_logger.warning("Langfuse auth check failed: %s", reason) + return AuthCheckFailure(reason) + + +class _ApiErrorDetail(BaseModel): + """The status and body of an ``ApiError``, whose own ``str`` also dumps every response header.""" + + model_config = ConfigDict(frozen=True, from_attributes=True) + status_code: int | None + body: object + + +def _api_error_reason(error: ApiError) -> str: + detail: Final = _ApiErrorDetail.model_validate(error) + return f"status_code: {detail.status_code}, body: {detail.body}" + + +class LangfusePromptError(Exception): + """An ``ApiError`` without its ``headers``, which the proxy would otherwise forward to its own client.""" + + def __init__(self, error: ApiError) -> None: + detail: Final = _ApiErrorDetail.model_validate(error) + super().__init__(f"status_code: {detail.status_code}, body: {detail.body}") + self.status_code: Final = detail.status_code + self.body: Final = detail.body + + +def _is_server_error(error: ApiError) -> bool: + return error.status_code is not None and error.status_code >= 500 + + +class LangfuseApiClient: + """litellm's handle on one Langfuse project over its REST API: prompts, ``auth_check`` and the project id. + + The SDK's ``Langfuse`` client is deliberately not constructed. It keeps one tracing bundle per + public key and hands it to every ``Langfuse()`` a host application builds for the same key, so + litellm's exporter, host and masking would leak into that application. Observations travel + over ``LangfuseTracing``; nothing here exports spans. + + Prompts are cached for ``LANGFUSE_PROMPT_CACHE_DEFAULT_TTL_SECONDS`` (60 by default) as the SDK + does. A stale prompt is served at once and refreshed on a background thread, so the request + that finds it stale, and the event loop it runs on, never wait for the REST round trip; a + refresh that fails keeps serving the stale prompt rather than failing the request, again like the SDK. + """ + + def __init__(self, api: LangfuseAPI, *, prompt_cache_ttl_seconds: float) -> None: + self.api: Final = api + self.prompt_cache_ttl_seconds: Final = prompt_cache_ttl_seconds + # mutable-ok: per-client prompt cache, guarded by _lock + self._prompts: Final[dict[_PromptKey, _CachedPrompt]] = {} + # mutable-ok: keys with a refresh in flight, guarded by _lock + self._refreshing: Final[set[_PromptKey]] = set() + self._lock: Final = threading.Lock() + + def auth_check(self) -> AuthCheckFailure | None: + """``None`` when the keys reach a project; otherwise the reason, which is also logged. + + Mirrors the SDK's ``Langfuse.auth_check``: a 200 with no project is a failure too, and a server + error or a transport failure is reported as itself rather than as bad credentials. + """ + try: + projects: Final = self.api.projects.get(request_options=_NO_REST_RETRIES).data + except ApiError as error: + return _auth_check_failure(_api_error_reason(error)) + except Exception as error: # noqa: BLE001 # httpx transport errors or a body the response model rejects + return _auth_check_failure(str(error) or type(error).__name__) + if not projects: + return _auth_check_failure("no project found for the keys provided") + return None + + def project_id(self) -> str | None: + projects: Final = self.api.projects.get(request_options=_NO_REST_RETRIES).data + return projects[0].id if projects else None + + def get_prompt(self, name: str, *, label: str | None = None, version: int | None = None) -> PromptClient: + key: Final[_PromptKey] = (name, version, label) + with self._lock: + cached: Final = self._prompts.get(key) + if cached is None: + return self._fetch(key) + if monotonic() - cached.fetched_at >= self.prompt_cache_ttl_seconds: + self._refresh_in_background(key) + return cached.prompt + + def _fetch(self, key: _PromptKey) -> PromptClient: + fetched: Final = _prompt_client(self._request_prompt(key)) + with self._lock: + self._prompts[key] = _CachedPrompt(prompt=fetched, fetched_at=monotonic()) + return fetched + + def _request_prompt(self, key: _PromptKey) -> Prompt: + """Retried once, at once, after a 5xx or a transport failure: a cold miss runs on the caller's event + loop, so the generated client's sleeping retries stay off.""" + name, version, label = key + request: Final = partial( + self.api.prompts.get, quote(name, safe=""), version=version, label=label, request_options=_NO_REST_RETRIES + ) + try: + return request() + except ApiError as error: + if not _is_server_error(error): + raise LangfusePromptError(error) from None + verbose_logger.debug("Langfuse prompt %r fetch failed (%s), retrying once", name, _api_error_reason(error)) + except httpx.TransportError as error: + verbose_logger.debug("Langfuse prompt %r fetch failed (%s), retrying once", name, error) + try: + return request() + except ApiError as error: + raise LangfusePromptError(error) from None + + def _refresh_in_background(self, key: _PromptKey) -> None: + with self._lock: + if key in self._refreshing: + return + self._refreshing.add(key) + threading.Thread(target=self._refresh, args=(key,), name="langfuse-prompt-refresh", daemon=True).start() + + def _refresh(self, key: _PromptKey) -> None: + try: + self._fetch(key) + except Exception as error: # noqa: BLE001 # a failed refresh keeps the stale prompt in service + verbose_logger.warning("Langfuse prompt %r refresh failed, serving the cached version: %s", key[0], error) + finally: + with self._lock: + self._refreshing.discard(key) + + +def build_langfuse_client( + *, + public_key: str | None, + secret_key: str | None, + base_url: str, + httpx_client: httpx.Client | None, +) -> LangfuseApiClient: + """The REST client for prompt management, ``auth_check`` and the Slack project link. + + Missing keys are passed through as absent credentials: the server answers 401, which + ``auth_check`` reports as a failure rather than raising at construction. + """ + return LangfuseApiClient( + LangfuseAPI( + base_url=base_url, + username=public_key, + password=secret_key, + x_langfuse_sdk_name="python", + x_langfuse_sdk_version=version("langfuse"), + x_langfuse_public_key=public_key, + httpx_client=httpx_client, + timeout=configured_timeout(), + ), + prompt_cache_ttl_seconds=configured_prompt_cache_ttl(), + ) diff --git a/litellm/litellm_core_utils/litellm_logging.py b/litellm/litellm_core_utils/litellm_logging.py index 8e5af4e5cd6..83ab2bc11a2 100644 --- a/litellm/litellm_core_utils/litellm_logging.py +++ b/litellm/litellm_core_utils/litellm_logging.py @@ -36,7 +36,7 @@ from litellm._logging import ( ) from litellm._uuid import uuid from litellm.batches.batch_utils import _handle_completed_batch, batch_cost_is_final -from litellm.caching.caching import DualCache, InMemoryCache +from litellm.caching.caching import DualCache from litellm.caching.caching_handler import LLMCachingHandler from litellm.constants import ( DEFAULT_MOCK_RESPONSE_COMPLETION_TOKEN_COUNT, @@ -221,6 +221,7 @@ from .initialize_dynamic_callback_params import ( initialize_standard_callback_dynamic_params as _initialize_standard_callback_dynamic_params, ) from .specialty_caches.dynamic_logging_cache import DynamicLoggingCache +from .specialty_caches.service_trace_id_cache import in_memory_trace_id_cache if TYPE_CHECKING: from mcp.types import CallToolResult, EmbeddedResource, ImageContent, TextContent @@ -349,21 +350,6 @@ last_fetched_at_keys: Final = None #### -class ServiceTraceIDCache: - def __init__(self) -> None: - self.cache = InMemoryCache() - - def get_cache(self, litellm_call_id: str, service_name: str) -> str | None: - key_name: Final = f"{service_name}:{litellm_call_id}" - response: Final = self.cache.get_cache(key=key_name) - return response - - def set_cache(self, litellm_call_id: str, service_name: str, trace_id: str) -> None: - key_name: Final = f"{service_name}:{litellm_call_id}" - self.cache.set_cache(key=key_name, value=trace_id) - - -in_memory_trace_id_cache: Final = ServiceTraceIDCache() in_memory_dynamic_logger_cache: Final = DynamicLoggingCache() # Cached lazy import for PrometheusLogger @@ -3979,40 +3965,6 @@ class Logging(LiteLLMLoggingBaseClass): return trace_id - def _get_callback_object(self, service_name: Literal["langfuse"]) -> Any | None: - """ - Return dynamic callback object. - - Meant to solve issue when doing key-based/team-based logging - """ - global langFuseLogger - - if service_name == "langfuse": - if langFuseLogger is None or ( - ( - self.standard_callback_dynamic_params.get("langfuse_public_key") is not None - and self.standard_callback_dynamic_params.get("langfuse_public_key") != langFuseLogger.public_key - ) - or ( - self.standard_callback_dynamic_params.get("langfuse_public_key") is not None - and self.standard_callback_dynamic_params.get("langfuse_public_key") != langFuseLogger.public_key - ) - or ( - self.standard_callback_dynamic_params.get("langfuse_host") is not None - and self.standard_callback_dynamic_params.get("langfuse_host") != langFuseLogger.langfuse_host - ) - ): - return LangFuseLogger( - langfuse_public_key=self.standard_callback_dynamic_params.get("langfuse_public_key"), - langfuse_secret=self.standard_callback_dynamic_params.get("langfuse_secret") - or self.standard_callback_dynamic_params.get("langfuse_secret_key"), - langfuse_host=self.standard_callback_dynamic_params.get("langfuse_host"), - allow_env_credentials=self.standard_callback_dynamic_params.get("langfuse_host") is None, - ) - return langFuseLogger - - return None - def handle_sync_success_callbacks_for_async_calls( self, result: Any, diff --git a/litellm/litellm_core_utils/specialty_caches/dynamic_logging_cache.py b/litellm/litellm_core_utils/specialty_caches/dynamic_logging_cache.py index da3ac366bfd..73aca909ce3 100644 --- a/litellm/litellm_core_utils/specialty_caches/dynamic_logging_cache.py +++ b/litellm/litellm_core_utils/specialty_caches/dynamic_logging_cache.py @@ -1,10 +1,8 @@ """ This is a cache for LangfuseLoggers. -Langfuse Python SDK initializes a thread for each client. - This ensures we do -1. Proper cleanup of Langfuse initialized clients. +1. Release the initialized-client slot a LangfuseLogger holds when it expires. 2. Re-use created langfuse clients. """ @@ -21,45 +19,34 @@ from ...caching import InMemoryCache class LangfuseInMemoryCache(InMemoryCache): """ - Ensures we do proper cleanup of Langfuse initialized clients. + Decrements ``litellm.initialized_langfuse_clients`` when a LangFuseLogger entry expires. - Langfuse Python SDK initializes a thread for each client, we need to call Langfuse.shutdown() to properly cleanup. - - This ensures we do proper cleanup of Langfuse initialized clients. + The counter is a soft budget: loggers built concurrently for one credential set before the + first lands in the cache each take a slot, and only the cached one gives it back on expiry. + The logger's ``stop()`` below hands its shared export channel back + (https://github.com/BerriAI/litellm/issues/11169). """ def _remove_key(self, key: str) -> None: - """ - Override _remove_key in InMemoryCache to ensure we do proper cleanup of Langfuse initialized clients. - - LangfuseLoggers consume threads when initalized, this shuts them down when they are expired - - Relevant Issue: https://github.com/BerriAI/litellm/issues/11169 - """ from litellm.integrations.langfuse.langfuse import LangFuseLogger - if isinstance(self.cache_dict[key], LangFuseLogger): - _created_langfuse_logger: Final[LangFuseLogger] = self.cache_dict[key] - ######################################################### - # Clean up Langfuse initialized clients - ######################################################### + evicted: Final = self.cache_dict.pop(key, None) + self.ttl_dict.pop(key, None) + if evicted is None: + return + + if isinstance(evicted, LangFuseLogger): litellm.initialized_langfuse_clients -= 1 - _created_langfuse_logger.Langfuse.flush() - _created_langfuse_logger.Langfuse.shutdown() # Loggers with a periodic flush task (e.g. NewRelicMetricsLogger) expose # stop() so eviction actually ends the task instead of leaking it. - _evicted_stop: Final = getattr(self.cache_dict[key], "stop", None) - if callable(_evicted_stop): - try: - _evicted_stop() - except Exception: # noqa: BLE001 # a failing stop() must not block eviction - verbose_logger.debug("DynamicLoggingCache: stop() raised during eviction", exc_info=True) - - ######################################################### - # Call parent class to remove key from cache - ######################################################### - return super()._remove_key(key) + _evicted_stop: Final = getattr(evicted, "stop", None) + if not callable(_evicted_stop): + return + try: + _evicted_stop() + except Exception: # noqa: BLE001 # a failing stop() must not block eviction + verbose_logger.debug("DynamicLoggingCache: stop() raised during eviction", exc_info=True) class DynamicLoggingCache: diff --git a/litellm/litellm_core_utils/specialty_caches/service_trace_id_cache.py b/litellm/litellm_core_utils/specialty_caches/service_trace_id_cache.py new file mode 100644 index 00000000000..f1f60d3e7b8 --- /dev/null +++ b/litellm/litellm_core_utils/specialty_caches/service_trace_id_cache.py @@ -0,0 +1,20 @@ +from typing import Final + +from ...caching import InMemoryCache + + +class ServiceTraceIDCache: + def __init__(self) -> None: + self.cache = InMemoryCache() + + def get_cache(self, litellm_call_id: str, service_name: str) -> str | None: + key_name: Final = f"{service_name}:{litellm_call_id}" + response: Final = self.cache.get_cache(key=key_name) + return response + + def set_cache(self, litellm_call_id: str, service_name: str, trace_id: str) -> None: + key_name: Final = f"{service_name}:{litellm_call_id}" + self.cache.set_cache(key=key_name, value=trace_id) + + +in_memory_trace_id_cache: Final = ServiceTraceIDCache() diff --git a/litellm/proxy/health_endpoints/_health_endpoints.py b/litellm/proxy/health_endpoints/_health_endpoints.py index f8801e65c82..fbd4d57bf77 100644 --- a/litellm/proxy/health_endpoints/_health_endpoints.py +++ b/litellm/proxy/health_endpoints/_health_endpoints.py @@ -395,7 +395,9 @@ async def health_services_endpoint( from litellm.integrations.langfuse.langfuse import LangFuseLogger langfuse_logger: Final = LangFuseLogger() - langfuse_logger.Langfuse.auth_check() + auth_failure: Final = langfuse_logger.api_client.auth_check() + if auth_failure is not None: + raise ValueError(f"langfuse auth_check failed: {auth_failure.reason}") _ = litellm.completion( model="openai/litellm-mock-response-model", messages=[{"role": "user", "content": "Hey, how's it going?"}], diff --git a/litellm/proxy/proxy_server.py b/litellm/proxy/proxy_server.py index a5ae7ec0e44..ed4ea347c2e 100644 --- a/litellm/proxy/proxy_server.py +++ b/litellm/proxy/proxy_server.py @@ -68,6 +68,7 @@ from litellm.constants import ( DEFAULT_SHARED_HEALTH_CHECK_LOCK_TTL, DEFAULT_SHARED_HEALTH_CHECK_TTL, DEFAULT_SLACK_ALERTING_THRESHOLD, + LANGFUSE_SHUTDOWN_FLUSH_TIMEOUT_MILLIS, LITELLM_EMBEDDING_PROVIDERS_SUPPORTING_INPUT_ARRAY_OF_TOKENS, LITELLM_SETTINGS_SAFE_DB_OVERRIDES, LITELLM_UI_ALLOW_HEADERS, @@ -1122,17 +1123,21 @@ async def proxy_shutdown_event(worker_heartbeat: ProxyWorkerHeartbeat | None = N if shutdown_billing_metrics_recorder is not None: shutdown_billing_metrics_recorder() - # flush remaining langfuse logs - if "langfuse" in litellm.success_callback: + if "litellm.integrations.langfuse.langfuse_sdk" in sys.modules: try: - # flush langfuse logs on shutdow - from litellm.utils import langFuseLogger + from litellm.integrations.langfuse.langfuse_sdk import flush_langfuse_tracing - if langFuseLogger is not None: - langFuseLogger.Langfuse.flush() - except Exception: - # [DO NOT BLOCK shutdown events for this] - pass + flushed: Final = await asyncio.to_thread(flush_langfuse_tracing, LANGFUSE_SHUTDOWN_FLUSH_TIMEOUT_MILLIS) + if flushed: + verbose_proxy_logger.info("Langfuse export channels flushed") + else: + verbose_proxy_logger.warning( + "Langfuse shutdown flush incomplete: a channel did not finish within %dms or a batch was rejected " + "(see the export errors above); remaining spans are left to the background exporter", + LANGFUSE_SHUTDOWN_FLUSH_TIMEOUT_MILLIS, + ) + except Exception as e: # noqa: BLE001 # shutdown must continue even if the flush fails + verbose_proxy_logger.exception("Error flushing Langfuse export channels on shutdown: %s", e) ## RESET CUSTOM VARIABLES ## cleanup_router_config_variables() diff --git a/litellm/types/integrations/langfuse.py b/litellm/types/integrations/langfuse.py index 6742aefea39..fe070a3dd18 100644 --- a/litellm/types/integrations/langfuse.py +++ b/litellm/types/integrations/langfuse.py @@ -14,3 +14,8 @@ class LangfuseUsageDetails(TypedDict): total: int | None cache_creation_input_tokens: int | None cache_read_input_tokens: int | None + + +class LangfuseLoggedEvent(TypedDict): + trace_id: ReadOnly[str | None] + generation_id: ReadOnly[str | None] diff --git a/pyproject.toml b/pyproject.toml index 15eb8f0c4f4..f2364b5e77b 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -171,11 +171,11 @@ proxy-runtime = [ "anthropic[vertex]>=0.84.0,<1.0", "grpcio==1.78.0", "prometheus-client>=0.20.0,<1.0", - "langfuse>=2.59.7,<3.0", - "opentelemetry-api==1.28.0", - "opentelemetry-sdk==1.28.0", - "opentelemetry-exporter-otlp==1.28.0", - "opentelemetry-instrumentation-fastapi==0.49b0", + "langfuse>=4.7,<5.0", + "opentelemetry-api==1.33.1", + "opentelemetry-sdk==1.33.1", + "opentelemetry-exporter-otlp==1.33.1", + "opentelemetry-instrumentation-fastapi==0.54b1", "ddtrace>=4.8.2,<5.0", "sentry-sdk>=2.21.0,<3.0", "mangum>=0.17.0,<1.0", @@ -222,11 +222,11 @@ dev = [ "types-PyYAML==6.0.12.20250915", "botocore-stubs==1.43.14", "types-boto3[bedrock,bedrock-agent,bedrock-runtime,kms,s3,sagemaker-runtime,sts]==1.43.30", - "opentelemetry-api==1.28.0", - "opentelemetry-sdk==1.28.0", - "opentelemetry-exporter-otlp==1.28.0", - "opentelemetry-instrumentation-fastapi==0.49b0", - "langfuse==2.59.7", + "opentelemetry-api==1.33.1", + "opentelemetry-sdk==1.33.1", + "opentelemetry-exporter-otlp==1.33.1", + "opentelemetry-instrumentation-fastapi==0.54b1", + "langfuse>=4.7,<5.0", "fastapi-offline==1.7.6", "fakeredis==2.34.1", "pytest-rerunfailures==15.1", @@ -249,10 +249,10 @@ proxy-dev = [ "prisma==0.11.0", "hypercorn==0.17.3", "prometheus-client==0.20.0", - "opentelemetry-api==1.28.0", - "opentelemetry-sdk==1.28.0", - "opentelemetry-exporter-otlp==1.28.0", - "opentelemetry-instrumentation-fastapi==0.49b0", + "opentelemetry-api==1.33.1", + "opentelemetry-sdk==1.33.1", + "opentelemetry-exporter-otlp==1.33.1", + "opentelemetry-instrumentation-fastapi==0.54b1", "azure-identity==1.25.2", "a2a-sdk==1.1.0", ] @@ -272,7 +272,7 @@ ci = [ "lunary==1.4.36; python_version == '3.10'", "lunary==1.4.37; python_version >= '3.11'", "logfire==4.6.0", - "traceloop-sdk==0.33.12", + "traceloop-sdk==0.34.0", "detect-secrets==1.5.0", "PyGithub==2.8.1", "aiodynamo==24.7", diff --git a/tests/integration/observability/test_langfuse_delivery.py b/tests/integration/observability/test_langfuse_delivery.py new file mode 100644 index 00000000000..5a3ffdb9965 --- /dev/null +++ b/tests/integration/observability/test_langfuse_delivery.py @@ -0,0 +1,270 @@ +import base64 +import json +import time +import uuid +from collections.abc import Sequence +from pathlib import Path +from typing import Final + +import yaml +from integration._support.client import Gateway, eventually +from integration._support.process import owned_proxy +from integration._support.wire import Reply, Request, Wire, wire_server +from opentelemetry.proto.collector.trace.v1.trace_service_pb2 import ExportTraceServiceRequest +from opentelemetry.proto.common.v1.common_pb2 import KeyValue +from opentelemetry.proto.trace.v1.trace_pb2 import Span +from pydantic import BaseModel, TypeAdapter + +PUBLIC_KEY: Final = "pk-lf-integration" +SECRET_KEY: Final = "sk-lf-integration" +PROJECTS_PATH: Final = "/api/public/projects" +TRACES_PATH: Final = "/api/public/otel/v1/traces" +PROMPTS_PATH: Final = "/api/public/v2/prompts/" +_PROXY_CONFIG: Final = TypeAdapter(dict[str, object]) +_SETTINGS: Final = TypeAdapter(dict[str, object]) + + +class _ProviderBody(BaseModel): + messages: list[object] + + +def _completion(text: str) -> Reply: + return Reply( + body=json.dumps( + { + "id": "chatcmpl-" + text, + "object": "chat.completion", + "created": 1, + "model": "gpt-4o-mini", + "choices": [{"index": 0, "message": {"role": "assistant", "content": text}, "finish_reason": "stop"}], + "usage": {"prompt_tokens": 11, "completion_tokens": 4, "total_tokens": 15}, + } + ).encode() + ) + + +def _projects() -> Reply: + return Reply(body=json.dumps({"data": [{"id": "integration-project", "name": "integration"}]}).encode()) + + +def _text_prompt(name: str) -> Reply: + return Reply( + body=json.dumps( + { + "type": "text", + "name": name, + "version": 1, + "prompt": "Say {{word}}", + "config": {}, + "labels": ["production"], + "tags": [], + } + ).encode() + ) + + +def _langfuse_config(tmp_path: Path) -> Path: + config: Final = _PROXY_CONFIG.validate_python( + yaml.safe_load(Path("tests/integration/proxy_config.yaml").read_text()) + ) + settings: Final = { + **_SETTINGS.validate_python(config["litellm_settings"]), + "success_callback": ["langfuse"], + "failure_callback": ["langfuse"], + } + path: Final = tmp_path / "langfuse.yaml" + path.write_text(yaml.safe_dump({**config, "litellm_settings": settings})) + return path + + +def _langfuse_environment(langfuse: Wire) -> dict[str, str]: + return { + "LANGFUSE_HOST": langfuse.url, + "LANGFUSE_PUBLIC_KEY": PUBLIC_KEY, + "LANGFUSE_SECRET_KEY": SECRET_KEY, + "LANGFUSE_FLUSH_INTERVAL": "1", + } + + +def _attribute(entries: Sequence[KeyValue], key: str) -> str | list[str] | None: + for entry in entries: + if entry.key != key: + continue + if entry.value.HasField("array_value"): + return [item.string_value for item in entry.value.array_value.values] + return entry.value.string_value + return None + + +def _spans(batches: Sequence[Request]) -> tuple[Span, ...]: + return tuple( + span + for batch in batches + if batch.target == TRACES_PATH and batch.headers.get("content-type") == "application/x-protobuf" + for resource_spans in ExportTraceServiceRequest.FromString(batch.body).resource_spans + for scope_spans in resource_spans.scope_spans + for span in scope_spans.spans + ) + + +def test_langfuse_callback_delivers_the_generation_over_otlp_v4_with_the_caller_trace_fields( + gateway: Gateway, tmp_path: Path +) -> None: + marker: Final = "langfuse" + uuid.uuid4().hex + trace_id: Final = uuid.uuid4().hex + provider_secret: Final = "synthetic-provider-secret-" + marker + + def upstream(request: Request) -> Reply: + assert request.headers["authorization"] == f"Bearer {provider_secret}" + return _completion(marker + "-answer") + + def langfuse(request: Request) -> Reply: + if request.method == "GET" and request.target.startswith(PROJECTS_PATH): + return _projects() + return Reply(body=b"", content_type="application/x-protobuf") + + with ( + wire_server(upstream) as provider, + wire_server(langfuse) as destination, + owned_proxy( + gateway, tmp_path, _langfuse_environment(destination), config=_langfuse_config(tmp_path) + ) as candidate, + candidate.scenario() as scenario, + ): + model: Final = scenario.model(api_base=provider.url + "/v1", api_key=provider_secret) + response: Final = candidate.request( + "POST", + "/v1/chat/completions", + { + "model": model, + "messages": [{"role": "user", "content": marker + "-question"}], + "metadata": { + "trace_id": trace_id, + "trace_name": marker + "-trace", + "generation_name": marker, + "trace_user_id": marker + "-user", + "session_id": marker + "-session", + "tags": [marker], + }, + "cache": {"no-cache": True}, + }, + ) + assert response.status_code == 200, response.text + received: Final[list[Request]] = [] # mutable-ok: drain() consumes the queue, later polls keep earlier ones + + def exported() -> tuple[Span, ...]: + received.extend(destination.drain()) + return tuple(span for span in _spans(received) if span.name == marker) + + spans: Final = eventually(exported, lambda values: len(values) == 1, seconds=20) + span: Final = spans[0] + posts: Final = tuple(request for request in received if request.method == "POST") + assert {request.target for request in posts} == {TRACES_PATH}, [request.target for request in received] + basic: Final = "Basic " + base64.b64encode(f"{PUBLIC_KEY}:{SECRET_KEY}".encode()).decode() + for request in posts: + assert request.headers["authorization"] == basic + assert request.headers["content-type"] == "application/x-protobuf" + assert request.headers["x-langfuse-ingestion-version"] == "4" + assert provider_secret.encode() not in request.body + assert candidate.key.encode() not in request.body + + assert span.trace_id.hex() == trace_id + assert span.parent_span_id == b"" + attributes: Final = span.attributes + assert _attribute(attributes, "langfuse.observation.type") == "generation" + assert _attribute(attributes, "langfuse.trace.name") == marker + "-trace" + assert _attribute(attributes, "user.id") == marker + "-user" + assert _attribute(attributes, "session.id") == marker + "-session" + assert marker in (_attribute(attributes, "langfuse.trace.tags") or ()) + assert _attribute(attributes, "langfuse.observation.model.name") == "openai/gpt-4o-mini" + assert json.loads(str(_attribute(attributes, "langfuse.observation.usage_details"))) == { + "input": 11, + "output": 4, + "total": 15, + "cache_creation_input_tokens": 0, + "cache_read_input_tokens": 0, + } + assert marker + "-question" in str(_attribute(attributes, "langfuse.observation.input")) + assert marker + "-answer" in str(_attribute(attributes, "langfuse.observation.output")) + assert ( + _attribute(attributes, "langfuse.observation.metadata.litellm_call_id") + == response.headers["x-litellm-call-id"] + ) + + +def test_prompt_fetch_encodes_the_name_retries_a_5xx_once_and_keeps_langfuse_headers_off_the_client( + gateway: Gateway, tmp_path: Path +) -> None: + marker: Final = "prompt" + uuid.uuid4().hex + leak: Final = "leak-" + marker + flaky_prompt: Final = f"{marker}/what?" + encoded_flaky_prompt: Final = f"{marker}%2Fwhat%3F" + missing_prompt: Final = marker + "-missing" + + seen_prompt_gets: Final[list[str]] = [] # mutable-ok: the double counts attempts across requests + + def upstream(request: Request) -> Reply: + return _completion(marker + "-answer") + + def langfuse(request: Request) -> Reply: + if request.method == "GET" and request.target.startswith(PROJECTS_PATH): + return _projects() + if request.method == "POST": + return Reply(body=b"", content_type="application/x-protobuf") + assert request.target.startswith(PROMPTS_PATH), request.target + assert request.headers["authorization"].startswith("Basic ") + if request.target.startswith(PROMPTS_PATH + encoded_flaky_prompt): + prior: Final = sum(1 for seen in seen_prompt_gets if seen.startswith(PROMPTS_PATH + encoded_flaky_prompt)) + seen_prompt_gets.append(request.target) + if prior == 0: + return Reply(status=503, body=b'{"message":"try later"}', headers={"retry-after": "30"}) + return _text_prompt(flaky_prompt) + seen_prompt_gets.append(request.target) + return Reply( + status=404, + body=b'{"message":"Prompt not found","error":"LangfuseNotFoundError"}', + headers={"set-cookie": f"session={leak}; Path=/", "x-upstream-internal": leak, "server": leak}, + ) + + with ( + wire_server(upstream) as provider, + wire_server(langfuse) as destination, + owned_proxy( + gateway, tmp_path, _langfuse_environment(destination), config=_langfuse_config(tmp_path) + ) as candidate, + candidate.scenario() as scenario, + ): + flaky: Final = scenario.model( + model="langfuse/gpt-4o-mini", prompt_id=flaky_prompt, api_base=provider.url + "/v1", api_key="synthetic" + ) + missing: Final = scenario.model( + model="langfuse/gpt-4o-mini", prompt_id=missing_prompt, api_base=provider.url + "/v1", api_key="synthetic" + ) + started: Final = time.monotonic() + response: Final = candidate.request( + "POST", + "/v1/chat/completions", + {"model": flaky, "messages": [{"role": "user", "content": marker}], "prompt_variables": {"word": marker}}, + ) + elapsed: Final = time.monotonic() - started + assert response.status_code == 200, response.text + assert elapsed < 5, f"a retried cold prompt miss took {elapsed:.1f}s" + attempts: Final = tuple( + target for target in seen_prompt_gets if target.startswith(PROMPTS_PATH + encoded_flaky_prompt) + ) + assert len(attempts) == 2, seen_prompt_gets + assert all(target.split("?", 1)[0] == PROMPTS_PATH + encoded_flaky_prompt for target in attempts), attempts + sent: Final = _ProviderBody.model_validate_json(provider.drain()[-1].body).messages + assert any("Say " + marker in json.dumps(message) for message in sent), sent + + failure: Final = candidate.request( + "POST", + "/v1/chat/completions", + {"model": missing, "messages": [{"role": "user", "content": marker}], "prompt_variables": {"word": marker}}, + ) + assert failure.status_code == 404, failure.text + assert "Prompt not found" in failure.text + assert leak not in failure.text + assert leak not in json.dumps(dict(failure.headers)) + assert "set-cookie" not in failure.headers and "x-upstream-internal" not in failure.headers + assert sum(1 for target in seen_prompt_gets if target.startswith(PROMPTS_PATH + missing_prompt)) == 1 diff --git a/tests/litellm_utils_tests/test_utils.py b/tests/litellm_utils_tests/test_utils.py index fb20cdf7e0e..92947fbf6fe 100644 --- a/tests/litellm_utils_tests/test_utils.py +++ b/tests/litellm_utils_tests/test_utils.py @@ -842,6 +842,7 @@ def test_logging_trace_id(langfuse_trace_id, langfuse_existing_trace_id): """ - Unit test for `_get_trace_id` function in Logging obj """ + from litellm.integrations.langfuse.langfuse_sdk import resolve_trace_id from litellm.litellm_core_utils.litellm_logging import Logging litellm.success_callback = ["langfuse"] @@ -874,24 +875,18 @@ def test_logging_trace_id(langfuse_trace_id, langfuse_existing_trace_id): time.sleep(3) assert litellm_logging_obj._get_trace_id(service_name="langfuse") is not None - ## if existing_trace_id exists + # langfuse addresses a trace by a 32-hex id, so the id litellm reports back is the + # resolved form of whichever source won; that is what the alerting deep link needs if langfuse_existing_trace_id is not None: - assert ( - litellm_logging_obj._get_trace_id(service_name="langfuse") - == langfuse_existing_trace_id - ) - ## if trace_id exists + expected_source = langfuse_existing_trace_id elif langfuse_trace_id is not None: - assert ( - litellm_logging_obj._get_trace_id(service_name="langfuse") - == langfuse_trace_id - ) - ## if no trace_id or existing_trace_id is provided, use litellm_trace_id + expected_source = langfuse_trace_id else: - assert ( - litellm_logging_obj._get_trace_id(service_name="langfuse") - == litellm_logging_obj.litellm_trace_id - ) + expected_source = litellm_logging_obj.litellm_trace_id + + assert litellm_logging_obj._get_trace_id(service_name="langfuse") == resolve_trace_id( + expected_source + ) def test_convert_model_response_object(): diff --git a/tests/local_testing/test_alangfuse.py b/tests/local_testing/test_alangfuse.py index 7b1f7f203e3..a9d111843fd 100644 --- a/tests/local_testing/test_alangfuse.py +++ b/tests/local_testing/test_alangfuse.py @@ -11,6 +11,7 @@ logging.basicConfig(level=logging.DEBUG) import litellm from litellm import completion from litellm.caching import InMemoryCache +from litellm.integrations.langfuse.langfuse_sdk import resolve_trace_id litellm.num_retries = 3 litellm.success_callback = ["langfuse"] @@ -36,7 +37,7 @@ def langfuse_client(): langfuse_client = langfuse.Langfuse( public_key=os.environ["LANGFUSE_PUBLIC_KEY"], secret_key=os.environ["LANGFUSE_SECRET_KEY"], - host="https://us.cloud.langfuse.com", + host=os.environ.get("LANGFUSE_HOST", "https://us.cloud.langfuse.com"), ) litellm.in_memory_llm_clients_cache.set_cache( key=_langfuse_cache_key, @@ -227,29 +228,27 @@ async def test_langfuse_logging_without_request_response(stream, langfuse_client print(chunk) langfuse_client.flush() - await asyncio.sleep(5) - # get trace with _unique_trace_name - trace = langfuse_client.get_generations(trace_id=_unique_trace_name) - - print("trace_from_langfuse", trace) - - _trace_data = trace.data - - if ( - len(_trace_data) == 0 - ): # prevent infrequent list index out of range error from langfuse api - return + for _ in range(30): + _trace_data = langfuse_client.api.observations.get_many( + trace_id=resolve_trace_id(_unique_trace_name), + type="GENERATION", + fields="core,io", + ).data + if _trace_data: + break + await asyncio.sleep(3) print(f"_trace_data: {_trace_data}") - assert _trace_data[0].input == { + assert json.loads(_trace_data[0].input) == { "messages": [{"content": "redacted-by-litellm", "role": "user"}] } - assert _trace_data[0].output == { + assert json.loads(_trace_data[0].output) == { "role": "assistant", "content": "redacted-by-litellm", "function_call": None, "tool_calls": None, + "provider_specific_fields": None, } except Exception as e: diff --git a/tests/logging_callback_tests/langfuse_expected_request_body/completion.json b/tests/logging_callback_tests/langfuse_expected_request_body/completion.json index e252e8a128f..906b1a42a8f 100644 --- a/tests/logging_callback_tests/langfuse_expected_request_body/completion.json +++ b/tests/logging_callback_tests/langfuse_expected_request_body/completion.json @@ -1,99 +1,38 @@ { - "batch": [ - { - "id": "7e00e081-468b-4fe9-a409-eb12ac7d3d2d", - "type": "trace-create", - "body": { - "id": "litellm-test-793c217f-9417-4e77-84a7-8dcc16e5b72b", - "timestamp": "2025-01-16T19:28:55.124873Z", - "name": "litellm-acompletion", - "input": { - "messages": [ - { - "role": "user", - "content": "Hello!" - } - ] - }, - "output": { - "content": "Hello! How can I assist you today?", - "role": "assistant", - "tool_calls": null, - "function_call": null, - "provider_specific_fields": null - }, - "tags": [] - }, - "timestamp": "2025-01-16T19:28:55.125002Z" + "name": "litellm-acompletion", + "parent_span_id": null, + "attributes": { + "langfuse.observation.cost_details": { + "total": 3.5e-05 }, - { - "id": "b9ec2c0f-18df-46c7-9e90-624c60bf78ee", - "type": "generation-create", - "body": { - "name": "litellm-acompletion", - "startTime": "2025-01-16T11:28:54.796360-08:00", - "metadata": { - "hidden_params": { - "model_id": null, - "cache_key": null, - "api_base": "https://api.openai.com", - "response_cost": 3.5e-05, - "additional_headers": {}, - "litellm_overhead_time_ms": null, - "batch_models": null, - "litellm_model_name": "gpt-3.5-turbo", - "usage_object": null - }, - "litellm_response_cost": 3.5e-05, - "cache_hit": false, - "requester_metadata": {} - }, - "input": { - "messages": [ - { - "role": "user", - "content": "Hello!" - } - ] - }, - "output": { - "content": "Hello! How can I assist you today?", - "role": "assistant", - "tool_calls": null, - "function_call": null, - "provider_specific_fields": null - }, - "level": "DEFAULT", - "id": "time-11-28-54-796360_chatcmpl-521e530f-5e29-4d0a-8d1a-58fca0a847c2", - "endTime": "2025-01-16T11:28:55.124353-08:00", - "completionStartTime": "2025-01-16T11:28:55.124353-08:00", - "model": "gpt-3.5-turbo", - "modelParameters": { - "extra_body": "{}" - }, - "usage": { - "input": 10, - "output": 20, - "unit": "TOKENS", - "totalCost": 3.5e-05 - }, - "usageDetails": { - "input": 10, - "output": 20, - "total": 30, - "cache_creation_input_tokens": 0, - "cache_read_input_tokens": 0 - }, - "traceId": "litellm-test-6a51ae70-a4e7-499e-afcd-dce2a3b31850" - }, - "timestamp": "2025-01-16T19:28:55.125258Z" - } - ], - "metadata": { - "batch_size": 2, - "sdk_integration": "litellm", - "sdk_name": "python", - "sdk_version": "2.44.1", - "public_key": "pk-lf-03734ab3-8790-4c09-b5fb-8c3b663413b6" + "langfuse.observation.input": { + "messages": [ + { + "role": "user", + "content": "Hello!" + } + ] + }, + "langfuse.observation.level": "DEFAULT", + "langfuse.observation.model.name": "gpt-3.5-turbo", + "langfuse.observation.model.parameters": { + "extra_body": "{}" + }, + "langfuse.observation.output": { + "content": "Hello! How can I assist you today?", + "role": "assistant", + "tool_calls": null, + "function_call": null, + "provider_specific_fields": null + }, + "langfuse.observation.type": "generation", + "langfuse.observation.usage_details": { + "input": 10, + "output": 20, + "total": 30, + "cache_creation_input_tokens": 0, + "cache_read_input_tokens": 0 + }, + "langfuse.trace.name": "litellm-acompletion" } -} \ No newline at end of file +} diff --git a/tests/logging_callback_tests/langfuse_expected_request_body/completion_with_bedrock_call.json b/tests/logging_callback_tests/langfuse_expected_request_body/completion_with_bedrock_call.json index dd49d9751f1..6f359380245 100644 --- a/tests/logging_callback_tests/langfuse_expected_request_body/completion_with_bedrock_call.json +++ b/tests/logging_callback_tests/langfuse_expected_request_body/completion_with_bedrock_call.json @@ -1,85 +1,31 @@ { - "batch": [ - { - "id": "3c9b544f-ef3f-449e-8ec1-763acbb56bec", - "type": "trace-create", - "body": { - "id": "litellm-test-c4c1c850-e8c9-4b16-b5a4-bff2bf9fa4f6", - "timestamp": "2025-05-26T21:13:16.796768Z", - "name": "litellm-acompletion", - "input": { - "messages": [ - { - "role": "user", - "content": "Hello!" - } - ] - }, - "tags": [] - }, - "timestamp": "2025-05-26T21:13:16.796875Z" + "name": "litellm-acompletion", + "parent_span_id": null, + "attributes": { + "langfuse.observation.cost_details": { + "total": 6e-05 }, - { - "id": "90e6bc70-05d9-4444-8b87-4523a9a54c17", - "type": "generation-create", - "body": { - "traceId": "litellm-test-c4c1c850-e8c9-4b16-b5a4-bff2bf9fa4f6", - "name": "litellm-acompletion", - "startTime": "2025-05-26T14:13:16.469836-07:00", - "metadata": { - "hidden_params": { - "model_id": null, - "cache_key": null, - "api_base": null, - "response_cost": 6e-05, - "additional_headers": {}, - "litellm_overhead_time_ms": null, - "batch_models": null, - "litellm_model_name": "bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0", - "usage_object": null - }, - "litellm_response_cost": 6e-05, - "cache_hit": false, - "requester_metadata": {} - }, - "input": { - "messages": [ - { - "role": "user", - "content": "Hello!" - } - ] - }, - "level": "DEFAULT", - "id": "time-14-13-16-469836_chatcmpl-3803a9e9-aa68-4493-94d9-247f354830d6", - "endTime": "2025-05-26T14:13:16.795438-07:00", - "completionStartTime": "2025-05-26T14:13:16.795438-07:00", - "model": "bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0", - "modelParameters": { - "aws_region": "us-east-1" - }, - "usage": { - "input": 10, - "output": 10, - "unit": "TOKENS", - "totalCost": 6e-05 - }, - "usageDetails": { - "input": 10, - "output": 10, - "total": 20, - "cache_creation_input_tokens": 0, - "cache_read_input_tokens": 0 + "langfuse.observation.input": { + "messages": [ + { + "role": "user", + "content": "Hello!" } - }, - "timestamp": "2025-05-26T21:13:16.797156Z" - } - ], - "metadata": { - "batch_size": 2, - "sdk_integration": "litellm", - "sdk_name": "python", - "sdk_version": "2.44.1", - "public_key": "pk-lf-3bfc4db9-217f-48e9-92e0-142566e3c204" + ] + }, + "langfuse.observation.level": "DEFAULT", + "langfuse.observation.model.name": "bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0", + "langfuse.observation.model.parameters": { + "aws_region": "us-east-1" + }, + "langfuse.observation.type": "generation", + "langfuse.observation.usage_details": { + "input": 10, + "output": 10, + "total": 20, + "cache_creation_input_tokens": 0, + "cache_read_input_tokens": 0 + }, + "langfuse.trace.name": "litellm-acompletion" } -} \ No newline at end of file +} diff --git a/tests/logging_callback_tests/langfuse_expected_request_body/completion_with_complex_metadata.json b/tests/logging_callback_tests/langfuse_expected_request_body/completion_with_complex_metadata.json index 15794de7a07..906b1a42a8f 100644 --- a/tests/logging_callback_tests/langfuse_expected_request_body/completion_with_complex_metadata.json +++ b/tests/logging_callback_tests/langfuse_expected_request_body/completion_with_complex_metadata.json @@ -1,138 +1,38 @@ { - "batch": [ - { - "id": "9ee9100b-c4aa-4e40-a10d-bc189f8b4242", - "type": "trace-create", - "body": { - "id": "litellm-test-c414db10-dd68-406e-9d9e-03839bc2f346", - "timestamp": "2025-01-22T17:27:51.702596Z", - "name": "litellm-acompletion", - "input": { - "messages": [ - { - "role": "user", - "content": "Hello!" - } - ] - }, - "output": { - "content": "Hello! How can I assist you today?", - "role": "assistant", - "tool_calls": null, - "function_call": null, - "provider_specific_fields": null - }, - "tags": [] - }, - "timestamp": "2025-01-22T17:27:51.702716Z" + "name": "litellm-acompletion", + "parent_span_id": null, + "attributes": { + "langfuse.observation.cost_details": { + "total": 3.5e-05 }, - { - "id": "f8d20489-ed58-429f-b609-87380e223746", - "type": "generation-create", - "body": { - "traceId": "litellm-test-c414db10-dd68-406e-9d9e-03839bc2f346", - "name": "litellm-acompletion", - "startTime": "2025-01-22T09:27:51.150898-08:00", - "metadata": { - "string_value": "hello", - "int_value": 42, - "float_value": 3.14, - "bool_value": true, - "nested_dict": { - "key1": "value1", - "key2": { - "inner_key": "inner_value" - } - }, - "list_value": [ - 1, - 2, - 3 - ], - "set_value": [ - 1, - 2, - 3 - ], - "complex_list": [ - { - "dict_in_list": "value" - }, - "simple_string", - [ - 1, - 2, - 3 - ] - ], - "user": { - "name": "John", - "age": 30, - "tags": [ - "customer", - "active" - ] - }, - "hidden_params": { - "model_id": null, - "cache_key": null, - "api_base": "https://api.openai.com", - "response_cost": 5.4999999999999995e-05, - "additional_headers": {}, - "litellm_overhead_time_ms": null, - "batch_models": null, - "litellm_model_name": "gpt-3.5-turbo", - "usage_object": null - }, - "litellm_response_cost": 5.4999999999999995e-05, - "cache_hit": false, - "requester_metadata": {} - }, - "input": { - "messages": [ - { - "role": "user", - "content": "Hello!" - } - ] - }, - "output": { - "content": "Hello! How can I assist you today?", - "role": "assistant", - "tool_calls": null, - "function_call": null, - "provider_specific_fields": null - }, - "level": "DEFAULT", - "id": "time-09-27-51-150898_chatcmpl-b783291c-dc76-4660-bfef-b79be9d54e57", - "endTime": "2025-01-22T09:27:51.702048-08:00", - "completionStartTime": "2025-01-22T09:27:51.702048-08:00", - "model": "gpt-3.5-turbo", - "modelParameters": { - "extra_body": "{}" - }, - "usage": { - "input": 10, - "output": 20, - "unit": "TOKENS", - "totalCost": 3.5e-05 - }, - "usageDetails": { - "input": 10, - "output": 20, - "total": 30, - "cache_creation_input_tokens": 0, - "cache_read_input_tokens": 0 + "langfuse.observation.input": { + "messages": [ + { + "role": "user", + "content": "Hello!" } - }, - "timestamp": "2025-01-22T17:27:51.703046Z" - } - ], - "metadata": { - "batch_size": 2, - "sdk_integration": "litellm", - "sdk_name": "python", - "sdk_version": "2.44.1", - "public_key": "pk-lf-e02aaea3-8668-4c9f-8c69-771a4ea1f5c9" + ] + }, + "langfuse.observation.level": "DEFAULT", + "langfuse.observation.model.name": "gpt-3.5-turbo", + "langfuse.observation.model.parameters": { + "extra_body": "{}" + }, + "langfuse.observation.output": { + "content": "Hello! How can I assist you today?", + "role": "assistant", + "tool_calls": null, + "function_call": null, + "provider_specific_fields": null + }, + "langfuse.observation.type": "generation", + "langfuse.observation.usage_details": { + "input": 10, + "output": 20, + "total": 30, + "cache_creation_input_tokens": 0, + "cache_read_input_tokens": 0 + }, + "langfuse.trace.name": "litellm-acompletion" } -} \ No newline at end of file +} diff --git a/tests/logging_callback_tests/langfuse_expected_request_body/completion_with_langfuse_metadata.json b/tests/logging_callback_tests/langfuse_expected_request_body/completion_with_langfuse_metadata.json index 8d5d08894ef..5ed49cde972 100644 --- a/tests/logging_callback_tests/langfuse_expected_request_body/completion_with_langfuse_metadata.json +++ b/tests/logging_callback_tests/langfuse_expected_request_body/completion_with_langfuse_metadata.json @@ -1,116 +1,62 @@ { - "batch": [ - { - "id": "872a0a1c-4328-431b-80b6-fd55a8a44477", - "type": "trace-create", - "body": { - "id": "litellm-test-533ffb2d-a0a3-45b5-911c-7940466cdc8e", - "timestamp": "2025-01-22T17:19:11.234960Z", - "name": "test_trace_name", - "userId": "test_user_id", - "input": { - "messages": [ - { - "role": "user", - "content": "Hello!" - } - ] - }, - "output": { - "content": "Hello! How can I assist you today?", - "role": "assistant", - "tool_calls": null, - "function_call": null, - "provider_specific_fields": null - }, - "sessionId": "test_session_id", - "version": "test_trace_version", - "metadata": { - "test_key": "test_value" - }, - "tags": [ - "test_tag", - "test_tag_2" - ] - }, - "timestamp": "2025-01-22T17:19:11.235169Z" + "name": "test_generation_name", + "parent_span_id": "0d9cfbb24ef808cd", + "attributes": { + "langfuse.observation.cost_details": { + "total": 3.5e-05 }, - { - "id": "18d6f044-e522-4376-96e0-7eec765677ed", - "type": "generation-create", - "body": { - "traceId": "litellm-test-533ffb2d-a0a3-45b5-911c-7940466cdc8e", - "name": "test_generation_name", - "startTime": "2025-01-22T09:19:10.957072-08:00", - "metadata": { - "tags": [ - "test_tag", - "test_tag_2" - ], - "parent_observation_id": "test_parent_observation_id", - "version": "test_version", - "hidden_params": { - "model_id": null, - "cache_key": null, - "api_base": "https://api.openai.com", - "response_cost": 3.5e-05, - "additional_headers": {}, - "litellm_overhead_time_ms": null, - "batch_models": null, - "litellm_model_name": "gpt-3.5-turbo", - "usage_object": null - }, - "litellm_response_cost": 3.5e-05, - "cache_hit": false, - "requester_metadata": {} - }, - "input": { - "messages": [ - { - "role": "user", - "content": "Hello!" - } - ] - }, - "output": { - "content": "Hello! How can I assist you today?", - "role": "assistant", - "tool_calls": null, - "function_call": null, - "provider_specific_fields": null - }, - "level": "DEFAULT", - "parentObservationId": "test_parent_observation_id", - "version": "test_version", - "id": "time-09-19-10-957072_chatcmpl-4da65aba-32e4-400d-aaa2-6bfe096d8141", - "endTime": "2025-01-22T09:19:11.234200-08:00", - "completionStartTime": "2025-01-22T09:19:11.234200-08:00", - "model": "gpt-3.5-turbo", - "modelParameters": { - "extra_body": "{}" - }, - "usage": { - "input": 10, - "output": 20, - "unit": "TOKENS", - "totalCost": 3.5e-05 - }, - "usageDetails": { - "input": 10, - "output": 20, - "total": 30, - "cache_creation_input_tokens": 0, - "cache_read_input_tokens": 0 + "langfuse.observation.input": { + "messages": [ + { + "role": "user", + "content": "Hello!" } - }, - "timestamp": "2025-01-22T17:19:11.235541Z" - } - ], - "metadata": { - "batch_size": 2, - "sdk_integration": "litellm", - "sdk_name": "python", - "sdk_version": "2.44.1", - "public_key": "pk-lf-e02aaea3-8668-4c9f-8c69-771a4ea1f5c9" + ] + }, + "langfuse.observation.level": "DEFAULT", + "langfuse.observation.model.name": "gpt-3.5-turbo", + "langfuse.observation.model.parameters": { + "extra_body": "{}" + }, + "langfuse.observation.output": { + "content": "Hello! How can I assist you today?", + "role": "assistant", + "tool_calls": null, + "function_call": null, + "provider_specific_fields": null + }, + "langfuse.observation.type": "generation", + "langfuse.observation.usage_details": { + "input": 10, + "output": 20, + "total": 30, + "cache_creation_input_tokens": 0, + "cache_read_input_tokens": 0 + }, + "langfuse.release": "test_trace_release", + "langfuse.trace.input": { + "messages": [ + { + "role": "user", + "content": "Hello!" + } + ] + }, + "langfuse.trace.metadata.test_key": "test_value", + "langfuse.trace.name": "test_trace_name", + "langfuse.trace.output": { + "content": "Hello! How can I assist you today?", + "role": "assistant", + "tool_calls": null, + "function_call": null, + "provider_specific_fields": null + }, + "langfuse.trace.tags": [ + "test_tag", + "test_tag_2" + ], + "langfuse.version": "test_trace_version", + "session.id": "test_session_id", + "user.id": "test_user_id" } -} \ No newline at end of file +} diff --git a/tests/logging_callback_tests/langfuse_expected_request_body/completion_with_no_choices.json b/tests/logging_callback_tests/langfuse_expected_request_body/completion_with_no_choices.json index ff8419ee392..b5a0737cf39 100644 --- a/tests/logging_callback_tests/langfuse_expected_request_body/completion_with_no_choices.json +++ b/tests/logging_callback_tests/langfuse_expected_request_body/completion_with_no_choices.json @@ -1,85 +1,31 @@ { - "batch": [ - { - "id": "1f1d7517-4602-4c59-a322-7fc0306f1b7a", - "type": "trace-create", - "body": { - "id": "litellm-test-dbadfdfc-f4e7-4f05-8992-984c37359166", - "timestamp": "2025-02-07T00:23:27.669634Z", - "name": "litellm-acompletion", - "input": { - "messages": [ - { - "role": "user", - "content": "Hello!" - } - ] - }, - "tags": [] - }, - "timestamp": "2025-02-07T00:23:27.669809Z" + "name": "litellm-acompletion", + "parent_span_id": null, + "attributes": { + "langfuse.observation.cost_details": { + "total": 1.9999999999999998e-05 }, - { - "id": "fbe610b6-f500-4c7d-8e34-d40a0e8c487b", - "type": "generation-create", - "body": { - "traceId": "litellm-test-dbadfdfc-f4e7-4f05-8992-984c37359166", - "name": "litellm-acompletion", - "startTime": "2025-02-06T16:23:27.220129-08:00", - "metadata": { - "hidden_params": { - "model_id": null, - "cache_key": null, - "api_base": "https://api.openai.com", - "response_cost": 3.5e-05, - "additional_headers": {}, - "litellm_overhead_time_ms": null, - "batch_models": null, - "litellm_model_name": "gpt-3.5-turbo", - "usage_object": null - }, - "litellm_response_cost": 3.5e-05, - "cache_hit": false, - "requester_metadata": {} - }, - "input": { - "messages": [ - { - "role": "user", - "content": "Hello!" - } - ] - }, - "level": "DEFAULT", - "id": "time-16-23-27-220129_chatcmpl-565360d7-965f-4533-9c09-db789af77a7d", - "endTime": "2025-02-06T16:23:27.644253-08:00", - "completionStartTime": "2025-02-06T16:23:27.644253-08:00", - "model": "gpt-3.5-turbo", - "modelParameters": { - "extra_body": "{}" - }, - "usage": { - "input": 10, - "output": 10, - "unit": "TOKENS", - "totalCost": 1.9999999999999998e-05 - }, - "usageDetails": { - "input": 10, - "output": 10, - "total": 20, - "cache_creation_input_tokens": 0, - "cache_read_input_tokens": 0 + "langfuse.observation.input": { + "messages": [ + { + "role": "user", + "content": "Hello!" } - }, - "timestamp": "2025-02-07T00:23:27.670175Z" - } - ], - "metadata": { - "batch_size": 2, - "sdk_integration": "litellm", - "sdk_name": "python", - "sdk_version": "2.44.1", - "public_key": "pk-lf-e02aaea3-8668-4c9f-8c69-771a4ea1f5c9" + ] + }, + "langfuse.observation.level": "DEFAULT", + "langfuse.observation.model.name": "gpt-3.5-turbo", + "langfuse.observation.model.parameters": { + "extra_body": "{}" + }, + "langfuse.observation.type": "generation", + "langfuse.observation.usage_details": { + "input": 10, + "output": 10, + "total": 20, + "cache_creation_input_tokens": 0, + "cache_read_input_tokens": 0 + }, + "langfuse.trace.name": "litellm-acompletion" } -} \ No newline at end of file +} diff --git a/tests/logging_callback_tests/langfuse_expected_request_body/completion_with_router.json b/tests/logging_callback_tests/langfuse_expected_request_body/completion_with_router.json index df99b11d26b..749796e0d04 100644 --- a/tests/logging_callback_tests/langfuse_expected_request_body/completion_with_router.json +++ b/tests/logging_callback_tests/langfuse_expected_request_body/completion_with_router.json @@ -1,95 +1,33 @@ { - "batch": [ - { - "id": "45eb9b25-605c-4c4a-b2b3-8241e079cd31", - "type": "trace-create", - "body": { - "id": "litellm-test-32702f3d-8a1c-4912-a3d6-286e59a9c568", - "timestamp": "2025-05-24T17:01:19.408179Z", - "name": "litellm-acompletion", - "input": { - "messages": [ - { - "role": "user", - "content": "Hello!" - } - ] - }, - "tags": [] - }, - "timestamp": "2025-05-24T17:01:19.408284Z" + "name": "litellm-acompletion", + "parent_span_id": null, + "attributes": { + "langfuse.observation.cost_details": { + "total": 1.9999999999999998e-05 }, - { - "id": "9f5e9b7d-0cea-4776-b4b9-5c2e8f4bad3c", - "type": "generation-create", - "body": { - "traceId": "litellm-test-32702f3d-8a1c-4912-a3d6-286e59a9c568", - "name": "litellm-acompletion", - "startTime": "2025-05-24T10:01:19.142356-07:00", - "metadata": { - "model_group": "gpt-3.5-turbo", - "model_group_size": 1, - "deployment": "gpt-3.5-turbo", - "model_info": { - "id": "0f1cd8f9e6a22e499303d479486395563ea04decade83fe7334dc2f079a857c2", - "db_model": false - }, - "api_base": null, - "hidden_params": { - "model_id": "0f1cd8f9e6a22e499303d479486395563ea04decade83fe7334dc2f079a857c2", - "cache_key": null, - "api_base": "https://api.openai.com", - "response_cost": 3.5e-05, - "additional_headers": {}, - "litellm_overhead_time_ms": null, - "batch_models": null, - "litellm_model_name": "gpt-3.5-turbo", - "usage_object": null - }, - "litellm_response_cost": 3.5e-05, - "cache_hit": false, - "requester_metadata": {} - }, - "input": { - "messages": [ - { - "role": "user", - "content": "Hello!" - } - ] - }, - "level": "DEFAULT", - "id": "time-10-01-19-142356_chatcmpl-16b215b7-e51e-47b0-8fe5-9dd6f226fda1", - "endTime": "2025-05-24T10:01:19.406531-07:00", - "completionStartTime": "2025-05-24T10:01:19.406531-07:00", - "model": "gpt-3.5-turbo", - "modelParameters": { - "stream": false, - "max_retries": 0, - "extra_body": "{}" - }, - "usage": { - "input": 10, - "output": 10, - "unit": "TOKENS", - "totalCost": 1.9999999999999998e-05 - }, - "usageDetails": { - "input": 10, - "output": 10, - "total": 20, - "cache_creation_input_tokens": 0, - "cache_read_input_tokens": 0 + "langfuse.observation.input": { + "messages": [ + { + "role": "user", + "content": "Hello!" } - }, - "timestamp": "2025-05-24T17:01:19.408586Z" - } - ], - "metadata": { - "batch_size": 2, - "sdk_integration": "litellm", - "sdk_name": "python", - "sdk_version": "2.44.1", - "public_key": "pk-lf-3bfc4db9-217f-48e9-92e0-142566e3c204" + ] + }, + "langfuse.observation.level": "DEFAULT", + "langfuse.observation.model.name": "gpt-3.5-turbo", + "langfuse.observation.model.parameters": { + "stream": false, + "max_retries": 0, + "extra_body": "{}" + }, + "langfuse.observation.type": "generation", + "langfuse.observation.usage_details": { + "input": 10, + "output": 10, + "total": 20, + "cache_creation_input_tokens": 0, + "cache_read_input_tokens": 0 + }, + "langfuse.trace.name": "litellm-acompletion" } -} \ No newline at end of file +} diff --git a/tests/logging_callback_tests/langfuse_expected_request_body/completion_with_tags.json b/tests/logging_callback_tests/langfuse_expected_request_body/completion_with_tags.json index fd3d3194a5b..39e26f5957d 100644 --- a/tests/logging_callback_tests/langfuse_expected_request_body/completion_with_tags.json +++ b/tests/logging_callback_tests/langfuse_expected_request_body/completion_with_tags.json @@ -1,106 +1,42 @@ { - "batch": [ - { - "id": "42be960a-5dde-47df-9cbc-1fdd0fdcaa7d", - "type": "trace-create", - "body": { - "id": "litellm-test-f3ab679b-1e1d-43fd-9a9a-f11287aeb339", - "timestamp": "2025-01-22T15:31:28.963419Z", - "name": "litellm-acompletion", - "input": { - "messages": [ - { - "role": "user", - "content": "Hello!" - } - ] - }, - "output": { - "content": "Hello! How can I assist you today?", - "role": "assistant", - "tool_calls": null, - "function_call": null, - "provider_specific_fields": null - }, - "tags": [ - "test_tag", - "test_tag_2" - ] - }, - "timestamp": "2025-01-22T15:31:28.963706Z" + "name": "litellm-acompletion", + "parent_span_id": null, + "attributes": { + "langfuse.observation.cost_details": { + "total": 3.5e-05 }, - { - "id": "5486df5a-3776-4adf-abd0-bd22e51f7fb4", - "type": "generation-create", - "body": { - "traceId": "litellm-test-f3ab679b-1e1d-43fd-9a9a-f11287aeb339", - "name": "litellm-acompletion", - "startTime": "2025-01-22T07:31:28.960749-08:00", - "metadata": { - "tags": [ - "test_tag", - "test_tag_2" - ], - "hidden_params": { - "model_id": null, - "cache_key": null, - "api_base": "https://api.openai.com", - "response_cost": 5.4999999999999995e-05, - "additional_headers": {}, - "litellm_overhead_time_ms": null, - "batch_models": null, - "litellm_model_name": "gpt-3.5-turbo", - "usage_object": null - }, - "litellm_response_cost": 5.4999999999999995e-05, - "cache_hit": false, - "requester_metadata": {} - }, - "input": { - "messages": [ - { - "role": "user", - "content": "Hello!" - } - ] - }, - "output": { - "content": "Hello! How can I assist you today?", - "role": "assistant", - "tool_calls": null, - "function_call": null, - "provider_specific_fields": null - }, - "level": "DEFAULT", - "id": "time-07-31-28-960749_chatcmpl-f06338f0-8c49-45d8-be35-2854a89723c1", - "endTime": "2025-01-22T07:31:28.962389-08:00", - "completionStartTime": "2025-01-22T07:31:28.962389-08:00", - "model": "gpt-3.5-turbo", - "modelParameters": { - "extra_body": "{}" - }, - "usage": { - "input": 10, - "output": 20, - "unit": "TOKENS", - "totalCost": 3.5e-05 - }, - "usageDetails": { - "input": 10, - "output": 20, - "total": 30, - "cache_creation_input_tokens": 0, - "cache_read_input_tokens": 0 + "langfuse.observation.input": { + "messages": [ + { + "role": "user", + "content": "Hello!" } - }, - "timestamp": "2025-01-22T15:31:28.964179Z" - } - ], - "metadata": { - "batch_size": 2, - "sdk_integration": "litellm", - "sdk_name": "python", - "sdk_version": "2.44.1", - "public_key": "pk-lf-e02aaea3-8668-4c9f-8c69-771a4ea1f5c9" + ] + }, + "langfuse.observation.level": "DEFAULT", + "langfuse.observation.model.name": "gpt-3.5-turbo", + "langfuse.observation.model.parameters": { + "extra_body": "{}" + }, + "langfuse.observation.output": { + "content": "Hello! How can I assist you today?", + "role": "assistant", + "tool_calls": null, + "function_call": null, + "provider_specific_fields": null + }, + "langfuse.observation.type": "generation", + "langfuse.observation.usage_details": { + "input": 10, + "output": 20, + "total": 30, + "cache_creation_input_tokens": 0, + "cache_read_input_tokens": 0 + }, + "langfuse.trace.name": "litellm-acompletion", + "langfuse.trace.tags": [ + "test_tag", + "test_tag_2" + ] } -} \ No newline at end of file +} diff --git a/tests/logging_callback_tests/langfuse_expected_request_body/completion_with_tags_stream.json b/tests/logging_callback_tests/langfuse_expected_request_body/completion_with_tags_stream.json index af15f351189..5223538f919 100644 --- a/tests/logging_callback_tests/langfuse_expected_request_body/completion_with_tags_stream.json +++ b/tests/logging_callback_tests/langfuse_expected_request_body/completion_with_tags_stream.json @@ -1,106 +1,42 @@ { - "batch": [ - { - "id": "06b8fa9f-151b-4e74-9fbf-8af5222a7f40", - "type": "trace-create", - "body": { - "id": "litellm-test-54368a51-a382-493c-b0a8-3f1af23e18c4", - "timestamp": "2025-01-22T16:38:26.016582Z", - "name": "litellm-acompletion", - "input": { - "messages": [ - { - "role": "user", - "content": "Hello!" - } - ] - }, - "output": { - "content": "Hello! How can I assist you today?", - "role": "assistant", - "tool_calls": null, - "function_call": null, - "provider_specific_fields": null - }, - "tags": [ - "test_tag_stream", - "test_tag_2_stream" - ] - }, - "timestamp": "2025-01-22T16:38:26.016828Z" + "name": "litellm-acompletion", + "parent_span_id": null, + "attributes": { + "langfuse.observation.cost_details": { + "total": 3.5e-05 }, - { - "id": "4ca1fd78-53e3-41b5-95d9-417b09e3f0eb", - "type": "generation-create", - "body": { - "traceId": "litellm-test-54368a51-a382-493c-b0a8-3f1af23e18c4", - "name": "litellm-acompletion", - "startTime": "2025-01-22T08:38:25.665692-08:00", - "metadata": { - "tags": [ - "test_tag_stream", - "test_tag_2_stream" - ], - "hidden_params": { - "model_id": null, - "cache_key": null, - "api_base": "https://api.openai.com", - "response_cost": 5.4999999999999995e-05, - "additional_headers": {}, - "litellm_overhead_time_ms": null, - "batch_models": null, - "litellm_model_name": "gpt-3.5-turbo", - "usage_object": null - }, - "litellm_response_cost": 5.4999999999999995e-05, - "cache_hit": false, - "requester_metadata": {} - }, - "input": { - "messages": [ - { - "role": "user", - "content": "Hello!" - } - ] - }, - "output": { - "content": "Hello! How can I assist you today?", - "role": "assistant", - "tool_calls": null, - "function_call": null, - "provider_specific_fields": null - }, - "level": "DEFAULT", - "id": "time-08-38-25-665692_chatcmpl-8b67ffb8-4326-4e1b-bf4a-f70930c11c00", - "endTime": "2025-01-22T08:38:26.015666-08:00", - "completionStartTime": "2025-01-22T08:38:26.015666-08:00", - "model": "gpt-3.5-turbo", - "modelParameters": { - "extra_body": "{}" - }, - "usage": { - "input": 10, - "output": 20, - "unit": "TOKENS", - "totalCost": 3.5e-05 - }, - "usageDetails": { - "input": 10, - "output": 20, - "total": 30, - "cache_creation_input_tokens": 0, - "cache_read_input_tokens": 0 + "langfuse.observation.input": { + "messages": [ + { + "role": "user", + "content": "Hello!" } - }, - "timestamp": "2025-01-22T16:38:26.017252Z" - } - ], - "metadata": { - "batch_size": 2, - "sdk_integration": "litellm", - "sdk_name": "python", - "sdk_version": "2.44.1", - "public_key": "pk-lf-e02aaea3-8668-4c9f-8c69-771a4ea1f5c9" + ] + }, + "langfuse.observation.level": "DEFAULT", + "langfuse.observation.model.name": "gpt-3.5-turbo", + "langfuse.observation.model.parameters": { + "extra_body": "{}" + }, + "langfuse.observation.output": { + "content": "Hello! How can I assist you today?", + "role": "assistant", + "tool_calls": null, + "function_call": null, + "provider_specific_fields": null + }, + "langfuse.observation.type": "generation", + "langfuse.observation.usage_details": { + "input": 10, + "output": 20, + "total": 30, + "cache_creation_input_tokens": 0, + "cache_read_input_tokens": 0 + }, + "langfuse.trace.name": "litellm-acompletion", + "langfuse.trace.tags": [ + "test_tag_stream", + "test_tag_2_stream" + ] } -} \ No newline at end of file +} diff --git a/tests/logging_callback_tests/langfuse_expected_request_body/completion_with_vertex_call.json b/tests/logging_callback_tests/langfuse_expected_request_body/completion_with_vertex_call.json index 5998c52659c..d7d292390a0 100644 --- a/tests/logging_callback_tests/langfuse_expected_request_body/completion_with_vertex_call.json +++ b/tests/logging_callback_tests/langfuse_expected_request_body/completion_with_vertex_call.json @@ -1,83 +1,29 @@ { - "batch": [ - { - "id": "7d33d536-2730-4815-8957-80866c09c053", - "type": "trace-create", - "body": { - "id": "litellm-test-72861437-ff5b-4c48-89c0-a143534d9e7a", - "timestamp": "2025-05-26T21:15:40.610459Z", - "name": "litellm-acompletion", - "input": { - "messages": [ - { - "role": "user", - "content": "Hello!" - } - ] - }, - "tags": [] - }, - "timestamp": "2025-05-26T21:15:40.610603Z" + "name": "litellm-acompletion", + "parent_span_id": null, + "attributes": { + "langfuse.observation.cost_details": { + "total": 3.5e-05 }, - { - "id": "ebb5079c-7726-4adb-9616-e1862735e1d8", - "type": "generation-create", - "body": { - "traceId": "litellm-test-72861437-ff5b-4c48-89c0-a143534d9e7a", - "name": "litellm-acompletion", - "startTime": "2025-05-26T14:15:40.349639-07:00", - "metadata": { - "hidden_params": { - "model_id": null, - "cache_key": null, - "api_base": null, - "response_cost": 3.5e-05, - "additional_headers": {}, - "litellm_overhead_time_ms": null, - "batch_models": null, - "litellm_model_name": "vertex_ai/gemini-3-flash-preview", - "usage_object": null - }, - "litellm_response_cost": 3.5e-05, - "cache_hit": false, - "requester_metadata": {} - }, - "input": { - "messages": [ - { - "role": "user", - "content": "Hello!" - } - ] - }, - "level": "DEFAULT", - "id": "time-14-15-40-349639_chatcmpl-59a988d0-7ef1-4dc4-bc18-d2e78961817f", - "endTime": "2025-05-26T14:15:40.607266-07:00", - "completionStartTime": "2025-05-26T14:15:40.607266-07:00", - "model": "gemini-3-flash-preview", - "modelParameters": {}, - "usage": { - "input": 10, - "output": 10, - "unit": "TOKENS", - "totalCost": 3.5e-05 - }, - "usageDetails": { - "input": 10, - "output": 10, - "total": 20, - "cache_creation_input_tokens": 0, - "cache_read_input_tokens": 0 + "langfuse.observation.input": { + "messages": [ + { + "role": "user", + "content": "Hello!" } - }, - "timestamp": "2025-05-26T21:15:40.610953Z" - } - ], - "metadata": { - "batch_size": 2, - "sdk_integration": "litellm", - "sdk_name": "python", - "sdk_version": "2.44.1", - "public_key": "pk-lf-3bfc4db9-217f-48e9-92e0-142566e3c204" + ] + }, + "langfuse.observation.level": "DEFAULT", + "langfuse.observation.model.name": "gemini-3-flash-preview", + "langfuse.observation.model.parameters": {}, + "langfuse.observation.type": "generation", + "langfuse.observation.usage_details": { + "input": 10, + "output": 10, + "total": 20, + "cache_creation_input_tokens": 0, + "cache_read_input_tokens": 0 + }, + "langfuse.trace.name": "litellm-acompletion" } -} \ No newline at end of file +} diff --git a/tests/logging_callback_tests/langfuse_expected_request_body/complex_metadata.json b/tests/logging_callback_tests/langfuse_expected_request_body/complex_metadata.json index 82a115a0899..906b1a42a8f 100644 --- a/tests/logging_callback_tests/langfuse_expected_request_body/complex_metadata.json +++ b/tests/logging_callback_tests/langfuse_expected_request_body/complex_metadata.json @@ -1,113 +1,38 @@ { - "batch": [ - { - "id": "ddf567e5-a1b5-4e38-8a7c-f48bc847f721", - "type": "trace-create", - "body": { - "id": "litellm-test-46551fc7-c916-4a83-aeef-4274b5582ce1", - "timestamp": "2025-01-22T17:59:39.367430Z", - "name": "litellm-acompletion", - "input": { - "messages": [ - { - "role": "user", - "content": "Hello!" - } - ] - }, - "output": { - "content": "Hello! How can I assist you today?", - "role": "assistant", - "tool_calls": null, - "function_call": null, - "provider_specific_fields": null - }, - "tags": [] - }, - "timestamp": "2025-01-22T17:59:39.367707Z" + "name": "litellm-acompletion", + "parent_span_id": null, + "attributes": { + "langfuse.observation.cost_details": { + "total": 3.5e-05 }, - { - "id": "d3eb2c9e-e123-419d-b27b-c8283a505ae8", - "type": "generation-create", - "body": { - "traceId": "litellm-test-46551fc7-c916-4a83-aeef-4274b5582ce1", - "name": "litellm-acompletion", - "startTime": "2025-01-22T09:59:39.362554-08:00", - "metadata": { - "int": 42, - "str": "hello", - "list": [ - 1, - 2, - 3 - ], - "set": [ - 4, - 5 - ], - "dict": { - "nested": "value" - }, - "hidden_params": { - "model_id": null, - "cache_key": null, - "api_base": "https://api.openai.com", - "response_cost": 5.4999999999999995e-05, - "additional_headers": {}, - "litellm_overhead_time_ms": null, - "batch_models": null, - "litellm_model_name": "gpt-3.5-turbo", - "usage_object": null - }, - "litellm_response_cost": 5.4999999999999995e-05, - "cache_hit": false, - "requester_metadata": {} - }, - "input": { - "messages": [ - { - "role": "user", - "content": "Hello!" - } - ] - }, - "output": { - "content": "Hello! How can I assist you today?", - "role": "assistant", - "tool_calls": null, - "function_call": null, - "provider_specific_fields": null - }, - "level": "DEFAULT", - "id": "time-09-59-39-362554_chatcmpl-d20ba1d9-cda6-4773-822e-921ebcd426a0", - "endTime": "2025-01-22T09:59:39.365756-08:00", - "completionStartTime": "2025-01-22T09:59:39.365756-08:00", - "model": "gpt-3.5-turbo", - "modelParameters": { - "extra_body": "{}" - }, - "usage": { - "input": 10, - "output": 20, - "unit": "TOKENS", - "totalCost": 3.5e-05 - }, - "usageDetails": { - "input": 10, - "output": 20, - "total": 30, - "cache_creation_input_tokens": 0, - "cache_read_input_tokens": 0 + "langfuse.observation.input": { + "messages": [ + { + "role": "user", + "content": "Hello!" } - }, - "timestamp": "2025-01-22T17:59:39.368310Z" - } - ], - "metadata": { - "batch_size": 2, - "sdk_integration": "litellm", - "sdk_name": "python", - "sdk_version": "2.44.1", - "public_key": "pk-lf-e02aaea3-8668-4c9f-8c69-771a4ea1f5c9" + ] + }, + "langfuse.observation.level": "DEFAULT", + "langfuse.observation.model.name": "gpt-3.5-turbo", + "langfuse.observation.model.parameters": { + "extra_body": "{}" + }, + "langfuse.observation.output": { + "content": "Hello! How can I assist you today?", + "role": "assistant", + "tool_calls": null, + "function_call": null, + "provider_specific_fields": null + }, + "langfuse.observation.type": "generation", + "langfuse.observation.usage_details": { + "input": 10, + "output": 20, + "total": 30, + "cache_creation_input_tokens": 0, + "cache_read_input_tokens": 0 + }, + "langfuse.trace.name": "litellm-acompletion" } -} \ No newline at end of file +} diff --git a/tests/logging_callback_tests/langfuse_expected_request_body/complex_metadata_2.json b/tests/logging_callback_tests/langfuse_expected_request_body/complex_metadata_2.json index 33e6b01bee3..906b1a42a8f 100644 --- a/tests/logging_callback_tests/langfuse_expected_request_body/complex_metadata_2.json +++ b/tests/logging_callback_tests/langfuse_expected_request_body/complex_metadata_2.json @@ -1,105 +1,38 @@ { - "batch": [ - { - "id": "ea3d694a-ce6b-417e-86e3-23ac17c6f6c6", - "type": "trace-create", - "body": { - "id": "litellm-test-38dcf290-8742-4fc5-ad03-c5d47e91dec0", - "timestamp": "2025-01-22T18:06:50.959206Z", - "name": "litellm-acompletion", - "input": { - "messages": [ - { - "role": "user", - "content": "Hello!" - } - ] - }, - "output": { - "content": "Hello! How can I assist you today?", - "role": "assistant", - "tool_calls": null, - "function_call": null, - "provider_specific_fields": null - }, - "tags": [] - }, - "timestamp": "2025-01-22T18:06:50.959409Z" + "name": "litellm-acompletion", + "parent_span_id": null, + "attributes": { + "langfuse.observation.cost_details": { + "total": 3.5e-05 }, - { - "id": "5fe03133-5798-4f87-8eec-ae0264f1eccc", - "type": "generation-create", - "body": { - "traceId": "litellm-test-38dcf290-8742-4fc5-ad03-c5d47e91dec0", - "name": "litellm-acompletion", - "startTime": "2025-01-22T10:06:50.957097-08:00", - "metadata": { - "list": [ - "list", - "not", - "a", - "dict" - ], - "hidden_params": { - "model_id": null, - "cache_key": null, - "api_base": "https://api.openai.com", - "response_cost": 5.4999999999999995e-05, - "additional_headers": {}, - "litellm_overhead_time_ms": null, - "batch_models": null, - "litellm_model_name": "gpt-3.5-turbo", - "usage_object": null - }, - "litellm_response_cost": 5.4999999999999995e-05, - "cache_hit": false, - "requester_metadata": {} - }, - "input": { - "messages": [ - { - "role": "user", - "content": "Hello!" - } - ] - }, - "output": { - "content": "Hello! How can I assist you today?", - "role": "assistant", - "tool_calls": null, - "function_call": null, - "provider_specific_fields": null - }, - "level": "DEFAULT", - "id": "time-10-06-50-957097_chatcmpl-62d4ad7c-291b-4fc7-a8a4-3ed0fc3912a5", - "endTime": "2025-01-22T10:06:50.958374-08:00", - "completionStartTime": "2025-01-22T10:06:50.958374-08:00", - "model": "gpt-3.5-turbo", - "modelParameters": { - "extra_body": "{}" - }, - "usage": { - "input": 10, - "output": 20, - "unit": "TOKENS", - "totalCost": 3.5e-05 - }, - "usageDetails": { - "input": 10, - "output": 20, - "total": 30, - "cache_creation_input_tokens": 0, - "cache_read_input_tokens": 0 + "langfuse.observation.input": { + "messages": [ + { + "role": "user", + "content": "Hello!" } - }, - "timestamp": "2025-01-22T18:06:50.959850Z" - } - ], - "metadata": { - "batch_size": 2, - "sdk_integration": "litellm", - "sdk_name": "python", - "sdk_version": "2.44.1", - "public_key": "pk-lf-e02aaea3-8668-4c9f-8c69-771a4ea1f5c9" + ] + }, + "langfuse.observation.level": "DEFAULT", + "langfuse.observation.model.name": "gpt-3.5-turbo", + "langfuse.observation.model.parameters": { + "extra_body": "{}" + }, + "langfuse.observation.output": { + "content": "Hello! How can I assist you today?", + "role": "assistant", + "tool_calls": null, + "function_call": null, + "provider_specific_fields": null + }, + "langfuse.observation.type": "generation", + "langfuse.observation.usage_details": { + "input": 10, + "output": 20, + "total": 30, + "cache_creation_input_tokens": 0, + "cache_read_input_tokens": 0 + }, + "langfuse.trace.name": "litellm-acompletion" } -} \ No newline at end of file +} diff --git a/tests/logging_callback_tests/langfuse_expected_request_body/empty_metadata.json b/tests/logging_callback_tests/langfuse_expected_request_body/empty_metadata.json index f4040f1f8fc..906b1a42a8f 100644 --- a/tests/logging_callback_tests/langfuse_expected_request_body/empty_metadata.json +++ b/tests/logging_callback_tests/langfuse_expected_request_body/empty_metadata.json @@ -1,99 +1,38 @@ { - "batch": [ - { - "id": "28d0c943-284b-4151-bf0d-8acf0f449865", - "type": "trace-create", - "body": { - "id": "litellm-test-d9506624-457c-40bc-9a37-578b896fa22a", - "timestamp": "2025-01-22T17:59:32.888622Z", - "name": "litellm-acompletion", - "input": { - "messages": [ - { - "role": "user", - "content": "Hello!" - } - ] - }, - "output": { - "content": "Hello! How can I assist you today?", - "role": "assistant", - "tool_calls": null, - "function_call": null, - "provider_specific_fields": null - }, - "tags": [] - }, - "timestamp": "2025-01-22T17:59:32.888940Z" + "name": "litellm-acompletion", + "parent_span_id": null, + "attributes": { + "langfuse.observation.cost_details": { + "total": 3.5e-05 }, - { - "id": "384e9fb4-3516-47b2-a4ae-1666337ec4a7", - "type": "generation-create", - "body": { - "traceId": "litellm-test-d9506624-457c-40bc-9a37-578b896fa22a", - "name": "litellm-acompletion", - "startTime": "2025-01-22T09:59:32.878577-08:00", - "metadata": { - "hidden_params": { - "model_id": null, - "cache_key": null, - "api_base": "https://api.openai.com", - "response_cost": 5.4999999999999995e-05, - "additional_headers": {}, - "litellm_overhead_time_ms": null, - "batch_models": null, - "litellm_model_name": "gpt-3.5-turbo", - "usage_object": null - }, - "litellm_response_cost": 5.4999999999999995e-05, - "cache_hit": false, - "requester_metadata": {} - }, - "input": { - "messages": [ - { - "role": "user", - "content": "Hello!" - } - ] - }, - "output": { - "content": "Hello! How can I assist you today?", - "role": "assistant", - "tool_calls": null, - "function_call": null, - "provider_specific_fields": null - }, - "level": "DEFAULT", - "id": "time-09-59-32-878577_chatcmpl-1195f870-fd4d-4e38-8dc8-99dd3da5ab0b", - "endTime": "2025-01-22T09:59:32.880691-08:00", - "completionStartTime": "2025-01-22T09:59:32.880691-08:00", - "model": "gpt-3.5-turbo", - "modelParameters": { - "extra_body": "{}" - }, - "usage": { - "input": 10, - "output": 20, - "unit": "TOKENS", - "totalCost": 3.5e-05 - }, - "usageDetails": { - "input": 10, - "output": 20, - "total": 30, - "cache_creation_input_tokens": 0, - "cache_read_input_tokens": 0 + "langfuse.observation.input": { + "messages": [ + { + "role": "user", + "content": "Hello!" } - }, - "timestamp": "2025-01-22T17:59:32.889548Z" - } - ], - "metadata": { - "batch_size": 2, - "sdk_integration": "litellm", - "sdk_name": "python", - "sdk_version": "2.44.1", - "public_key": "pk-lf-e02aaea3-8668-4c9f-8c69-771a4ea1f5c9" + ] + }, + "langfuse.observation.level": "DEFAULT", + "langfuse.observation.model.name": "gpt-3.5-turbo", + "langfuse.observation.model.parameters": { + "extra_body": "{}" + }, + "langfuse.observation.output": { + "content": "Hello! How can I assist you today?", + "role": "assistant", + "tool_calls": null, + "function_call": null, + "provider_specific_fields": null + }, + "langfuse.observation.type": "generation", + "langfuse.observation.usage_details": { + "input": 10, + "output": 20, + "total": 30, + "cache_creation_input_tokens": 0, + "cache_read_input_tokens": 0 + }, + "langfuse.trace.name": "litellm-acompletion" } -} \ No newline at end of file +} diff --git a/tests/logging_callback_tests/langfuse_expected_request_body/metadata_with_function.json b/tests/logging_callback_tests/langfuse_expected_request_body/metadata_with_function.json index 77ca252c86d..906b1a42a8f 100644 --- a/tests/logging_callback_tests/langfuse_expected_request_body/metadata_with_function.json +++ b/tests/logging_callback_tests/langfuse_expected_request_body/metadata_with_function.json @@ -1,99 +1,38 @@ { - "batch": [ - { - "id": "88b1898a-cc5d-4e8e-93bc-3e71300c5e8d", - "type": "trace-create", - "body": { - "id": "litellm-test-a46356d9-ecff-44c8-a3da-fed3588b5128", - "timestamp": "2025-01-22T17:59:36.162545Z", - "name": "litellm-acompletion", - "input": { - "messages": [ - { - "role": "user", - "content": "Hello!" - } - ] - }, - "output": { - "content": "Hello! How can I assist you today?", - "role": "assistant", - "tool_calls": null, - "function_call": null, - "provider_specific_fields": null - }, - "tags": [] - }, - "timestamp": "2025-01-22T17:59:36.162702Z" + "name": "litellm-acompletion", + "parent_span_id": null, + "attributes": { + "langfuse.observation.cost_details": { + "total": 3.5e-05 }, - { - "id": "96bb77a6-a350-431b-bfd8-425491259728", - "type": "generation-create", - "body": { - "traceId": "litellm-test-a46356d9-ecff-44c8-a3da-fed3588b5128", - "name": "litellm-acompletion", - "startTime": "2025-01-22T09:59:36.161090-08:00", - "metadata": { - "hidden_params": { - "model_id": null, - "cache_key": null, - "api_base": "https://api.openai.com", - "response_cost": 5.4999999999999995e-05, - "additional_headers": {}, - "litellm_overhead_time_ms": null, - "batch_models": null, - "litellm_model_name": "gpt-3.5-turbo", - "usage_object": null - }, - "litellm_response_cost": 5.4999999999999995e-05, - "cache_hit": false, - "requester_metadata": {} - }, - "input": { - "messages": [ - { - "role": "user", - "content": "Hello!" - } - ] - }, - "output": { - "content": "Hello! How can I assist you today?", - "role": "assistant", - "tool_calls": null, - "function_call": null, - "provider_specific_fields": null - }, - "level": "DEFAULT", - "id": "time-09-59-36-161090_chatcmpl-1ee988c9-9133-4655-bbe4-b97ffb6e3dc9", - "endTime": "2025-01-22T09:59:36.161959-08:00", - "completionStartTime": "2025-01-22T09:59:36.161959-08:00", - "model": "gpt-3.5-turbo", - "modelParameters": { - "extra_body": "{}" - }, - "usage": { - "input": 10, - "output": 20, - "unit": "TOKENS", - "totalCost": 3.5e-05 - }, - "usageDetails": { - "input": 10, - "output": 20, - "total": 30, - "cache_creation_input_tokens": 0, - "cache_read_input_tokens": 0 + "langfuse.observation.input": { + "messages": [ + { + "role": "user", + "content": "Hello!" } - }, - "timestamp": "2025-01-22T17:59:36.162997Z" - } - ], - "metadata": { - "batch_size": 2, - "sdk_integration": "litellm", - "sdk_name": "python", - "sdk_version": "2.44.1", - "public_key": "pk-lf-e02aaea3-8668-4c9f-8c69-771a4ea1f5c9" + ] + }, + "langfuse.observation.level": "DEFAULT", + "langfuse.observation.model.name": "gpt-3.5-turbo", + "langfuse.observation.model.parameters": { + "extra_body": "{}" + }, + "langfuse.observation.output": { + "content": "Hello! How can I assist you today?", + "role": "assistant", + "tool_calls": null, + "function_call": null, + "provider_specific_fields": null + }, + "langfuse.observation.type": "generation", + "langfuse.observation.usage_details": { + "input": 10, + "output": 20, + "total": 30, + "cache_creation_input_tokens": 0, + "cache_read_input_tokens": 0 + }, + "langfuse.trace.name": "litellm-acompletion" } -} \ No newline at end of file +} diff --git a/tests/logging_callback_tests/langfuse_expected_request_body/metadata_with_lock.json b/tests/logging_callback_tests/langfuse_expected_request_body/metadata_with_lock.json index f4040f1f8fc..906b1a42a8f 100644 --- a/tests/logging_callback_tests/langfuse_expected_request_body/metadata_with_lock.json +++ b/tests/logging_callback_tests/langfuse_expected_request_body/metadata_with_lock.json @@ -1,99 +1,38 @@ { - "batch": [ - { - "id": "28d0c943-284b-4151-bf0d-8acf0f449865", - "type": "trace-create", - "body": { - "id": "litellm-test-d9506624-457c-40bc-9a37-578b896fa22a", - "timestamp": "2025-01-22T17:59:32.888622Z", - "name": "litellm-acompletion", - "input": { - "messages": [ - { - "role": "user", - "content": "Hello!" - } - ] - }, - "output": { - "content": "Hello! How can I assist you today?", - "role": "assistant", - "tool_calls": null, - "function_call": null, - "provider_specific_fields": null - }, - "tags": [] - }, - "timestamp": "2025-01-22T17:59:32.888940Z" + "name": "litellm-acompletion", + "parent_span_id": null, + "attributes": { + "langfuse.observation.cost_details": { + "total": 3.5e-05 }, - { - "id": "384e9fb4-3516-47b2-a4ae-1666337ec4a7", - "type": "generation-create", - "body": { - "traceId": "litellm-test-d9506624-457c-40bc-9a37-578b896fa22a", - "name": "litellm-acompletion", - "startTime": "2025-01-22T09:59:32.878577-08:00", - "metadata": { - "hidden_params": { - "model_id": null, - "cache_key": null, - "api_base": "https://api.openai.com", - "response_cost": 5.4999999999999995e-05, - "additional_headers": {}, - "litellm_overhead_time_ms": null, - "batch_models": null, - "litellm_model_name": "gpt-3.5-turbo", - "usage_object": null - }, - "litellm_response_cost": 5.4999999999999995e-05, - "cache_hit": false, - "requester_metadata": {} - }, - "input": { - "messages": [ - { - "role": "user", - "content": "Hello!" - } - ] - }, - "output": { - "content": "Hello! How can I assist you today?", - "role": "assistant", - "tool_calls": null, - "function_call": null, - "provider_specific_fields": null - }, - "level": "DEFAULT", - "id": "time-09-59-32-878577_chatcmpl-1195f870-fd4d-4e38-8dc8-99dd3da5ab0b", - "endTime": "2025-01-22T09:59:32.880691-08:00", - "completionStartTime": "2025-01-22T09:59:32.880691-08:00", - "model": "gpt-3.5-turbo", - "modelParameters": { - "extra_body": "{}" - }, - "usage": { - "input": 10, - "output": 20, - "unit": "TOKENS", - "totalCost": 3.5e-05 - }, - "usageDetails": { - "input": 10, - "output": 20, - "total": 30, - "cache_creation_input_tokens": 0, - "cache_read_input_tokens": 0 + "langfuse.observation.input": { + "messages": [ + { + "role": "user", + "content": "Hello!" } - }, - "timestamp": "2025-01-22T17:59:32.889548Z" - } - ], - "metadata": { - "batch_size": 2, - "sdk_integration": "litellm", - "sdk_name": "python", - "sdk_version": "2.44.1", - "public_key": "pk-lf-e02aaea3-8668-4c9f-8c69-771a4ea1f5c9" + ] + }, + "langfuse.observation.level": "DEFAULT", + "langfuse.observation.model.name": "gpt-3.5-turbo", + "langfuse.observation.model.parameters": { + "extra_body": "{}" + }, + "langfuse.observation.output": { + "content": "Hello! How can I assist you today?", + "role": "assistant", + "tool_calls": null, + "function_call": null, + "provider_specific_fields": null + }, + "langfuse.observation.type": "generation", + "langfuse.observation.usage_details": { + "input": 10, + "output": 20, + "total": 30, + "cache_creation_input_tokens": 0, + "cache_read_input_tokens": 0 + }, + "langfuse.trace.name": "litellm-acompletion" } -} \ No newline at end of file +} diff --git a/tests/logging_callback_tests/langfuse_expected_request_body/nested_metadata.json b/tests/logging_callback_tests/langfuse_expected_request_body/nested_metadata.json index f4a1bb9dcea..906b1a42a8f 100644 --- a/tests/logging_callback_tests/langfuse_expected_request_body/nested_metadata.json +++ b/tests/logging_callback_tests/langfuse_expected_request_body/nested_metadata.json @@ -1,105 +1,38 @@ { - "batch": [ - { - "id": "44f179be-e3b9-486f-986f-030fc50614f0", - "type": "trace-create", - "body": { - "id": "litellm-test-8a04085c-1859-48fa-9fd8-1ec487fe455e", - "timestamp": "2025-01-22T17:55:28.854927Z", - "name": "litellm-acompletion", - "input": { - "messages": [ - { - "role": "user", - "content": "Hello!" - } - ] - }, - "output": { - "content": "Hello! How can I assist you today?", - "role": "assistant", - "tool_calls": null, - "function_call": null, - "provider_specific_fields": null - }, - "tags": [] - }, - "timestamp": "2025-01-22T17:55:28.855187Z" + "name": "litellm-acompletion", + "parent_span_id": null, + "attributes": { + "langfuse.observation.cost_details": { + "total": 3.5e-05 }, - { - "id": "2175ee64-58a3-41ab-96df-405b76695f5f", - "type": "generation-create", - "body": { - "traceId": "litellm-test-8a04085c-1859-48fa-9fd8-1ec487fe455e", - "name": "litellm-acompletion", - "startTime": "2025-01-22T09:55:28.852503-08:00", - "metadata": { - "a": { - "nested_a": 1 - }, - "b": { - "nested_b": 2 - }, - "hidden_params": { - "model_id": null, - "cache_key": null, - "api_base": "https://api.openai.com", - "response_cost": 5.4999999999999995e-05, - "additional_headers": {}, - "litellm_overhead_time_ms": null, - "batch_models": null, - "litellm_model_name": "gpt-3.5-turbo", - "usage_object": null - }, - "litellm_response_cost": 5.4999999999999995e-05, - "cache_hit": false, - "requester_metadata": {} - }, - "input": { - "messages": [ - { - "role": "user", - "content": "Hello!" - } - ] - }, - "output": { - "content": "Hello! How can I assist you today?", - "role": "assistant", - "tool_calls": null, - "function_call": null, - "provider_specific_fields": null - }, - "level": "DEFAULT", - "id": "time-09-55-28-852503_chatcmpl-131cf0da-a47b-4cd1-850b-50fa077362ac", - "endTime": "2025-01-22T09:55:28.853979-08:00", - "completionStartTime": "2025-01-22T09:55:28.853979-08:00", - "model": "gpt-3.5-turbo", - "modelParameters": { - "extra_body": "{}" - }, - "usage": { - "input": 10, - "output": 20, - "unit": "TOKENS", - "totalCost": 3.5e-05 - }, - "usageDetails": { - "input": 10, - "output": 20, - "total": 30, - "cache_creation_input_tokens": 0, - "cache_read_input_tokens": 0 + "langfuse.observation.input": { + "messages": [ + { + "role": "user", + "content": "Hello!" } - }, - "timestamp": "2025-01-22T17:55:28.855732Z" - } - ], - "metadata": { - "batch_size": 2, - "sdk_integration": "litellm", - "sdk_name": "python", - "sdk_version": "2.44.1", - "public_key": "pk-lf-e02aaea3-8668-4c9f-8c69-771a4ea1f5c9" + ] + }, + "langfuse.observation.level": "DEFAULT", + "langfuse.observation.model.name": "gpt-3.5-turbo", + "langfuse.observation.model.parameters": { + "extra_body": "{}" + }, + "langfuse.observation.output": { + "content": "Hello! How can I assist you today?", + "role": "assistant", + "tool_calls": null, + "function_call": null, + "provider_specific_fields": null + }, + "langfuse.observation.type": "generation", + "langfuse.observation.usage_details": { + "input": 10, + "output": 20, + "total": 30, + "cache_creation_input_tokens": 0, + "cache_read_input_tokens": 0 + }, + "langfuse.trace.name": "litellm-acompletion" } -} \ No newline at end of file +} diff --git a/tests/logging_callback_tests/langfuse_expected_request_body/simple_metadata.json b/tests/logging_callback_tests/langfuse_expected_request_body/simple_metadata.json index d895378e2c6..906b1a42a8f 100644 --- a/tests/logging_callback_tests/langfuse_expected_request_body/simple_metadata.json +++ b/tests/logging_callback_tests/langfuse_expected_request_body/simple_metadata.json @@ -1,105 +1,38 @@ { - "batch": [ - { - "id": "02c74119-76b7-4f79-91cb-c55f1495c100", - "type": "trace-create", - "body": { - "id": "litellm-test-e58116c7-ead0-417e-9f86-b35f1e5bc242", - "timestamp": "2025-01-22T17:53:53.754012Z", - "name": "litellm-acompletion", - "input": { - "messages": [ - { - "role": "user", - "content": "Hello!" - } - ] - }, - "output": { - "content": "Hello! How can I assist you today?", - "role": "assistant", - "tool_calls": null, - "function_call": null, - "provider_specific_fields": null - }, - "tags": [] - }, - "timestamp": "2025-01-22T17:53:53.754178Z" + "name": "litellm-acompletion", + "parent_span_id": null, + "attributes": { + "langfuse.observation.cost_details": { + "total": 3.5e-05 }, - { - "id": "097968e0-52e9-46b5-9e8e-e6e08dd00e72", - "type": "generation-create", - "body": { - "traceId": "litellm-test-e58116c7-ead0-417e-9f86-b35f1e5bc242", - "name": "litellm-acompletion", - "startTime": "2025-01-22T09:53:53.752422-08:00", - "metadata": { - "a": { - "nested_a": 1 - }, - "b": { - "nested_b": 2 - }, - "hidden_params": { - "model_id": null, - "cache_key": null, - "api_base": "https://api.openai.com", - "response_cost": 5.4999999999999995e-05, - "additional_headers": {}, - "litellm_overhead_time_ms": null, - "batch_models": null, - "litellm_model_name": "gpt-3.5-turbo", - "usage_object": null - }, - "litellm_response_cost": 5.4999999999999995e-05, - "cache_hit": false, - "requester_metadata": {} - }, - "input": { - "messages": [ - { - "role": "user", - "content": "Hello!" - } - ] - }, - "output": { - "content": "Hello! How can I assist you today?", - "role": "assistant", - "tool_calls": null, - "function_call": null, - "provider_specific_fields": null - }, - "level": "DEFAULT", - "id": "time-09-53-53-752422_chatcmpl-e99bc1d3-a393-493f-8afe-4507c0acff15", - "endTime": "2025-01-22T09:53:53.753431-08:00", - "completionStartTime": "2025-01-22T09:53:53.753431-08:00", - "model": "gpt-3.5-turbo", - "modelParameters": { - "extra_body": "{}" - }, - "usage": { - "input": 10, - "output": 20, - "unit": "TOKENS", - "totalCost": 3.5e-05 - }, - "usageDetails": { - "input": 10, - "output": 20, - "total": 30, - "cache_creation_input_tokens": 0, - "cache_read_input_tokens": 0 + "langfuse.observation.input": { + "messages": [ + { + "role": "user", + "content": "Hello!" } - }, - "timestamp": "2025-01-22T17:53:53.754511Z" - } - ], - "metadata": { - "batch_size": 2, - "sdk_integration": "litellm", - "sdk_name": "python", - "sdk_version": "2.44.1", - "public_key": "pk-lf-e02aaea3-8668-4c9f-8c69-771a4ea1f5c9" + ] + }, + "langfuse.observation.level": "DEFAULT", + "langfuse.observation.model.name": "gpt-3.5-turbo", + "langfuse.observation.model.parameters": { + "extra_body": "{}" + }, + "langfuse.observation.output": { + "content": "Hello! How can I assist you today?", + "role": "assistant", + "tool_calls": null, + "function_call": null, + "provider_specific_fields": null + }, + "langfuse.observation.type": "generation", + "langfuse.observation.usage_details": { + "input": 10, + "output": 20, + "total": 30, + "cache_creation_input_tokens": 0, + "cache_read_input_tokens": 0 + }, + "langfuse.trace.name": "litellm-acompletion" } -} \ No newline at end of file +} diff --git a/tests/logging_callback_tests/langfuse_expected_request_body/simple_metadata2.json b/tests/logging_callback_tests/langfuse_expected_request_body/simple_metadata2.json index 87eba33cfff..906b1a42a8f 100644 --- a/tests/logging_callback_tests/langfuse_expected_request_body/simple_metadata2.json +++ b/tests/logging_callback_tests/langfuse_expected_request_body/simple_metadata2.json @@ -1,109 +1,38 @@ { - "batch": [ - { - "id": "1a55383a-e6fa-41f9-81fe-e7aa58c55f40", - "type": "trace-create", - "body": { - "id": "litellm-test-08fd1578-4a67-49b4-ac23-2dff1c112c80", - "timestamp": "2025-01-22T17:56:35.477276Z", - "name": "litellm-acompletion", - "input": { - "messages": [ - { - "role": "user", - "content": "Hello!" - } - ] - }, - "output": { - "content": "Hello! How can I assist you today?", - "role": "assistant", - "tool_calls": null, - "function_call": null, - "provider_specific_fields": null - }, - "tags": [] - }, - "timestamp": "2025-01-22T17:56:35.477571Z" + "name": "litellm-acompletion", + "parent_span_id": null, + "attributes": { + "langfuse.observation.cost_details": { + "total": 3.5e-05 }, - { - "id": "13ba66e8-f72b-4f57-a6cc-57c0be2829b1", - "type": "generation-create", - "body": { - "traceId": "litellm-test-08fd1578-4a67-49b4-ac23-2dff1c112c80", - "name": "litellm-acompletion", - "startTime": "2025-01-22T09:56:35.474752-08:00", - "metadata": { - "a": [ - 1, - 2, - 3 - ], - "b": [ - 4, - 5, - 6 - ], - "hidden_params": { - "model_id": null, - "cache_key": null, - "api_base": "https://api.openai.com", - "response_cost": 5.4999999999999995e-05, - "additional_headers": {}, - "litellm_overhead_time_ms": null, - "batch_models": null, - "litellm_model_name": "gpt-3.5-turbo", - "usage_object": null - }, - "litellm_response_cost": 5.4999999999999995e-05, - "cache_hit": false, - "requester_metadata": {} - }, - "input": { - "messages": [ - { - "role": "user", - "content": "Hello!" - } - ] - }, - "output": { - "content": "Hello! How can I assist you today?", - "role": "assistant", - "tool_calls": null, - "function_call": null, - "provider_specific_fields": null - }, - "level": "DEFAULT", - "id": "time-09-56-35-474752_chatcmpl-9b152610-3d1e-4731-a84e-d0341ea69a0f", - "endTime": "2025-01-22T09:56:35.476236-08:00", - "completionStartTime": "2025-01-22T09:56:35.476236-08:00", - "model": "gpt-3.5-turbo", - "modelParameters": { - "extra_body": "{}" - }, - "usage": { - "input": 10, - "output": 20, - "unit": "TOKENS", - "totalCost": 3.5e-05 - }, - "usageDetails": { - "input": 10, - "output": 20, - "total": 30, - "cache_creation_input_tokens": 0, - "cache_read_input_tokens": 0 + "langfuse.observation.input": { + "messages": [ + { + "role": "user", + "content": "Hello!" } - }, - "timestamp": "2025-01-22T17:56:35.478171Z" - } - ], - "metadata": { - "batch_size": 2, - "sdk_integration": "litellm", - "sdk_name": "python", - "sdk_version": "2.44.1", - "public_key": "pk-lf-e02aaea3-8668-4c9f-8c69-771a4ea1f5c9" + ] + }, + "langfuse.observation.level": "DEFAULT", + "langfuse.observation.model.name": "gpt-3.5-turbo", + "langfuse.observation.model.parameters": { + "extra_body": "{}" + }, + "langfuse.observation.output": { + "content": "Hello! How can I assist you today?", + "role": "assistant", + "tool_calls": null, + "function_call": null, + "provider_specific_fields": null + }, + "langfuse.observation.type": "generation", + "langfuse.observation.usage_details": { + "input": 10, + "output": 20, + "total": 30, + "cache_creation_input_tokens": 0, + "cache_read_input_tokens": 0 + }, + "langfuse.trace.name": "litellm-acompletion" } -} \ No newline at end of file +} diff --git a/tests/logging_callback_tests/langfuse_expected_request_body/simple_metadata3.json b/tests/logging_callback_tests/langfuse_expected_request_body/simple_metadata3.json index dd3bb4a301f..906b1a42a8f 100644 --- a/tests/logging_callback_tests/langfuse_expected_request_body/simple_metadata3.json +++ b/tests/logging_callback_tests/langfuse_expected_request_body/simple_metadata3.json @@ -1,113 +1,38 @@ { - "batch": [ - { - "id": "7fb1f295-a7af-47af-afbd-e2f2d08280aa", - "type": "trace-create", - "body": { - "id": "litellm-test-c3acc34b-3c06-4868-bcee-87a3c4c1367e", - "timestamp": "2025-01-22T17:56:38.786515Z", - "name": "litellm-acompletion", - "input": { - "messages": [ - { - "role": "user", - "content": "Hello!" - } - ] - }, - "output": { - "content": "Hello! How can I assist you today?", - "role": "assistant", - "tool_calls": null, - "function_call": null, - "provider_specific_fields": null - }, - "tags": [] - }, - "timestamp": "2025-01-22T17:56:38.786742Z" + "name": "litellm-acompletion", + "parent_span_id": null, + "attributes": { + "langfuse.observation.cost_details": { + "total": 3.5e-05 }, - { - "id": "412870bc-fc50-4426-a0dc-9e8b016e14bb", - "type": "generation-create", - "body": { - "traceId": "litellm-test-c3acc34b-3c06-4868-bcee-87a3c4c1367e", - "name": "litellm-acompletion", - "startTime": "2025-01-22T09:56:38.784548-08:00", - "metadata": { - "a": [ - 1, - 2 - ], - "b": [ - 3, - 4 - ], - "c": { - "d": [ - 5, - 6 - ] - }, - "hidden_params": { - "model_id": null, - "cache_key": null, - "api_base": "https://api.openai.com", - "response_cost": 5.4999999999999995e-05, - "additional_headers": {}, - "litellm_overhead_time_ms": null, - "batch_models": null, - "litellm_model_name": "gpt-3.5-turbo", - "usage_object": null - }, - "litellm_response_cost": 5.4999999999999995e-05, - "cache_hit": false, - "requester_metadata": {} - }, - "input": { - "messages": [ - { - "role": "user", - "content": "Hello!" - } - ] - }, - "output": { - "content": "Hello! How can I assist you today?", - "role": "assistant", - "tool_calls": null, - "function_call": null, - "provider_specific_fields": null - }, - "level": "DEFAULT", - "id": "time-09-56-38-784548_chatcmpl-438c8727-86b3-44d9-9b46-42330922cf50", - "endTime": "2025-01-22T09:56:38.785762-08:00", - "completionStartTime": "2025-01-22T09:56:38.785762-08:00", - "model": "gpt-3.5-turbo", - "modelParameters": { - "extra_body": "{}" - }, - "usage": { - "input": 10, - "output": 20, - "unit": "TOKENS", - "totalCost": 3.5e-05 - }, - "usageDetails": { - "input": 10, - "output": 20, - "total": 30, - "cache_creation_input_tokens": 0, - "cache_read_input_tokens": 0 + "langfuse.observation.input": { + "messages": [ + { + "role": "user", + "content": "Hello!" } - }, - "timestamp": "2025-01-22T17:56:38.787196Z" - } - ], - "metadata": { - "batch_size": 2, - "sdk_integration": "litellm", - "sdk_name": "python", - "sdk_version": "2.44.1", - "public_key": "pk-lf-e02aaea3-8668-4c9f-8c69-771a4ea1f5c9" + ] + }, + "langfuse.observation.level": "DEFAULT", + "langfuse.observation.model.name": "gpt-3.5-turbo", + "langfuse.observation.model.parameters": { + "extra_body": "{}" + }, + "langfuse.observation.output": { + "content": "Hello! How can I assist you today?", + "role": "assistant", + "tool_calls": null, + "function_call": null, + "provider_specific_fields": null + }, + "langfuse.observation.type": "generation", + "langfuse.observation.usage_details": { + "input": 10, + "output": 20, + "total": 30, + "cache_creation_input_tokens": 0, + "cache_read_input_tokens": 0 + }, + "langfuse.trace.name": "litellm-acompletion" } -} \ No newline at end of file +} diff --git a/tests/logging_callback_tests/test_langfuse_dynamic_credentials.py b/tests/logging_callback_tests/test_langfuse_dynamic_credentials.py index 2346a5ee047..92466a9470c 100644 --- a/tests/logging_callback_tests/test_langfuse_dynamic_credentials.py +++ b/tests/logging_callback_tests/test_langfuse_dynamic_credentials.py @@ -1,6 +1,3 @@ -import sys -from types import ModuleType, SimpleNamespace - import litellm from litellm.integrations.langfuse.langfuse import resolve_langfuse_credentials from litellm.integrations.langfuse.langfuse_handler import LangFuseHandler @@ -51,37 +48,29 @@ def test_resolve_langfuse_credentials_keeps_env_for_global_config(monkeypatch): assert host == "https://admin-configured.example" -def test_upstream_langfuse_debug_env_is_passed(monkeypatch): +def test_upstream_langfuse_env_only_warns_and_opens_no_second_channel(monkeypatch, caplog): + """UPSTREAM_LANGFUSE_* configured a second v2 ingestion client. v4 has one export channel per + credential set, so the values are ignored with a startup warning and never build anything.""" + from litellm.integrations.langfuse import langfuse_sdk from litellm.integrations.langfuse.langfuse import LangFuseLogger - class FakeLangfuse: - instances = [] - - def __init__(self, **kwargs): - self.kwargs = kwargs - FakeLangfuse.instances.append(self) - - fake_langfuse_module = ModuleType("langfuse") - fake_langfuse_module.Langfuse = FakeLangfuse - fake_langfuse_module.version = SimpleNamespace(__version__="2.6.0") - - monkeypatch.setitem(sys.modules, "langfuse", fake_langfuse_module) monkeypatch.setattr(litellm, "initialized_langfuse_clients", 0) + monkeypatch.setattr(langfuse_sdk, "_TRACING", {}) monkeypatch.setenv("LANGFUSE_MOCK", "true") monkeypatch.setenv("UPSTREAM_LANGFUSE_SECRET_KEY", "upstream-secret") monkeypatch.setenv("UPSTREAM_LANGFUSE_PUBLIC_KEY", "upstream-public") monkeypatch.setenv("UPSTREAM_LANGFUSE_HOST", "https://upstream.example") - monkeypatch.setenv("UPSTREAM_LANGFUSE_RELEASE", "release") - monkeypatch.setenv("UPSTREAM_LANGFUSE_DEBUG", "true") - logger = LangFuseLogger( - langfuse_public_key="public", - langfuse_secret="secret", - langfuse_host="https://langfuse.example", - ) + with caplog.at_level("WARNING", logger="LiteLLM"): + logger = LangFuseLogger( + langfuse_public_key="public", + langfuse_secret="secret", + langfuse_host="https://langfuse.example", + ) - assert logger.upstream_langfuse_debug == "true" - assert FakeLangfuse.instances[-1].kwargs["debug"] is True + assert any("UPSTREAM_LANGFUSE_* is no longer supported" in record.getMessage() for record in caplog.records) + assert [lease.tracing for lease in langfuse_sdk._TRACING.values()] == [logger.tracing] + assert all(key.public_key == "public" for key in langfuse_sdk._TRACING) def test_langfuse_handler_accepts_secret_key_alias(monkeypatch): diff --git a/tests/logging_callback_tests/test_langfuse_e2e_test.py b/tests/logging_callback_tests/test_langfuse_e2e_test.py index 76ebd2b9a28..61a93a175d9 100644 --- a/tests/logging_callback_tests/test_langfuse_e2e_test.py +++ b/tests/logging_callback_tests/test_langfuse_e2e_test.py @@ -1,166 +1,140 @@ import asyncio -import copy import json import logging import os import threading -from typing import Any, Optional +from collections.abc import Mapping +from typing import Final from unittest.mock import AsyncMock, MagicMock, patch import httpx +from opentelemetry.proto.collector.trace.v1.trace_service_pb2 import ExportTraceServiceRequest +from opentelemetry.proto.common.v1.common_pb2 import AnyValue logging.basicConfig(level=logging.DEBUG) import litellm -from litellm import completion -from litellm.caching import InMemoryCache +from litellm.integrations.langfuse.langfuse_sdk import resolve_observation_id, resolve_trace_id from litellm.llms.custom_httpx.http_handler import AsyncHTTPHandler litellm.num_retries = 3 litellm.success_callback = ["langfuse"] os.environ["LANGFUSE_DEBUG"] = "True" -import time import pytest import pytest_asyncio +LANGFUSE_EXPORT_POST: Final = "litellm.llms.custom_httpx.http_handler.HTTPHandler.post" +LANGFUSE_EXPORT_PATH: Final = "/api/public/otel/v1/traces" + +_PER_RUN_ATTRIBUTES: Final = frozenset( + { + "langfuse.observation.completion_start_time", + "langfuse.observation.metadata.applied_guardrails", + "langfuse.observation.metadata.cache_hit", + "langfuse.observation.metadata.hidden_params", + "langfuse.observation.metadata.litellm_call_id", + "langfuse.observation.metadata.litellm_response_cost", + "langfuse.observation.metadata.requester_metadata", + "langfuse.observation.metadata.response_id", + "langfuse.observation.metadata.usage_object", + } +) + + +def _decode_attribute(value: AnyValue) -> object: + match value.WhichOneof("value"): + case "string_value": + try: + return json.loads(value.string_value) + except json.JSONDecodeError: + return value.string_value + case "bool_value": + return value.bool_value + case "int_value": + return value.int_value + case "double_value": + return value.double_value + case "array_value": + return [_decode_attribute(item) for item in value.array_value.values] + case _: + return None + + +def _exported_spans(mock_post: MagicMock) -> list[dict[str, object]]: + spans: list[dict[str, object]] = [] + for call in mock_post.call_args_list: + url: str = call.args[0] if call.args else call.kwargs["url"] + assert url.endswith(LANGFUSE_EXPORT_PATH), url + request = ExportTraceServiceRequest.FromString(call.kwargs["data"]) + for resource_spans in request.resource_spans: + for scope_spans in resource_spans.scope_spans: + for span in scope_spans.spans: + spans.append( + { + "name": span.name, + "trace_id": span.trace_id.hex(), + "span_id": span.span_id.hex(), + "parent_span_id": span.parent_span_id.hex() or None, + "attributes": { + attribute.key: _decode_attribute(attribute.value) for attribute in span.attributes + }, + } + ) + return spans + + +def _comparable(span: Mapping[str, object]) -> dict[str, object]: + attributes = span["attributes"] + assert isinstance(attributes, dict) + return { + "name": span["name"], + "parent_span_id": span["parent_span_id"], + "attributes": {key: value for key, value in sorted(attributes.items()) if key not in _PER_RUN_ATTRIBUTES}, + } + def assert_langfuse_request_matches_expected( - actual_request_body: dict, + spans: list[dict[str, object]], expected_file_name: str, - trace_id: Optional[str] = None, + trace_id: str, ): - """ - Helper function to compare actual Langfuse request body with expected JSON file. - - Args: - actual_request_body (dict): The actual request body received from the API call - expected_file_name (str): Name of the JSON file containing expected request body (e.g., "transcription.json") - """ - # Get the current directory and read the expected request body + """Compare the generation langfuse exported for ``trace_id`` with the expected JSON file.""" pwd = os.path.dirname(os.path.realpath(__file__)) - expected_body_path = os.path.join( - pwd, "langfuse_expected_request_body", expected_file_name - ) - + expected_body_path = os.path.join(pwd, "langfuse_expected_request_body", expected_file_name) with open(expected_body_path, "r") as f: - expected_request_body = json.load(f) + expected_generation = json.load(f) - # Filter out events that don't match the trace_id - if trace_id: - actual_request_body["batch"] = [ - item - for item in actual_request_body["batch"] - if (item["type"] == "trace-create" and item["body"].get("id") == trace_id) - or ( - item["type"] == "generation-create" - and item["body"].get("traceId") == trace_id - ) - ] - - # When aggregating from multiple flush cycles, deduplicate by keeping - # only one trace-create and one generation-create per trace_id. - seen_types: dict = {} - deduped_batch: list = [] - for item in actual_request_body["batch"]: - item_type = item["type"] - if item_type not in seen_types: - seen_types[item_type] = True - deduped_batch.append(item) - actual_request_body["batch"] = deduped_batch - - # Ensure canonical order: trace-create first, generation-create second - actual_request_body["batch"].sort( - key=lambda x: 0 if x["type"] == "trace-create" else 1 + otel_trace_id: Final = resolve_trace_id(trace_id) + generations: Final = [ + span + for span in spans + if span["trace_id"] == otel_trace_id and span["attributes"]["langfuse.observation.type"] == "generation" # pyright: ignore[reportIndexIssue] # built as dict in _exported_spans + ] + assert len(generations) == 1, ( + f"Expected exactly one generation for trace_id={trace_id} ({otel_trace_id}), " + f"got {len(generations)}. Spans: {json.dumps(spans, indent=2)}" ) - print( - "actual_request_body after filtering", json.dumps(actual_request_body, indent=4) + actual_generation: Final = _comparable(generations[0]) + assert actual_generation == expected_generation, ( + f"Difference in exported generation: {json.dumps(actual_generation, indent=2)} " + f"!= {json.dumps(expected_generation, indent=2)}" ) - assert len(actual_request_body["batch"]) >= 2, ( - f"Expected at least 2 batch items (trace-create + generation-create) " - f"after filtering by trace_id={trace_id}, " - f"but got {len(actual_request_body['batch'])}. " - f"Items: {json.dumps(actual_request_body['batch'], indent=2)}" - ) - - # Replace dynamic values in actual request body - for item in actual_request_body["batch"]: - - # Replace IDs with expected IDs - if item["type"] == "trace-create": - item["id"] = expected_request_body["batch"][0]["id"] - item["body"]["id"] = expected_request_body["batch"][0]["body"]["id"] - item["timestamp"] = expected_request_body["batch"][0]["timestamp"] - item["body"]["timestamp"] = expected_request_body["batch"][0]["body"][ - "timestamp" - ] - elif item["type"] == "generation-create": - item["id"] = expected_request_body["batch"][1]["id"] - item["body"]["id"] = expected_request_body["batch"][1]["body"]["id"] - item["timestamp"] = expected_request_body["batch"][1]["timestamp"] - item["body"]["startTime"] = expected_request_body["batch"][1]["body"][ - "startTime" - ] - item["body"]["endTime"] = expected_request_body["batch"][1]["body"][ - "endTime" - ] - item["body"]["completionStartTime"] = expected_request_body["batch"][1][ - "body" - ]["completionStartTime"] - if trace_id is None: - print("popping traceId") - item["body"].pop("traceId") - else: - item["body"]["traceId"] = trace_id - expected_request_body["batch"][1]["body"]["traceId"] = trace_id - - # Replace SDK version with expected version - actual_request_body["batch"][0]["body"].pop("release", None) - actual_request_body["metadata"]["sdk_version"] = expected_request_body["metadata"][ - "sdk_version" - ] - # replace "public_key" with expected public key - actual_request_body["metadata"]["public_key"] = expected_request_body["metadata"][ - "public_key" - ] - actual_request_body["batch"][1]["body"]["metadata"] = expected_request_body[ - "batch" - ][1]["body"]["metadata"] - actual_request_body["metadata"]["sdk_integration"] = expected_request_body[ - "metadata" - ]["sdk_integration"] - actual_request_body["metadata"]["batch_size"] = expected_request_body["metadata"][ - "batch_size" - ] - # Assert the entire request body matches - assert ( - actual_request_body == expected_request_body - ), f"Difference in request bodies: {json.dumps(actual_request_body, indent=2)} != {json.dumps(expected_request_body, indent=2)}" - class TestLangfuseLogging: @pytest_asyncio.fixture async def mock_setup(self): """Common setup for Langfuse logging tests""" from litellm._uuid import uuid - from unittest.mock import AsyncMock, patch - import httpx - # Create a mock Response object - mock_response = AsyncMock(spec=httpx.Response) - mock_response.status_code = 200 - mock_response.json.return_value = {"status": "success"} - - # Create mock for httpx.Client.post - mock_post = AsyncMock() - mock_post.return_value = mock_response + mock_post = MagicMock(return_value=MagicMock(ok=True, status_code=200)) litellm.set_verbose = True litellm.success_callback = ["langfuse"] - return {"trace_id": f"litellm-test-{str(uuid.uuid4())}", "mock_post": mock_post} + return {"trace_id": f"litellm-test-{uuid.uuid4()!s}", "mock_post": mock_post} async def _verify_langfuse_call( self, @@ -168,41 +142,16 @@ class TestLangfuseLogging: expected_file_name: str, trace_id: str, ): - """Helper method to verify Langfuse API calls""" - await asyncio.sleep(3) - - # Verify at least one call was made - assert mock_post.call_count >= 1 - - # Aggregate batch items from ALL calls — the Langfuse SDK may split - # trace-create and generation-create across separate HTTP flushes. - langfuse_url = "https://us.cloud.langfuse.com/api/public/ingestion" - all_batch_items: list = [] - metadata: Optional[dict] = None - for call in mock_post.call_args_list: - url = call[0][0] - if url != langfuse_url: - continue - request_body = call[1].get("content") - if request_body: - body = json.loads(request_body) - all_batch_items.extend(body.get("batch", [])) - if metadata is None: - metadata = body.get("metadata") - - assert len(all_batch_items) > 0, "No Langfuse ingestion calls found" - assert metadata is not None, "No metadata found in Langfuse calls" - - actual_request_body = { - "batch": all_batch_items, - "metadata": metadata, - } - - print("\nMocked Request Details (aggregated from all calls):") - print(f"Request Body: {json.dumps(actual_request_body, indent=4)}") + """Wait for the batch processor to export, then compare the generation it shipped.""" + otel_trace_id: Final = resolve_trace_id(trace_id) + for _ in range(100): + if any(span["trace_id"] == otel_trace_id for span in _exported_spans(mock_post)): + break + await asyncio.sleep(0.1) + assert mock_post.call_count >= 1, "langfuse exported nothing" assert_langfuse_request_matches_expected( - actual_request_body, + _exported_spans(mock_post), expected_file_name, trace_id, ) @@ -212,23 +161,21 @@ class TestLangfuseLogging: async def test_langfuse_logging_completion(self, mock_setup): """Test Langfuse logging for chat completion""" setup = mock_setup - with patch("httpx.Client.post", setup["mock_post"]): + with patch(LANGFUSE_EXPORT_POST, setup["mock_post"]): await litellm.acompletion( model="gpt-3.5-turbo", messages=[{"role": "user", "content": "Hello!"}], mock_response="Hello! How can I assist you today?", metadata={"trace_id": setup["trace_id"]}, ) - await self._verify_langfuse_call( - setup["mock_post"], "completion.json", setup["trace_id"] - ) + await self._verify_langfuse_call(setup["mock_post"], "completion.json", setup["trace_id"]) @pytest.mark.asyncio @pytest.mark.flaky(retries=3, delay=1) async def test_langfuse_logging_completion_with_tags(self, mock_setup): """Test Langfuse logging for chat completion with tags""" setup = mock_setup - with patch("httpx.Client.post", setup["mock_post"]): + with patch(LANGFUSE_EXPORT_POST, setup["mock_post"]): await litellm.acompletion( model="gpt-3.5-turbo", messages=[{"role": "user", "content": "Hello!"}], @@ -238,16 +185,14 @@ class TestLangfuseLogging: "tags": ["test_tag", "test_tag_2"], }, ) - await self._verify_langfuse_call( - setup["mock_post"], "completion_with_tags.json", setup["trace_id"] - ) + await self._verify_langfuse_call(setup["mock_post"], "completion_with_tags.json", setup["trace_id"]) @pytest.mark.asyncio @pytest.mark.flaky(retries=3, delay=1) async def test_langfuse_logging_completion_with_tags_stream(self, mock_setup): """Test Langfuse logging for chat completion with tags""" setup = mock_setup - with patch("httpx.Client.post", setup["mock_post"]): + with patch(LANGFUSE_EXPORT_POST, setup["mock_post"]): await litellm.acompletion( model="gpt-3.5-turbo", messages=[{"role": "user", "content": "Hello!"}], @@ -263,12 +208,33 @@ class TestLangfuseLogging: setup["trace_id"], ) + @pytest.mark.asyncio + @pytest.mark.flaky(retries=3, delay=1) + async def test_langfuse_generation_id_metadata_names_the_exported_observation(self, mock_setup): + """v2 let callers pick the generation id; v4 only has span ids, so the requested id must become one.""" + setup = mock_setup + with patch(LANGFUSE_EXPORT_POST, setup["mock_post"]): + await litellm.acompletion( + model="gpt-3.5-turbo", + messages=[{"role": "user", "content": "Hello!"}], + mock_response="Hello! How can I assist you today?", + metadata={"trace_id": setup["trace_id"], "generation_id": "my-generation"}, + ) + await self._verify_langfuse_call(setup["mock_post"], "completion.json", setup["trace_id"]) + + generation: Final = next( + span + for span in _exported_spans(setup["mock_post"]) + if span["trace_id"] == resolve_trace_id(setup["trace_id"]) + ) + assert generation["span_id"] == resolve_observation_id("my-generation") + @pytest.mark.asyncio @pytest.mark.flaky(retries=3, delay=1) async def test_langfuse_logging_completion_with_langfuse_metadata(self, mock_setup): """Test Langfuse logging for chat completion with metadata for langfuse""" setup = mock_setup - with patch("httpx.Client.post", setup["mock_post"]): + with patch(LANGFUSE_EXPORT_POST, setup["mock_post"]): await litellm.acompletion( model="gpt-3.5-turbo", messages=[{"role": "user", "content": "Hello!"}], @@ -297,12 +263,12 @@ class TestLangfuseLogging: @pytest.mark.flaky(retries=3, delay=1) async def test_langfuse_logging_with_non_serializable_metadata(self, mock_setup): """Test Langfuse logging with metadata that requires preparation (Pydantic models, sets, etc)""" - from pydantic import BaseModel - from typing import Set import datetime + from pydantic import BaseModel + class UserPreferences(BaseModel): - favorite_colors: Set[str] + favorite_colors: set[str] last_login: datetime.datetime settings: dict @@ -325,8 +291,8 @@ class TestLangfuseLogging: "trace_id": setup["trace_id"], } - with patch("httpx.Client.post", setup["mock_post"]): - response = await litellm.acompletion( + with patch(LANGFUSE_EXPORT_POST, setup["mock_post"]): + await litellm.acompletion( model="gpt-3.5-turbo", messages=[{"role": "user", "content": "Hello!"}], mock_response="Hello! How can I assist you today?", @@ -375,18 +341,14 @@ class TestLangfuseLogging: ], ) @pytest.mark.flaky(retries=6, delay=1) - async def test_langfuse_logging_with_various_metadata_types( - self, mock_setup, test_metadata, response_json_file - ): + async def test_langfuse_logging_with_various_metadata_types(self, mock_setup, test_metadata, response_json_file): """Test Langfuse logging with various metadata types including non-serializable objects""" - import threading - setup = mock_setup if test_metadata is not None: test_metadata["trace_id"] = setup["trace_id"] - with patch("httpx.Client.post", setup["mock_post"]): + with patch(LANGFUSE_EXPORT_POST, setup["mock_post"]): await litellm.acompletion( model="gpt-3.5-turbo", messages=[{"role": "user", "content": "Hello!"}], @@ -402,13 +364,11 @@ class TestLangfuseLogging: @pytest.mark.asyncio @pytest.mark.flaky(retries=3, delay=1) - async def test_langfuse_logging_completion_with_malformed_llm_response( - self, mock_setup - ): + async def test_langfuse_logging_completion_with_malformed_llm_response(self, mock_setup): """Test Langfuse logging for chat completion with malformed LLM response""" setup = mock_setup litellm._turn_on_debug() - with patch("httpx.Client.post", setup["mock_post"]): + with patch(LANGFUSE_EXPORT_POST, setup["mock_post"]): mock_response = litellm.ModelResponse( choices=[], usage=litellm.Usage( @@ -426,19 +386,15 @@ class TestLangfuseLogging: mock_response=mock_response, metadata={"trace_id": setup["trace_id"]}, ) - await self._verify_langfuse_call( - setup["mock_post"], "completion_with_no_choices.json", setup["trace_id"] - ) + await self._verify_langfuse_call(setup["mock_post"], "completion_with_no_choices.json", setup["trace_id"]) @pytest.mark.asyncio @pytest.mark.flaky(retries=3, delay=1) - async def test_langfuse_logging_completion_with_bedrock_llm_response( - self, mock_setup - ): + async def test_langfuse_logging_completion_with_bedrock_llm_response(self, mock_setup): """Test Langfuse logging for chat completion with malformed LLM response""" setup = mock_setup litellm._turn_on_debug() - with patch("httpx.Client.post", setup["mock_post"]): + with patch(LANGFUSE_EXPORT_POST, setup["mock_post"]): mock_response = litellm.ModelResponse( choices=[], usage=litellm.Usage( @@ -467,13 +423,11 @@ class TestLangfuseLogging: @pytest.mark.asyncio @pytest.mark.flaky(retries=3, delay=1) - async def test_langfuse_logging_completion_with_vertex_llm_response( - self, mock_setup - ): + async def test_langfuse_logging_completion_with_vertex_llm_response(self, mock_setup): """Test Langfuse logging for chat completion with malformed LLM response""" setup = mock_setup litellm._turn_on_debug() - with patch("httpx.Client.post", setup["mock_post"]): + with patch(LANGFUSE_EXPORT_POST, setup["mock_post"]): mock_response = litellm.ModelResponse( choices=[], usage=litellm.Usage( @@ -525,7 +479,7 @@ class TestLangfuseLogging: mock_async_client = AsyncHTTPHandler() mock_async_client.post = AsyncMock(return_value=mock_vllm_response) - with patch("httpx.Client.post", setup["mock_post"]): + with patch(LANGFUSE_EXPORT_POST, setup["mock_post"]): await litellm.aembedding( model="hosted_vllm/BAAI/bge-small-en-v1.5", input=["Hello from litellm!"], @@ -539,9 +493,7 @@ class TestLangfuseLogging: actual_vllm_request = mock_async_client.post.call_args.kwargs["json"] pwd = os.path.dirname(os.path.realpath(__file__)) - expected_body_path = os.path.join( - pwd, "langfuse_expected_request_body", "embedding_with_vllm.json" - ) + expected_body_path = os.path.join(pwd, "langfuse_expected_request_body", "embedding_with_vllm.json") with open(expected_body_path, "r") as f: expected_vllm_request = json.load(f) @@ -568,7 +520,7 @@ class TestLangfuseLogging: } ] ) - with patch("httpx.Client.post", mock_setup["mock_post"]): + with patch(LANGFUSE_EXPORT_POST, mock_setup["mock_post"]): mock_response = litellm.ModelResponse( choices=[], usage=litellm.Usage( diff --git a/tests/logging_callback_tests/test_langfuse_unit_tests.py b/tests/logging_callback_tests/test_langfuse_unit_tests.py index 405b6e9e48e..61316204fc3 100644 --- a/tests/logging_callback_tests/test_langfuse_unit_tests.py +++ b/tests/logging_callback_tests/test_langfuse_unit_tests.py @@ -306,35 +306,63 @@ def test_get_langfuse_flush_interval(): def test_langfuse_e2e_sync(monkeypatch): - from litellm import completion - import litellm - import respx - import httpx + """A sync completion must reach langfuse over the wire, not just build a span. + + v4 exports OTLP over ``requests`` rather than the v2 ingestion endpoint over + httpx, so this stands up a real receiver and asserts langfuse posted to it. + """ + import threading import time + from http.server import BaseHTTPRequestHandler, HTTPServer - litellm.disable_aiohttp_transport = ( - True # since this uses respx, we need to set use_aiohttp_transport to False - ) + import litellm + from litellm import completion + from litellm.integrations.langfuse.langfuse import LangFuseLogger + from litellm.integrations.langfuse.langfuse_prompt_management import langfuse_client_init + from litellm.litellm_core_utils import litellm_logging - litellm._turn_on_debug() + received_paths = [] + + class _Receiver(BaseHTTPRequestHandler): + def do_POST(self): + received_paths.append(self.path) + self.rfile.read(int(self.headers.get("Content-Length") or 0)) + self.send_response(200) + self.send_header("Content-Length", "0") + self.end_headers() + + def log_message(self, *args): + pass + + server = HTTPServer(("127.0.0.1", 0), _Receiver) + threading.Thread(target=server.serve_forever, daemon=True).start() + monkeypatch.setenv("LANGFUSE_HOST", f"http://127.0.0.1:{server.server_port}") + monkeypatch.setenv("LANGFUSE_PUBLIC_KEY", "pk-e2e-sync") + monkeypatch.setenv("LANGFUSE_SECRET_KEY", "sk-e2e-sync") monkeypatch.setattr(litellm, "success_callback", ["langfuse"]) + monkeypatch.setattr(litellm_logging, "langFuseLogger", None) + monkeypatch.setattr(litellm_logging, "in_memory_dynamic_logger_cache", DynamicLoggingCache()) + monkeypatch.setattr(litellm_logging, "_in_memory_loggers", []) + langfuse_client_init.cache_clear() - with respx.mock: - # Mock Langfuse - # Mock any Langfuse endpoint - langfuse_mock = respx.post( - "https://*.cloud.langfuse.com/api/public/ingestion" - ).mock(return_value=httpx.Response(200)) + try: completion( model="openai/my-fake-endpoint", messages=[{"role": "user", "content": "hello from litellm"}], stream=False, mock_response="Hello from litellm 2", ) + for logger in litellm.logging_callback_manager._get_all_callbacks(): + if isinstance(logger, LangFuseLogger): + logger.flush() + deadline = time.time() + 10 + while not received_paths and time.time() < deadline: + time.sleep(0.1) + finally: + server.shutdown() - time.sleep(3) - - assert langfuse_mock.called + assert received_paths, "langfuse exported nothing" + assert all(path.endswith("/api/public/otel/v1/traces") for path in received_paths) def test_get_chat_content_for_langfuse(): diff --git a/tests/test_litellm/integrations/SlackAlerting/test_slack_alerting_utils.py b/tests/test_litellm/integrations/SlackAlerting/test_slack_alerting_utils.py index 403cd51701d..b02dbea64b8 100644 --- a/tests/test_litellm/integrations/SlackAlerting/test_slack_alerting_utils.py +++ b/tests/test_litellm/integrations/SlackAlerting/test_slack_alerting_utils.py @@ -1,15 +1,9 @@ -import json -from typing import Optional -from unittest.mock import MagicMock +from unittest.mock import AsyncMock, MagicMock import pytest # Adds the grandparent directory to sys.path to allow importing project modules - import litellm -from litellm.integrations.langfuse.langfuse_prompt_management import ( - LangfusePromptManagement, -) from litellm.integrations.SlackAlerting.utils import _add_langfuse_trace_id_to_alert from litellm.litellm_core_utils.logging_callback_manager import LoggingCallbackManager @@ -34,3 +28,95 @@ async def test_langfuse_not_initialized_returns_none_early(): # Verify the litellm_logging_obj was never accessed (early return) request_data["litellm_logging_obj"].assert_not_called() + + +@pytest.mark.asyncio +async def test_langfuse_trace_url_uses_the_request_host_without_building_a_logger(monkeypatch): + """Key-scoped callbacks point at their own Langfuse host; the alert link follows it. + + The lookup must not construct a LangFuseLogger per alert, or an alert storm + exhausts the initialized-client ceiling and takes the callback down with it. + """ + monkeypatch.setattr(litellm, "success_callback", ["langfuse"]) + monkeypatch.setattr(litellm, "initialized_langfuse_clients", 0) + logging_obj = MagicMock() + logging_obj._get_trace_id.return_value = "abc123" + logging_obj.standard_callback_dynamic_params = {"langfuse_host": "http://127.0.0.1:1"} + + result = await _add_langfuse_trace_id_to_alert({"litellm_logging_obj": logging_obj}) + + assert result == "http://127.0.0.1:1/trace/abc123" + assert litellm.initialized_langfuse_clients == 0 + + +@pytest.mark.asyncio +async def test_langfuse_trace_url_falls_back_to_the_env_host(monkeypatch): + monkeypatch.setattr(litellm, "success_callback", ["langfuse"]) + monkeypatch.setenv("LANGFUSE_HOST", "langfuse.internal:3000") + logging_obj = MagicMock() + logging_obj._get_trace_id.return_value = "abc123" + logging_obj.standard_callback_dynamic_params = {} + + assert await _add_langfuse_trace_id_to_alert({"litellm_logging_obj": logging_obj}) == ( + "http://langfuse.internal:3000/trace/abc123" + ) + + +@pytest.mark.asyncio +async def test_langfuse_trace_url_when_callback_registered_as_logger_instance(monkeypatch): + from litellm.integrations.langfuse.langfuse import LangFuseLogger + + logger = LangFuseLogger( + langfuse_public_key="pk-slack-instance", + langfuse_secret="sk-slack-instance", + langfuse_host="http://127.0.0.1:1", + ) + monkeypatch.setattr(litellm, "success_callback", [logger]) + monkeypatch.setattr(litellm, "failure_callback", []) + monkeypatch.setattr(litellm, "_async_success_callback", []) + monkeypatch.setattr(litellm, "_async_failure_callback", []) + monkeypatch.setattr(litellm, "callbacks", []) + monkeypatch.setenv("LANGFUSE_HOST", "http://env-host.invalid") + logging_obj = MagicMock() + logging_obj._get_trace_id.return_value = "trace-from-instance" + logging_obj.standard_callback_dynamic_params = {} + + result = await _add_langfuse_trace_id_to_alert({"litellm_logging_obj": logging_obj}) + + assert result == "http://127.0.0.1:1/trace/trace-from-instance" + + +@pytest.mark.asyncio +async def test_langfuse_trace_url_when_prompt_management_is_the_registered_callback(monkeypatch): + """Prompt management registers a LangFuseLogger subclass; the alert must read its host, not crash.""" + from litellm.integrations.langfuse.langfuse_prompt_management import LangfusePromptManagement + + prompt_callback = LangfusePromptManagement( + langfuse_public_key="pk-slack-prompt", + langfuse_secret="sk-slack-prompt", + langfuse_host="http://127.0.0.1:2", + ) + monkeypatch.setattr(litellm, "success_callback", ["langfuse"]) + monkeypatch.setattr(litellm, "failure_callback", []) + monkeypatch.setattr(litellm, "_async_success_callback", []) + monkeypatch.setattr(litellm, "_async_failure_callback", []) + monkeypatch.setattr(litellm, "callbacks", [prompt_callback]) + monkeypatch.setenv("LANGFUSE_HOST", "http://env-host.invalid") + logging_obj = MagicMock() + logging_obj._get_trace_id.return_value = "trace-from-prompt-callback" + logging_obj.standard_callback_dynamic_params = {} + + result = await _add_langfuse_trace_id_to_alert({"litellm_logging_obj": logging_obj}) + + assert result == "http://127.0.0.1:2/trace/trace-from-prompt-callback" + + +@pytest.mark.asyncio +async def test_langfuse_trace_url_absent_when_trace_id_never_arrives(monkeypatch): + monkeypatch.setattr(litellm, "success_callback", ["langfuse"]) + monkeypatch.setattr("litellm.integrations.SlackAlerting.utils.asyncio.sleep", AsyncMock()) + logging_obj = MagicMock() + logging_obj._get_trace_id.return_value = None + logging_obj.standard_callback_dynamic_params = {"langfuse_host": "http://127.0.0.1:1"} + + assert await _add_langfuse_trace_id_to_alert({"litellm_logging_obj": logging_obj}) is None diff --git a/tests/test_litellm/integrations/langfuse/test_langfuse_prompt_management.py b/tests/test_litellm/integrations/langfuse/test_langfuse_prompt_management.py index 7dea4e67cdd..a7a553b2d9f 100644 --- a/tests/test_litellm/integrations/langfuse/test_langfuse_prompt_management.py +++ b/tests/test_litellm/integrations/langfuse/test_langfuse_prompt_management.py @@ -1,9 +1,13 @@ -from types import MappingProxyType +import sys +from datetime import datetime, timezone from typing import Final from unittest.mock import MagicMock, patch import pytest +# langfuse_client_init imports this lazily; cache it before any test mocks +# sys.modules["langfuse"], or a single-file run dies on the real import +import litellm.integrations.langfuse.langfuse_sdk # noqa: F401 from litellm.integrations.langfuse.langfuse_prompt_management import ( LangfusePromptManagement, langfuse_client_init, @@ -17,9 +21,7 @@ class TestLangfusePromptManagement: # This also prevents test-ordering issues when earlier tests remove sys.modules["langfuse"]. self._mock_langfuse = MagicMock() self._mock_langfuse.version.__version__ = "3.0.0" - self._langfuse_patcher = patch.dict( - "sys.modules", {"langfuse": self._mock_langfuse} - ) + self._langfuse_patcher = patch.dict("sys.modules", {"langfuse": self._mock_langfuse}) self._langfuse_patcher.start() def teardown_method(self): @@ -31,9 +33,7 @@ class TestLangfusePromptManagement: patch.object( langfuse_prompt_management, "should_run_prompt_management" ) as mock_should_run_prompt_management, - patch.object( - langfuse_prompt_management, "_get_prompt_from_id" - ) as mock_get_prompt_from_id, + patch.object(langfuse_prompt_management, "_get_prompt_from_id") as mock_get_prompt_from_id, ): mock_should_run_prompt_management.return_value = True langfuse_prompt_management.get_chat_completion_prompt( @@ -51,9 +51,7 @@ class TestLangfusePromptManagement: def test_log_failure_event_runs_async_logger(self): langfuse_prompt_management = LangfusePromptManagement() - with patch( - "litellm.integrations.langfuse.langfuse_prompt_management.run_async_function" - ) as mock_run_async: + with patch("litellm.integrations.langfuse.langfuse_prompt_management.run_async_function") as mock_run_async: kwargs = {"standard_callback_dynamic_params": {}} start_time, end_time = 1, 2 @@ -65,10 +63,7 @@ class TestLangfusePromptManagement: ) mock_run_async.assert_called_once() - assert ( - mock_run_async.call_args[0][0] - == langfuse_prompt_management.async_log_failure_event - ) + assert mock_run_async.call_args[0][0] == langfuse_prompt_management.async_log_failure_event def test_langfuse_client_init_passes_dedicated_httpx_client(self): import httpx @@ -76,35 +71,28 @@ class TestLangfusePromptManagement: from litellm.llms.custom_httpx.http_handler import _get_httpx_client shared_client = _get_httpx_client().client - - mock_langfuse_class = MagicMock() + built = MagicMock() with ( patch( "litellm.integrations.langfuse.langfuse_prompt_management.resolve_langfuse_credentials", return_value=("pk-1234", "sk-1234", "https://localhost"), ), patch( - "litellm.integrations.langfuse.langfuse_prompt_management.LangFuseLogger._get_langfuse_flush_interval", - return_value=1, - ), - patch.dict("sys.modules", {"langfuse": self._mock_langfuse}), + "litellm.integrations.langfuse.langfuse_sdk.build_langfuse_client", built + ), # test-quality-ok: the REST client is built where langfuse_client_init resolves it; the transport it gets is the behavior under test patch( "litellm.llms.custom_httpx.http_handler.get_ssl_configuration", return_value=False, ) as mock_get_ssl, ): - self._mock_langfuse.Langfuse = mock_langfuse_class - langfuse_client_init( langfuse_public_key="pk-1234", langfuse_secret="sk-1234", langfuse_host="https://localhost", ) - mock_langfuse_class.assert_called_once() - call_kwargs = mock_langfuse_class.call_args[1] - assert "httpx_client" in call_kwargs - passed_client = call_kwargs["httpx_client"] + built.assert_called_once() + passed_client = built.call_args.kwargs["httpx_client"] assert isinstance(passed_client, httpx.Client) assert passed_client is not shared_client mock_get_ssl.assert_called_once() @@ -112,28 +100,181 @@ class TestLangfusePromptManagement: langfuse_client_init.cache_clear() -class _RecordingLangfuseForEnv: - last_environment: str | None = None - - def __init__(self, *, environment: str | None = None, **parameters: object) -> None: # kwargs-ok: records only environment out of whatever langfuse_client_init forwards - type(self).last_environment = environment - - @pytest.mark.parametrize( ("env_value", "expected"), (("Production", "default"), ("production ", "production"), ("prod", "prod")), ) -def test_langfuse_client_init_resolves_deployment_environment(monkeypatch, env_value, expected): - mock_langfuse_module: Final = MagicMock() - mock_langfuse_module.version.__version__ = "2.60.0" - mock_langfuse_module.Langfuse = _RecordingLangfuseForEnv +def test_prompt_management_logger_exports_the_resolved_deployment_environment(monkeypatch, env_value, expected): + from langfuse import LangfuseOtelSpanAttributes + + monkeypatch.setenv("LANGFUSE_PUBLIC_KEY", "pk-test") + monkeypatch.setenv("LANGFUSE_SECRET_KEY", "sk-test") + monkeypatch.setenv("LANGFUSE_HOST", "http://127.0.0.1:1") + monkeypatch.setenv("LANGFUSE_MOCK", "true") + monkeypatch.setenv("LANGFUSE_TRACING_ENVIRONMENT", env_value) + langfuse_client_init.cache_clear() + logger = LangfusePromptManagement() + langfuse_client_init.cache_clear() + assert logger.tracing.provider.resource.attributes[LangfuseOtelSpanAttributes.ENVIRONMENT] == expected + + +def test_langfuse_client_init_warns_that_upstream_langfuse_is_ignored(monkeypatch, caplog): + """The YAML `callbacks: ["langfuse"]` path builds its client here, not through LangFuseLogger.__init__, + so an operator who still sets UPSTREAM_LANGFUSE_* must get the same startup warning on this path.""" monkeypatch.setenv("LANGFUSE_PUBLIC_KEY", "pk-test") monkeypatch.setenv("LANGFUSE_SECRET_KEY", "sk-test") monkeypatch.setenv("LANGFUSE_HOST", "https://test.langfuse.com") - monkeypatch.setenv("LANGFUSE_TRACING_ENVIRONMENT", env_value) - monkeypatch.setattr(_RecordingLangfuseForEnv, "last_environment", None) - with patch.dict("sys.modules", MappingProxyType({"langfuse": mock_langfuse_module})): + monkeypatch.setenv("UPSTREAM_LANGFUSE_SECRET_KEY", "sk-upstream") + monkeypatch.setenv("UPSTREAM_LANGFUSE_HOST", "https://upstream.example") + with caplog.at_level("WARNING", logger="LiteLLM"): langfuse_client_init.cache_clear() langfuse_client_init() langfuse_client_init.cache_clear() - assert _RecordingLangfuseForEnv.last_environment == expected + assert any("UPSTREAM_LANGFUSE_* is no longer supported" in record.getMessage() for record in caplog.records) + + +def test_langfuse_client_init_mock_mode_makes_no_network_calls(monkeypatch): + """LANGFUSE_MOCK promises full execution without egress. + + The registry maps the "langfuse" callback to LangfusePromptManagement, so + this logger is the one the standard proxy path emits observations through; + they travel over litellm's own OTLP exporter, which the httpx mock cannot see. + """ + import threading + from http.server import BaseHTTPRequestHandler, HTTPServer + + import litellm + + received = [] + + class _Receiver(BaseHTTPRequestHandler): + def do_POST(self): + received.append(self.path) + self.rfile.read(int(self.headers.get("Content-Length") or 0)) + self.send_response(200) + self.send_header("Content-Length", "0") + self.end_headers() + + def log_message(self, *args): + pass + + server = HTTPServer(("127.0.0.1", 0), _Receiver) + threading.Thread(target=server.serve_forever, daemon=True).start() + monkeypatch.setenv("LANGFUSE_MOCK", "true") + monkeypatch.setenv("LANGFUSE_HOST", f"http://127.0.0.1:{server.server_port}") + monkeypatch.setenv("LANGFUSE_PUBLIC_KEY", "pk-pm-mock-egress") + monkeypatch.setenv("LANGFUSE_SECRET_KEY", "sk-pm-mock-egress") + langfuse_client_init.cache_clear() + now: Final = datetime.now(timezone.utc) + + try: + logger = LangfusePromptManagement() + logged = logger.log_event_on_langfuse( + kwargs={ + "litellm_call_id": "call-pm-mock-egress", + "call_type": "completion", + "litellm_params": {"metadata": {"trace_id": "a" * 32}}, + "messages": [{"role": "user", "content": "hi"}], + "optional_params": {}, + }, + response_obj=litellm.ModelResponse(choices=[{"message": {"role": "assistant", "content": "ok"}}]), + start_time=now, + end_time=now, + ) + logger.flush() + finally: + server.shutdown() + langfuse_client_init.cache_clear() + + assert logged["trace_id"] == "a" * 32 + assert received == [], f"LANGFUSE_MOCK still sent spans to the configured host: {received}" + + +def test_langfuse_debug_reaches_the_export_channel_through_the_registered_callback(monkeypatch): + """The registry maps ``langfuse`` to this class, whose constructor never runs ``LangFuseLogger.__init__``, + so wiring ``LANGFUSE_DEBUG`` only there left the flag a no-op on the YAML callback path.""" + import logging + + from litellm.integrations.langfuse.langfuse_sdk import release_langfuse_tracing + + monkeypatch.setenv("LANGFUSE_MOCK", "true") + monkeypatch.setenv("LANGFUSE_HOST", "http://127.0.0.1:1") + monkeypatch.setenv("LANGFUSE_PUBLIC_KEY", "pk-pm-debug-wire") + monkeypatch.setenv("LANGFUSE_SECRET_KEY", "sk-pm-debug-wire") + monkeypatch.setenv("LANGFUSE_DEBUG", "true") + langfuse_client_init.cache_clear() + langfuse_logger: Final = logging.getLogger("langfuse") + level_before: Final = langfuse_logger.level + langfuse_logger.setLevel(logging.WARNING) + try: + logger = LangfusePromptManagement() + assert langfuse_logger.level == logging.DEBUG + release_langfuse_tracing(logger.tracing, grace_seconds=0.0) + finally: + langfuse_logger.setLevel(level_before) + langfuse_client_init.cache_clear() + + +@pytest.mark.asyncio +async def test_async_log_failure_event_records_trace_id_for_alerting(monkeypatch): + from litellm.integrations.langfuse.langfuse_sdk import resolve_trace_id + from litellm.litellm_core_utils.specialty_caches.service_trace_id_cache import in_memory_trace_id_cache + + monkeypatch.setenv("LANGFUSE_MOCK", "true") + monkeypatch.setenv("LANGFUSE_HOST", "http://127.0.0.1:1") + monkeypatch.setenv("LANGFUSE_PUBLIC_KEY", "pk-pm-trace-cache") + monkeypatch.setenv("LANGFUSE_SECRET_KEY", "sk-pm-trace-cache") + langfuse_client_init.cache_clear() + call_id: Final = "call-trace-cache-1" + now: Final = datetime.now(timezone.utc) + kwargs: Final = { + "litellm_call_id": call_id, + "model": "gpt-5.4", + "messages": [{"role": "user", "content": "hi"}], + "litellm_params": {"metadata": {"trace_id": "alert-trace-1"}}, + "optional_params": {}, + "standard_callback_dynamic_params": {}, + "exception": RuntimeError("provider down"), + } + + try: + await LangfusePromptManagement().async_log_failure_event( + kwargs=kwargs, response_obj=None, start_time=now, end_time=now + ) + finally: + langfuse_client_init.cache_clear() + + assert in_memory_trace_id_cache.get_cache(litellm_call_id=call_id, service_name="langfuse") == resolve_trace_id( + "alert-trace-1" + ) + + +def test_old_sdk_fails_with_the_upgrade_message_before_the_otel_module_is_imported(monkeypatch): + """On a v2 install `langfuse_sdk` itself fails to import, so the version gate must run first.""" + import litellm.integrations.langfuse.langfuse_prompt_management as pm_module + + monkeypatch.setattr(pm_module, "installed_langfuse_version", lambda: "2.59.7") + monkeypatch.setitem(sys.modules, "litellm.integrations.langfuse.langfuse_sdk", None) + + with pytest.raises(ImportError) as raised: + LangfusePromptManagement( + langfuse_public_key="pk-old", langfuse_secret="sk-old", langfuse_host="http://127.0.0.1:1" + ) + + assert "2.59.7" in str(raised.value) + assert "langfuse_otel" in str(raised.value) + + +@pytest.mark.parametrize("raw", ["abc", "2.5"], ids=["text", "fraction"]) +def test_prompt_cache_ttl_typo_is_named_instead_of_reported_as_not_installed(monkeypatch, raw): + """The v4 SDK runs ``int()`` on this variable at import, and ``langfuse_client_init`` wraps any import + failure as "Langfuse not installed", so the gate has to run before that import.""" + monkeypatch.setenv("LANGFUSE_PROMPT_CACHE_DEFAULT_TTL_SECONDS", raw) + monkeypatch.setitem(sys.modules, "litellm.integrations.langfuse.langfuse_sdk", None) + langfuse_client_init.cache_clear() + + with pytest.raises(ValueError, match="LANGFUSE_PROMPT_CACHE_DEFAULT_TTL_SECONDS") as raised: + langfuse_client_init(langfuse_public_key="pk-ttl", langfuse_secret="sk-ttl", langfuse_host="http://127.0.0.1:1") + + assert "not installed" not in str(raised.value) + assert repr(raw) in str(raised.value) diff --git a/tests/test_litellm/integrations/langfuse/test_langfuse_sdk.py b/tests/test_litellm/integrations/langfuse/test_langfuse_sdk.py new file mode 100644 index 00000000000..c15a12c07cb --- /dev/null +++ b/tests/test_litellm/integrations/langfuse/test_langfuse_sdk.py @@ -0,0 +1,1759 @@ +"""Covers litellm's own Langfuse export channel: plain OTel spans carrying the v2 contracts. + +The timestamp assertions are the regression guard for the migration: the v4 SDK's public +API has no observation start time, so a callback running after the model call would +otherwise record its own duration instead of the call's. +""" + +import json +import logging +import threading +import uuid +from base64 import b64encode +from datetime import datetime, timedelta, timezone +from time import monotonic, sleep +from types import MappingProxyType +from typing import Final + +import httpx +import opentelemetry.trace as otel_trace +import pytest +from langfuse import LangfuseOtelSpanAttributes as A +from langfuse.api.core.api_error import ApiError +from langfuse.api.core.request_options import RequestOptions +from opentelemetry.proto.collector.trace.v1.trace_service_pb2 import ExportTraceServiceRequest +from opentelemetry.sdk.trace import SpanProcessor, TracerProvider +from opentelemetry.sdk.trace.export import SpanExporter, SpanExportResult +from opentelemetry.sdk.trace.export.in_memory_span_exporter import InMemorySpanExporter + +from litellm.integrations.langfuse.langfuse import ( + MINIMUM_LANGFUSE_VERSION, + installed_langfuse_version, + raise_if_unsupported_langfuse_version, +) +from litellm.integrations.langfuse.langfuse_sdk import ( + DiscardingSpanExporter, + LangfuseApiClient, + LangfusePromptError, + LangfuseSpanExporter, + LangfuseTracing, + _build_span_exporter, + _encode, + acquire_langfuse_tracing, + build_langfuse_client, + build_langfuse_tracing, + configured_flush_at, + configured_prompt_cache_ttl, + configured_sample_rate, + enable_langfuse_debug_logging, + flush_langfuse_tracing, + observation_attributes, + release_langfuse_tracing, + resolve_observation_id, + resolve_trace_id, + start_child_span, + start_generation, + to_unix_nanos, + trace_attributes, +) +from litellm.llms.custom_httpx.http_handler import HTTPHandler + +CALL_START = datetime(2024, 3, 1, 12, 0, 0, tzinfo=timezone.utc) +FIRST_TOKEN = CALL_START + timedelta(seconds=5) +CALL_END = CALL_START + timedelta(seconds=20) +TRACE_A = "a" * 32 +PARENT_C = "c" * 16 + + +@pytest.fixture(autouse=True) +def _own_channel_registry(monkeypatch: pytest.MonkeyPatch) -> None: + """Channels leaked by other test modules would otherwise take part in every process-wide flush here.""" + monkeypatch.setattr("litellm.integrations.langfuse.langfuse_sdk._TRACING", {}) + + +@pytest.fixture(name="channel") +def _channel() -> tuple[LangfuseTracing, InMemorySpanExporter]: + exporter = InMemorySpanExporter() + return ( + build_langfuse_tracing( + exporter=exporter, environment=None, release=None, sample_rate=1.0, flush_interval_millis=10 + ), + exporter, + ) + + +def _only_span(exporter, name): + return next(s for s in exporter.get_finished_spans() if s.name == name) + + +def _generation( + tracing, + *, + name="gen", + trace_id=TRACE_A, + parent=None, + existing=False, + observation_id=None, + public=None, + attributes=None, +): + return start_generation( + tracing=tracing, + trace_id=trace_id, + parent_observation_id=parent, + existing_trace=existing, + observation_id=observation_id, + name=name, + start_time=CALL_START, + public=public, + attributes=attributes if attributes is not None else {}, + ) + + +def test_generation_records_the_model_call_window_not_the_callback(channel): + tracing, exporter = channel + attributes = observation_attributes(observation_type="generation", completion_start_time=FIRST_TOKEN) + _generation(tracing, attributes=attributes).end(CALL_END) + tracing.flush() + + span = _only_span(exporter, "gen") + assert span.start_time == to_unix_nanos(CALL_START) + assert span.end_time == to_unix_nanos(CALL_END) + assert (span.end_time - span.start_time) == 20 * 1_000_000_000 + assert datetime.fromisoformat(json.loads(span.attributes[A.OBSERVATION_COMPLETION_START_TIME])) == FIRST_TOKEN + + +@pytest.mark.parametrize( + "supplied", + [1709294400.5, datetime(2024, 3, 1, 12, 0, 0, 500000, tzinfo=timezone.utc)], + ids=["unix-seconds-float", "datetime"], +) +def test_timestamps_accept_both_shapes_guardrails_and_callbacks_use(supplied): + """Guardrail entries carry unix seconds as floats, the callback carries datetimes.""" + assert to_unix_nanos(supplied) == 1709294400500000000 + + +def test_guardrail_span_with_float_timestamps_keeps_its_own_window_under_the_generation(channel): + tracing, exporter = channel + guardrail_start = 1709294400.0 + generation = _generation(tracing) + start_child_span( + tracing=tracing, parent=generation, name="guardrail", start_time=guardrail_start, attributes={} + ).end(guardrail_start + 2) + generation.end(CALL_END) + tracing.flush() + + guardrail = _only_span(exporter, "guardrail") + exported_generation = _only_span(exporter, "gen") + assert (guardrail.end_time - guardrail.start_time) == 2 * 1_000_000_000 + assert guardrail.context.trace_id == exported_generation.context.trace_id + assert guardrail.parent.span_id == exported_generation.context.span_id + + +def test_requested_trace_id_is_the_exported_trace_id_and_the_generation_is_its_root(channel): + """v2 ``trace(id=...)``: the caller's id is the trace and the generation has no parent.""" + tracing, exporter = channel + generation = _generation(tracing) + generation.end(CALL_END) + tracing.flush() + + span = _only_span(exporter, "gen") + assert generation.trace_id == TRACE_A + assert format(span.context.trace_id, "032x") == TRACE_A + assert span.parent is None + + +def test_parent_observation_id_nests_the_generation_under_the_callers_observation(channel): + tracing, exporter = channel + _generation(tracing, name="child-gen", parent=PARENT_C).end(CALL_END) + tracing.flush() + + span = _only_span(exporter, "child-gen") + assert format(span.context.trace_id, "032x") == TRACE_A + assert format(span.parent.span_id, "016x") == PARENT_C + assert span.parent.is_remote + + +def test_existing_trace_is_appended_to_rather_than_rewritten(channel): + """v2 ``existing_trace_id``: the generation joins the trace without becoming its root.""" + tracing, exporter = channel + _generation(tracing, name="continued", existing=True).end(CALL_END) + tracing.flush() + + span = _only_span(exporter, "continued") + assert format(span.context.trace_id, "032x") == TRACE_A + assert span.parent is not None + assert span.parent.span_id != 0 + + +def test_generation_does_not_hang_under_the_callers_active_span(channel): + """The caller's own OTel span must stay untouched and must not become the generation's parent.""" + tracing, exporter = channel + app_tracer = TracerProvider().get_tracer("app") + with app_tracer.start_as_current_span("app-span") as app_span: + _generation(tracing, trace_id="b" * 32).end(CALL_END) + attributes_after = dict(app_span.attributes or {}) + tracing.flush() + + span = _only_span(exporter, "gen") + assert span.parent is None + assert format(span.context.trace_id, "032x") == "b" * 32 + assert attributes_after == {} + + +def test_requested_observation_id_becomes_the_exported_span_id(channel): + """v2 ``generation(id=...)``: the caller's id is what the export carries and what ``.id`` returns.""" + tracing, exporter = channel + requested = resolve_observation_id("chatcmpl-123") + + generation = _generation(tracing, observation_id=requested) + generation.end(CALL_END) + tracing.flush() + + assert generation.id == requested + assert format(_only_span(exporter, "gen").context.span_id, "016x") == requested + + +def test_requested_ids_do_not_leak_into_the_next_span(channel): + tracing, exporter = channel + requested = resolve_observation_id("chatcmpl-123") + + _generation(tracing, name="first", observation_id=requested).end(CALL_END) + second = _generation(tracing, name="second", trace_id=resolve_trace_id(None)) + second.end(CALL_END) + child = start_child_span(tracing=tracing, parent=second, name="child", start_time=CALL_END, attributes={}) + child.end(CALL_END) + tracing.flush() + + assert second.id != requested + assert child.id not in (requested, second.id) + assert len({span.context.span_id for span in exporter.get_finished_spans()}) == 3 + + +@pytest.mark.parametrize("public", [True, False], ids=["public", "private"]) +def test_trace_public_flag_lands_on_the_root_observation(channel, public): + tracing, exporter = channel + _generation(tracing, public=public, attributes=trace_attributes(public=public)).end(CALL_END) + tracing.flush() + assert _only_span(exporter, "gen").attributes[A.TRACE_PUBLIC] is public + + +def test_trace_public_flag_is_absent_when_not_requested(channel): + tracing, exporter = channel + _generation(tracing, attributes=trace_attributes(public=None)).end(CALL_END) + tracing.flush() + assert A.TRACE_PUBLIC not in _only_span(exporter, "gen").attributes + + +@pytest.mark.parametrize("public", [True, False, None], ids=["public", "private", "unset"]) +def test_child_span_repeats_the_generation_public_flag(channel, public): + """The server folds ``public`` across observations and reads a missing value as False. + + A guardrail span without the flag turned a ``trace_public: true`` request private on Langfuse Cloud. + """ + tracing, exporter = channel + generation = _generation(tracing, public=public) + start_child_span(tracing=tracing, parent=generation, name="guardrail", start_time=CALL_END, attributes={}).end() + generation.end(CALL_END) + tracing.flush() + + assert _only_span(exporter, "guardrail").attributes.get(A.TRACE_PUBLIC) is public + + +def test_trace_attributes_carry_the_v2_trace_fields(): + attributes = trace_attributes( + name="trace-name", + user_id="user-1", + session_id="session-1", + version="v2", + release="rel-1", + tags=("a", "b"), + metadata={"tenant": "t1", "nested": {"k": 1}}, + input={"messages": []}, + output="answer", + ) + assert attributes[A.TRACE_NAME] == "trace-name" + assert attributes[A.TRACE_USER_ID] == "user-1" + assert attributes[A.TRACE_SESSION_ID] == "session-1" + assert attributes[A.VERSION] == "v2" + assert attributes[A.RELEASE] == "rel-1" + assert attributes[A.TRACE_TAGS] == ("a", "b") + assert attributes[f"{A.TRACE_METADATA}.tenant"] == "t1" + assert json.loads(attributes[f"{A.TRACE_METADATA}.nested"]) == {"k": 1} + assert json.loads(attributes[A.TRACE_INPUT]) == {"messages": []} + assert attributes[A.TRACE_OUTPUT] == "answer" + + +def test_trace_attributes_skip_what_the_request_did_not_supply(): + assert dict(trace_attributes()) == {} + + +def test_non_mapping_metadata_is_carried_whole_instead_of_raising(): + """A truthy non-dict ``trace_metadata`` used to blow up the callback on ``**`` unpacking.""" + attributes = trace_attributes(metadata=("not", "a", "dict")) + assert json.loads(attributes[A.TRACE_METADATA]) == ["not", "a", "dict"] + + +def test_observation_attributes_serialize_the_generation_fields(): + attributes = observation_attributes( + observation_type="generation", + input=[{"role": "user", "content": "hi"}], + output={"role": "assistant", "content": "hello"}, + metadata=MappingProxyType({"litellm_call_id": "call-1", "cache_hit": False}), + level="ERROR", + status_message="boom", + model="gpt-4o", + model_parameters={"temperature": 0.1}, + usage_details={"input": 1, "output": 2}, + cost_details={"total": 0.01}, + prompt="not-a-prompt-client", + ) + assert attributes[A.OBSERVATION_TYPE] == "generation" + assert attributes[A.OBSERVATION_LEVEL] == "ERROR" + assert attributes[A.OBSERVATION_STATUS_MESSAGE] == "boom" + assert attributes[A.OBSERVATION_MODEL] == "gpt-4o" + assert json.loads(attributes[A.OBSERVATION_INPUT]) == [{"role": "user", "content": "hi"}] + assert json.loads(attributes[A.OBSERVATION_OUTPUT]) == {"role": "assistant", "content": "hello"} + assert json.loads(attributes[A.OBSERVATION_MODEL_PARAMETERS]) == {"temperature": 0.1} + assert json.loads(attributes[A.OBSERVATION_USAGE_DETAILS]) == {"input": 1, "output": 2} + assert json.loads(attributes[A.OBSERVATION_COST_DETAILS]) == {"total": 0.01} + assert attributes[f"{A.OBSERVATION_METADATA}.litellm_call_id"] == "call-1" + assert attributes[f"{A.OBSERVATION_METADATA}.cache_hit"] is False + assert A.OBSERVATION_PROMPT_NAME not in attributes + + +@pytest.mark.parametrize( + "supplied, expected", + [ + ("0123456789abcdef0123456789abcdef", "0123456789abcdef0123456789abcdef"), + ("0123456789ABCDEF0123456789ABCDEF", "0123456789abcdef0123456789abcdef"), + ("3fe0c940-b69a-de3b-a77c-06102505349a", "3fe0c940b69ade3ba77c06102505349a"), + ], + ids=["already-hex", "uppercase-hex", "uuid-with-dashes"], +) +def test_trace_id_passes_through_when_it_is_already_usable(supplied, expected): + assert resolve_trace_id(supplied) == expected + + +def test_arbitrary_trace_id_is_hashed_deterministically(): + first = resolve_trace_id("order-4471") + assert first == resolve_trace_id("order-4471") + assert len(first) == 32 and first == first.lower() + assert first != resolve_trace_id("order-4472") + + +@pytest.mark.parametrize("supplied", [12345, 12.5, True, False], ids=["int", "float", "true", "false"]) +def test_non_string_trace_id_is_normalized(supplied): + resolved = resolve_trace_id(supplied) + + assert len(resolved) == 32 + assert resolved == resolve_trace_id(supplied) + + +@pytest.mark.parametrize("supplied", [12345, 12.5, True, False], ids=["int", "float", "true", "false"]) +def test_non_string_observation_id_is_normalized(supplied): + resolved = resolve_observation_id(supplied) + + assert len(resolved) == 16 + assert resolved == resolve_observation_id(supplied) + + +def test_all_zero_ids_are_hashed_instead_of_passed_through(): + zero_trace = "0" * 32 + zero_span = "0" * 16 + + assert resolve_trace_id(zero_trace) != zero_trace + assert resolve_trace_id(zero_trace) == resolve_trace_id(zero_trace) + assert int(resolve_trace_id(zero_trace), 16) != 0 + assert resolve_observation_id(zero_span) != zero_span + assert int(resolve_observation_id(zero_span), 16) != 0 + + +def test_hyphen_only_trace_ids_are_deterministic(): + assert resolve_trace_id("---") == resolve_trace_id("---") + + +def test_trace_id_with_trailing_newline_is_hashed(): + supplied = "a" * 32 + "\n" + + resolved = resolve_trace_id(supplied) + + assert resolved != supplied + assert len(resolved) == 32 + + +def test_missing_trace_id_still_yields_a_valid_trace_id(): + generated = resolve_trace_id(None) + assert len(generated) == 32 + assert int(generated, 16) >= 0 + + +@pytest.mark.parametrize( + "supplied, expected", + [ + ("0123456789abcdef", "0123456789abcdef"), + (None, None), + ("", None), + ], + ids=["already-hex", "none", "empty"], +) +def test_observation_id_normalisation(supplied, expected): + assert resolve_observation_id(supplied) == expected + + +def test_arbitrary_observation_id_is_hashed_to_a_span_id(): + resolved = resolve_observation_id("my-parent-observation") + assert len(resolved) == 16 + assert resolved == resolve_observation_id("my-parent-observation") + + +@pytest.mark.parametrize("unsupported", ["2.59.7", "3.15.0", "5.0.0"], ids=["v2", "v3", "v5"]) +def test_unsupported_sdk_fails_loudly_rather_than_dropping_every_event(unsupported): + with pytest.raises(ImportError) as raised: + raise_if_unsupported_langfuse_version(unsupported) + assert unsupported in str(raised.value) + assert MINIMUM_LANGFUSE_VERSION in str(raised.value) + + +def test_supported_sdk_is_accepted(): + assert raise_if_unsupported_langfuse_version(installed_langfuse_version()) is None + + +def test_channel_carries_environment_and_release_on_the_resource(): + tracing = build_langfuse_tracing( + exporter=DiscardingSpanExporter(), + environment="staging", + release="v9", + sample_rate=1.0, + flush_interval_millis=10, + ) + attributes = tracing.provider.resource.attributes + assert attributes[A.ENVIRONMENT] == "staging" + assert attributes[A.RELEASE] == "v9" + + +def _generations_exported_at(sample_rate: float, trace_ids: tuple[str, ...]) -> frozenset[str]: + exporter: Final = InMemorySpanExporter() + tracing: Final = build_langfuse_tracing( + exporter=exporter, environment=None, release=None, sample_rate=sample_rate, flush_interval_millis=10 + ) + for trace_id in trace_ids: + _generation(tracing, name="sampled", trace_id=trace_id).end(CALL_END) + tracing.flush() + return frozenset(format(span.context.trace_id, "032x") for span in exporter.get_finished_spans()) + + +def test_sample_rate_zero_drops_and_one_keeps_every_trace(): + trace_ids: Final = tuple(resolve_trace_id(uuid.uuid4()) for _ in range(20)) + assert _generations_exported_at(0, trace_ids) == frozenset() + assert _generations_exported_at(1, trace_ids) == frozenset(trace_ids) + + +def test_fractional_sample_rate_keeps_a_deterministic_share_of_uuid_trace_ids(): + trace_ids: Final = tuple(resolve_trace_id(uuid.uuid4()) for _ in range(400)) + kept: Final = _generations_exported_at(0.5, trace_ids) + assert 140 <= len(kept) <= 260 + assert _generations_exported_at(0.5, trace_ids) == kept + assert kept < _generations_exported_at(0.9, trace_ids) + + +@pytest.mark.parametrize("raw", ["1.5", "-0.5", "abc"]) +def test_unusable_sample_rate_warns_and_exports_everything( + monkeypatch: pytest.MonkeyPatch, raw: str, caplog: pytest.LogCaptureFixture +): + monkeypatch.setenv("LANGFUSE_SAMPLE_RATE", raw) + with caplog.at_level(logging.WARNING, logger="LiteLLM"): + assert configured_sample_rate() == 1.0 + assert "LANGFUSE_SAMPLE_RATE" in caplog.text + + +def test_configured_sample_rate_reads_the_env_var(monkeypatch: pytest.MonkeyPatch): + monkeypatch.delenv("LANGFUSE_SAMPLE_RATE", raising=False) + assert configured_sample_rate() == 1.0 + monkeypatch.setenv("LANGFUSE_SAMPLE_RATE", "0.25") + assert configured_sample_rate() == 0.25 + + +def _exported_generations(exporter: InMemorySpanExporter, tracing: LangfuseTracing, count: int) -> tuple: + for _ in range(count): + _generation(tracing, trace_id=resolve_trace_id(uuid.uuid4())).end(CALL_END) + tracing.flush() + return exporter.get_finished_spans() + + +def test_full_sample_rate_exports_every_trace_even_when_the_host_turned_otel_sampling_off( + monkeypatch: pytest.MonkeyPatch, +): + """A provider built without a sampler reads ``OTEL_TRACES_SAMPLER``, which belongs to the host's tracing.""" + monkeypatch.setenv("OTEL_TRACES_SAMPLER", "always_off") + exporter = InMemorySpanExporter() + tracing = build_langfuse_tracing( + exporter=exporter, environment=None, release=None, sample_rate=1.0, flush_interval_millis=10 + ) + assert len(_exported_generations(exporter, tracing, 5)) == 5 + + +@pytest.mark.parametrize( + ("variable", "value"), + [ + ("OTEL_SPAN_ATTRIBUTE_COUNT_LIMIT", "4"), + ("OTEL_ATTRIBUTE_COUNT_LIMIT", "4"), + ("OTEL_ATTRIBUTE_VALUE_LENGTH_LIMIT", "8"), + ("OTEL_SPAN_ATTRIBUTE_VALUE_LENGTH_LIMIT", "8"), + ], +) +def test_host_otel_span_limits_do_not_truncate_langfuse_observations( + monkeypatch: pytest.MonkeyPatch, variable: str, value: str +): + monkeypatch.setenv(variable, value) + exporter = InMemorySpanExporter() + tracing = build_langfuse_tracing( + exporter=exporter, environment=None, release=None, sample_rate=1.0, flush_interval_millis=10 + ) + attributes = {f"langfuse.observation.metadata.k{i}": "v" * 32 for i in range(40)} + _generation(tracing, attributes=attributes).end(CALL_END) + tracing.flush() + + span = _only_span(exporter, "gen") + assert span.dropped_attributes == 0 + assert all(span.attributes[key] == "v" * 32 for key in attributes) + + +def test_host_otel_resource_env_does_not_reach_the_langfuse_resource(monkeypatch: pytest.MonkeyPatch): + """``OTEL_RESOURCE_ATTRIBUTES`` and ``OTEL_SERVICE_NAME`` belong to the host's tracing; Langfuse files a + trace under any ``deployment.environment`` it finds on the resource, and v2 shipped no resource at all.""" + monkeypatch.setenv("OTEL_RESOURCE_ATTRIBUTES", "team.secret.note=internal-only,deployment.environment=hijack") + monkeypatch.setenv("OTEL_SERVICE_NAME", "the-hosts-own-service") + tracing = build_langfuse_tracing( + exporter=InMemorySpanExporter(), environment="prod", release="r1", sample_rate=1.0, flush_interval_millis=10 + ) + + assert dict(tracing.provider.resource.attributes) == {A.ENVIRONMENT: "prod", A.RELEASE: "r1"} + + +@pytest.mark.parametrize( + ("value", "encoded"), + [ + (2**53 - 1, ("int_value", 2**53 - 1)), + (-(2**53) + 1, ("int_value", -(2**53) + 1)), + (2**53, ("string_value", str(2**53))), + (2**63 - 1, ("string_value", str(2**63 - 1))), + (2**63, ("string_value", str(2**63))), + (10**20, ("string_value", str(10**20))), + (-(2**63) - 1, ("string_value", str(-(2**63) - 1))), + (True, ("bool_value", True)), + ], + ids=[ + "json-safe-max", + "json-safe-min", + "json-safe-plus-one", + "int64-max", + "int64-max-plus-one", + "huge", + "int64-min-minus-one", + "bool", + ], +) +def test_metadata_ints_past_the_json_safe_range_reach_the_wire_as_strings(value, encoded): + """OTLP carries int64 only and its encoder silently drops any attribute it cannot fit, while the export + still succeeds, and Langfuse's reader rounds ints past 2**53 (int64 max read back as 9223372036854776000 + on 2026-09-21, where the v2 leg showed the exact digits as a string), so both ranges go as strings.""" + from opentelemetry.exporter.otlp.proto.common.trace_encoder import encode_spans + + exporter = InMemorySpanExporter() + tracing = build_langfuse_tracing( + exporter=exporter, environment=None, release=None, sample_rate=1.0, flush_interval_millis=10 + ) + attributes = observation_attributes(observation_type="generation", metadata={"order_id": value, "sibling": "kept"}) + _generation(tracing, attributes=attributes).end(CALL_END) + tracing.flush() + + (encoded_span,) = encode_spans(exporter.get_finished_spans()).resource_spans[0].scope_spans[0].spans + wire = {kv.key: kv.value for kv in encoded_span.attributes} + order_id = wire[f"{A.OBSERVATION_METADATA}.order_id"] + carried = { + "int_value": order_id.int_value, + "string_value": order_id.string_value, + "bool_value": order_id.bool_value, + } + assert (order_id.WhichOneof("value"), carried[order_id.WhichOneof("value")]) == encoded + assert wire[f"{A.OBSERVATION_METADATA}.sibling"].string_value == "kept" + + +@pytest.mark.parametrize( + ("raw", "expected"), + [(None, 60.0), ("5", 5.0), ("0", 0.0), (" -1 ", 60.0), ("2.5", 60.0), ("abc", 60.0)], + ids=["unset", "whole", "zero", "negative", "fraction", "text"], +) +def test_prompt_cache_ttl_env_falls_back_instead_of_raising(monkeypatch: pytest.MonkeyPatch, raw, expected, caplog): + """The SDK reads this knob as whole seconds; a negative one passes its import but must not cache forever, + and anything else falls back rather than raising out of logger construction.""" + if raw is None: + monkeypatch.delenv("LANGFUSE_PROMPT_CACHE_DEFAULT_TTL_SECONDS", raising=False) + else: + monkeypatch.setenv("LANGFUSE_PROMPT_CACHE_DEFAULT_TTL_SECONDS", raw) + with caplog.at_level(logging.WARNING, logger="LiteLLM"): + assert configured_prompt_cache_ttl() == expected + assert ("LANGFUSE_PROMPT_CACHE_DEFAULT_TTL_SECONDS" in caplog.text) is (expected == 60.0 and raw is not None) + + +def test_many_metadata_keys_never_evict_the_generation_input_and_output(): + """OTel's default 128-attribute cap drops the earliest attributes, and v2 never capped metadata.""" + exporter = InMemorySpanExporter() + tracing = build_langfuse_tracing( + exporter=exporter, environment=None, release=None, sample_rate=1.0, flush_interval_millis=10 + ) + attributes = { + A.OBSERVATION_INPUT: "the-prompt", + A.OBSERVATION_OUTPUT: "the-completion", + **{f"langfuse.observation.metadata.k{i}": str(i) for i in range(300)}, + } + _generation(tracing, attributes=attributes).end(CALL_END) + tracing.flush() + + span = _only_span(exporter, "gen") + assert span.dropped_attributes == 0 + assert span.attributes[A.OBSERVATION_INPUT] == "the-prompt" + assert span.attributes[A.OBSERVATION_OUTPUT] == "the-completion" + assert span.attributes["langfuse.observation.metadata.k299"] == "299" + + +def test_otel_sdk_disabled_still_wins_but_is_called_out(monkeypatch: pytest.MonkeyPatch, caplog): + monkeypatch.setenv("OTEL_SDK_DISABLED", "true") + exporter = InMemorySpanExporter() + with caplog.at_level(logging.WARNING, logger="LiteLLM"): + tracing = build_langfuse_tracing( + exporter=exporter, environment=None, release=None, sample_rate=1.0, flush_interval_millis=10 + ) + assert "OTEL_SDK_DISABLED" in caplog.text + assert _exported_generations(exporter, tracing, 3) == () + + +def test_spans_carry_the_langfuse_sdk_scope_name(channel): + """Langfuse keys on the SDK's instrumentation scope (langfuse 4.15.2, ``langfuse/_client/constants.py``, + read 2026-09-17); any other scope is foreign OTel traffic whose raw attributes get echoed into metadata.""" + tracing, exporter = channel + _generation(tracing).end(CALL_END) + tracing.flush() + assert _only_span(exporter, "gen").instrumentation_scope.name == "langfuse-sdk" + + +class _GatedExporter(SpanExporter): + """Hold the export thread until released, so spans pile up in the processor queue.""" + + def __init__(self) -> None: + self.gate = threading.Event() + self.batches: list[int] = [] + + def export(self, spans) -> SpanExportResult: + self.gate.wait(timeout=30) + self.batches.append(len(spans)) + return SpanExportResult.SUCCESS + + def shutdown(self) -> None: + return None + + def force_flush(self, timeout_millis: int = 30_000) -> bool: + return True + + +def test_export_queue_holds_a_v2_sized_burst_while_the_destination_stalls(): + """v2 queued 100k events; OTel's default 2048 dropped most of a burst during a destination stall.""" + exporter = _GatedExporter() + tracing = build_langfuse_tracing( + exporter=exporter, environment=None, release=None, sample_rate=1.0, flush_interval_millis=10 + ) + for _ in range(6000): + _generation(tracing, trace_id=resolve_trace_id(uuid.uuid4())).end(CALL_END) + exporter.gate.set() + assert tracing.flush(timeout_millis=30_000) is True + assert sum(exporter.batches) == 6000 + + +@pytest.mark.parametrize( + ("raw", "expected"), + [(None, 512), ("64", 64), ("0", 512), ("-5", 512), ("abc", 512), ("100001", 512), ("100000", 100_000)], + ids=["unset", "valid", "zero", "negative", "text", "over-queue", "at-queue"], +) +def test_langfuse_flush_at_is_parsed_like_the_sdk_did(monkeypatch: pytest.MonkeyPatch, raw, expected, caplog): + if raw is None: + monkeypatch.delenv("LANGFUSE_FLUSH_AT", raising=False) + else: + monkeypatch.setenv("LANGFUSE_FLUSH_AT", raw) + with caplog.at_level(logging.WARNING, logger="LiteLLM"): + assert configured_flush_at() == expected + assert ("LANGFUSE_FLUSH_AT" in caplog.text) is (raw is not None and str(expected) != raw) + + +def test_langfuse_flush_at_sizes_the_export_batches(monkeypatch: pytest.MonkeyPatch): + monkeypatch.setenv("LANGFUSE_FLUSH_AT", "64") + exporter = _GatedExporter() + exporter.gate.set() + tracing = build_langfuse_tracing( + exporter=exporter, + environment=None, + release=None, + sample_rate=1.0, + flush_interval_millis=60_000, + flush_at=configured_flush_at(), + ) + for _ in range(200): + _generation(tracing, trace_id=resolve_trace_id(uuid.uuid4())).end(CALL_END) + tracing.flush() + assert sum(exporter.batches) == 200 + assert max(exporter.batches) == 64 + + +def test_acquired_channel_reads_langfuse_flush_at(monkeypatch: pytest.MonkeyPatch): + monkeypatch.setenv("LANGFUSE_FLUSH_AT", "7") + small = _acquire(public_key="pk-flush-at-test") + monkeypatch.setenv("LANGFUSE_FLUSH_AT", "9") + assert _acquire(public_key="pk-flush-at-test") is not small + + +def test_channel_does_not_take_over_the_process_tracer_provider(): + provider_before = otel_trace.get_tracer_provider() + + tracing = acquire_langfuse_tracing( + public_key="pk-global-test", + secret_key="sk", + base_url="http://127.0.0.1:1", + environment=None, + release=None, + flush_interval=1.0, + mock_mode=True, + ) + + assert otel_trace.get_tracer_provider() is provider_before + assert tracing.provider is not provider_before + + +def _acquire(**overrides): + parameters = { + "public_key": "pk-cache-test", + "secret_key": "sk-cache", + "base_url": "http://127.0.0.1:1", + "environment": None, + "release": None, + "flush_interval": 1.0, + "mock_mode": True, + } + return acquire_langfuse_tracing(**{**parameters, **overrides}) + + +def test_same_credentials_share_one_channel(): + assert _acquire() is _acquire() + + +@pytest.mark.parametrize( + "override", + [ + {"secret_key": "sk-rotated"}, + {"base_url": "http://127.0.0.1:2"}, + {"environment": "staging"}, + {"mock_mode": False}, + ], + ids=["secret", "host", "environment", "mock-to-live"], +) +def test_changed_credentials_or_settings_get_their_own_channel(override): + assert _acquire() is not _acquire(**override) + + +class _RecordsShutdown(InMemorySpanExporter): + def __init__(self) -> None: + super().__init__() + self.shutdowns = 0 + + def shutdown(self) -> None: + self.shutdowns += 1 + super().shutdown() + + +def _acquire_recorded(monkeypatch: pytest.MonkeyPatch, public_key: str) -> tuple[LangfuseTracing, _RecordsShutdown]: + exporter = _RecordsShutdown() + monkeypatch.setattr("litellm.integrations.langfuse.langfuse_sdk._build_span_exporter", lambda **_: exporter) + return _acquire(public_key=public_key, mock_mode=False, flush_interval=600.0), exporter + + +def test_channel_is_retired_only_after_its_last_holder_releases_it(monkeypatch: pytest.MonkeyPatch): + """Two loggers on one credential set share the channel: the first release must leave it + exporting for the second, and the last release must shut the batch thread down and drop + the registry entry so the next logger gets a fresh channel instead of a dead one.""" + first, exporter = _acquire_recorded(monkeypatch, "pk-lease-test") + second = _acquire(public_key="pk-lease-test", mock_mode=False, flush_interval=600.0) + assert second is first + + release_langfuse_tracing(first, grace_seconds=0.0) + second.tracer.start_span("generation").end() + assert exporter.shutdowns == 0 + assert flush_langfuse_tracing() is True + assert len(exporter.get_finished_spans()) == 1 + + release_langfuse_tracing(second, grace_seconds=0.0) + assert exporter.shutdowns == 1 + assert _acquire(public_key="pk-lease-test", mock_mode=False, flush_interval=600.0) is not first + + +def test_release_flushes_the_queued_spans_before_the_channel_goes_away(monkeypatch: pytest.MonkeyPatch): + tracing, exporter = _acquire_recorded(monkeypatch, "pk-lease-flush-test") + tracing.tracer.start_span("generation").end() + + release_langfuse_tracing(tracing, grace_seconds=0.0) + + assert len(exporter.get_finished_spans()) == 1 + + +def test_channel_reacquired_within_the_grace_is_kept(monkeypatch: pytest.MonkeyPatch): + """A logger rebuilt for the same credentials right after the old one expired, and a callback + that fetched the old logger just before expiry, both keep exporting through the same channel.""" + tracing, exporter = _acquire_recorded(monkeypatch, "pk-lease-grace-test") + + release_langfuse_tracing(tracing, grace_seconds=0.2) + assert _acquire(public_key="pk-lease-grace-test", mock_mode=False, flush_interval=600.0) is tracing + + threading.Event().wait(0.5) + tracing.tracer.start_span("generation").end() + assert exporter.shutdowns == 0 + assert flush_langfuse_tracing() is True + assert len(exporter.get_finished_spans()) == 1 + + +def test_retire_timer_of_an_earlier_release_cannot_kill_a_reacquired_channel(monkeypatch: pytest.MonkeyPatch): + """release, re-acquire, release: the first timer used to fire into a channel that a later holder still + counted on for its own grace period, shutting the batch thread down while spans were still queued.""" + tracing, exporter = _acquire_recorded(monkeypatch, "pk-lease-race-test") + + release_langfuse_tracing(tracing, grace_seconds=0.2) + assert _acquire(public_key="pk-lease-race-test", mock_mode=False, flush_interval=600.0) is tracing + release_langfuse_tracing(tracing, grace_seconds=600.0) + + threading.Event().wait(0.5) + assert exporter.shutdowns == 0 + assert _acquire(public_key="pk-lease-race-test", mock_mode=False, flush_interval=600.0) is tracing + tracing.tracer.start_span("generation").end() + assert tracing.flush() is True + assert len(exporter.get_finished_spans()) == 1 + + +class _RejectsEverything(SpanExporter): + def export(self, spans) -> SpanExportResult: + return SpanExportResult.FAILURE + + def shutdown(self) -> None: + return None + + def force_flush(self, timeout_millis: int = 30_000) -> bool: + return True + + +def test_flush_is_false_when_the_destination_rejected_a_batch(monkeypatch: pytest.MonkeyPatch): + """The shutdown hook logs "channels flushed" off this value; a drained queue whose batches all + failed at the destination is a loss, not a flush.""" + monkeypatch.setattr( + "litellm.integrations.langfuse.langfuse_sdk._build_span_exporter", lambda **_: _RejectsEverything() + ) + tracing = _acquire(public_key="pk-flush-truth-test", mock_mode=False, flush_interval=600.0) + tracing.tracer.start_span("generation").end() + + assert tracing.flush() is False + assert flush_langfuse_tracing() is True, "an empty queue after the loss has nothing left to fail" + + +def test_release_of_a_channel_the_registry_never_handed_out_is_a_no_op(): + exporter = InMemorySpanExporter() + tracing = build_langfuse_tracing( + exporter=exporter, environment=None, release=None, sample_rate=1.0, flush_interval_millis=10 + ) + + release_langfuse_tracing(tracing, grace_seconds=0.0) + tracing.tracer.start_span("generation").end() + + assert tracing.flush() is True + assert len(exporter.get_finished_spans()) == 1 + + +def test_flush_langfuse_tracing_exports_the_queued_spans_of_every_channel(monkeypatch: pytest.MonkeyPatch): + """The proxy shutdown hook flushes through this, so a span finished just before a + graceful restart must reach the exporter without waiting for the batch interval.""" + exporters: Final[ + list[InMemorySpanExporter] + ] = [] # mutable-ok: collects the exporters the patched builder hands out + + def build_in_memory(*, public_key: str, secret_key: str, base_url: str) -> InMemorySpanExporter: + exporters.append(InMemorySpanExporter()) + return exporters[-1] + + monkeypatch.setattr("litellm.integrations.langfuse.langfuse_sdk._build_span_exporter", build_in_memory) + for public_key in ("pk-flush-test-a", "pk-flush-test-b"): + _acquire(public_key=public_key, mock_mode=False, flush_interval=600.0).tracer.start_span("generation").end() + + assert [len(exporter.get_finished_spans()) for exporter in exporters] == [0, 0] + assert flush_langfuse_tracing() is True + assert [len(exporter.get_finished_spans()) for exporter in exporters] == [1, 1] + + +def test_flush_langfuse_tracing_flushes_channels_concurrently_under_one_deadline(monkeypatch: pytest.MonkeyPatch): + """A channel stuck on an unreachable host must not spend the whole deadline before the + next channel gets its turn; the first exporter here only returns once the second exported.""" + second_exported = threading.Event() + + class WaitsForTheOther(SpanExporter): + def export(self, spans): + return SpanExportResult.SUCCESS if second_exported.wait(timeout=5.0) else SpanExportResult.FAILURE + + def shutdown(self) -> None: + return None + + class Unblocks(SpanExporter): + def export(self, spans): + second_exported.set() + return SpanExportResult.SUCCESS + + def shutdown(self) -> None: + return None + + exporters = iter((WaitsForTheOther(), Unblocks())) + + def build_next(*, public_key: str, secret_key: str, base_url: str) -> SpanExporter: + return next(exporters) + + monkeypatch.setattr("litellm.integrations.langfuse.langfuse_sdk._build_span_exporter", build_next) + for public_key in ("pk-concurrent-flush-a", "pk-concurrent-flush-b"): + _acquire(public_key=public_key, mock_mode=False, flush_interval=600.0).tracer.start_span("generation").end() + + assert flush_langfuse_tracing(timeout_millis=2_000) is True + assert second_exported.is_set() + + +def test_flush_langfuse_tracing_leaves_an_overrunning_channel_on_a_daemon_thread(): + """A channel whose flush outlives the deadline is reported as failed and must not be able to + hold up interpreter exit, so the thread still flushing it has to be a daemon.""" + release = threading.Event() + + class BlocksUntilReleased(SpanProcessor): + def force_flush(self, timeout_millis: int = 30_000) -> bool: + return release.wait(timeout=10.0) + + _acquire(public_key="pk-overrunning-flush", mock_mode=True, flush_interval=600.0).provider.add_span_processor( + BlocksUntilReleased() + ) + try: + assert flush_langfuse_tracing(timeout_millis=200) is False + stuck = [thread for thread in threading.enumerate() if thread.name.startswith("langfuse-flush")] + assert stuck and all(thread.daemon for thread in stuck) + finally: + release.set() + + +def test_a_changed_sample_rate_rebuilds_the_channel(monkeypatch: pytest.MonkeyPatch): + monkeypatch.setenv("LANGFUSE_SAMPLE_RATE", "0.25") + quarter = _acquire(public_key="pk-resample-test") + monkeypatch.setenv("LANGFUSE_SAMPLE_RATE", "1") + full = _acquire(public_key="pk-resample-test") + + assert full is not quarter + assert quarter.provider.sampler.get_description() == "TraceIdHashSampler{0.25}" + assert "TraceIdHashSampler" not in full.provider.sampler.get_description() + + +def _recording_transport(requests: list[httpx.Request], status: int = 401) -> httpx.Client: + def record(request: httpx.Request) -> httpx.Response: + requests.append(request) + return httpx.Response(status, json=_PROJECTS_BODY if status == 200 else {"message": "unauthorized"}) + + return httpx.Client(transport=httpx.MockTransport(record)) + + +_PROJECTS_BODY: Final = { + "data": [{"id": "proj-under-test", "name": "p", "metadata": {}, "organization": {"id": "o", "name": "o"}}] +} + + +def test_rest_client_authenticates_with_the_credentials_it_was_built_with(): + """Two loggers for one public key but different secrets or hosts each talk to their own project.""" + requests: list[httpx.Request] = [] + build_langfuse_client( + public_key="pk-rest-test", + secret_key="sk-first", + base_url="http://127.0.0.1:1", + httpx_client=_recording_transport(requests), + ) + rotated = build_langfuse_client( + public_key="pk-rest-test", + secret_key="sk-second", + base_url="http://127.0.0.1:2", + httpx_client=_recording_transport(requests), + ) + + assert rotated.auth_check() is not None + assert requests[-1].url.host == "127.0.0.1" and requests[-1].url.port == 2 + assert requests[-1].headers["authorization"] == "Basic " + b64encode(b"pk-rest-test:sk-second").decode() + + +def test_rest_client_without_keys_fails_auth_check_instead_of_raising(monkeypatch): + """``/health/services?service=langfuse`` with no credentials must report a failed check, not crash.""" + for name in ("LANGFUSE_PUBLIC_KEY", "LANGFUSE_SECRET_KEY"): + monkeypatch.delenv(name, raising=False) + client = build_langfuse_client(public_key=None, secret_key=None, base_url="http://127.0.0.1:1", httpx_client=None) + assert client.auth_check() is not None + + +def test_auth_check_names_the_servers_rejection(caplog): + """``/health/services`` used to print the 401 verbatim; a generic credentials message hides a 403 or a 500.""" + client = build_langfuse_client( + public_key="pk", + secret_key="sk", + base_url="http://127.0.0.1:1", + httpx_client=_recording_transport([], status=401), + ) + with caplog.at_level(logging.WARNING, logger="LiteLLM"): + failure = client.auth_check() + assert failure is not None + assert failure.reason == "status_code: 401, body: {'message': 'unauthorized'}" + assert failure.reason in caplog.text + + +def test_auth_check_names_an_unreachable_destination_rather_than_the_keys(): + def refuse(request: httpx.Request) -> httpx.Response: + raise httpx.ConnectError("connection refused by lf.internal.example", request=request) + + client = build_langfuse_client( + public_key="pk", + secret_key="sk", + base_url="http://lf.internal.example", + httpx_client=httpx.Client(transport=httpx.MockTransport(refuse)), + ) + failure = client.auth_check() + assert failure is not None + assert "connection refused by lf.internal.example" in failure.reason + + +def test_auth_check_fails_when_the_keys_reach_no_project(): + """A 200 with an empty project list is what the SDK's own ``auth_check`` raises on; it is not a pass.""" + client = build_langfuse_client( + public_key="pk", + secret_key="sk", + base_url="http://127.0.0.1:1", + httpx_client=httpx.Client(transport=httpx.MockTransport(lambda _: httpx.Response(200, json={"data": []}))), + ) + failure = client.auth_check() + assert failure is not None + assert "no project" in failure.reason + + +@pytest.mark.parametrize("status", [500, 503, 429], ids=["http-500", "http-503", "http-429"]) +def test_auth_check_and_project_id_make_one_round_trip_when_langfuse_is_down(status): + """Both run on the event loop; the generated client's default retries sleep for seconds, or for Retry-After.""" + requests: list[httpx.Request] = [] + + def fail(request: httpx.Request) -> httpx.Response: + requests.append(request) + return httpx.Response(status, request=request, headers={"retry-after": "20"}, json={"message": "down"}) + + client = build_langfuse_client( + public_key="pk", + secret_key="sk", + base_url="http://127.0.0.1:1", + httpx_client=httpx.Client(transport=httpx.MockTransport(fail)), + ) + + started = monotonic() + failure = client.auth_check() + with pytest.raises(ApiError): + client.project_id() + assert failure is not None and f"status_code: {status}" in failure.reason + assert len(requests) == 2 + assert monotonic() - started < 0.5 + + +@pytest.mark.parametrize( + ("status", "round_trips"), + [(500, 2), (503, 2), (429, 1), (404, 1)], + ids=["http-500", "http-503", "http-429", "http-404"], +) +def test_cold_prompt_miss_never_sleeps_when_langfuse_is_down(status: int, round_trips: int): + """A cold ``get_prompt`` fetches inline on the event loop; with the generated client's default retries a + 429 carrying ``Retry-After: 30`` used to hold the loop for a minute. A 5xx gets the v2 client's one + quick retry, a 429 or 4xx none.""" + requests: list[httpx.Request] = [] + + def fail(request: httpx.Request) -> httpx.Response: + requests.append(request) + return httpx.Response(status, request=request, headers={"retry-after": "30"}, json={"message": "down"}) + + client = build_langfuse_client( + public_key="pk", + secret_key="sk", + base_url="http://127.0.0.1:1", + httpx_client=httpx.Client(transport=httpx.MockTransport(fail)), + ) + + started = monotonic() + with pytest.raises(LangfusePromptError) as caught: + client.get_prompt("greeting") + assert len(requests) == round_trips + assert monotonic() - started < 0.5 + assert caught.value.status_code == status + + +@pytest.mark.parametrize("first_failure", [503, "connect-error"], ids=["http-503", "connect-error"]) +def test_one_transient_failure_on_a_cold_prompt_miss_does_not_fail_the_call(first_failure: int | str): + """The v2 client retried a cold fetch once; a single Langfuse blip must not fail the LLM call.""" + requests: list[httpx.Request] = [] + + def flaky(request: httpx.Request) -> httpx.Response: + requests.append(request) + if len(requests) > 1: + return httpx.Response(200, request=request, json=_TEXT_PROMPT_BODY) + if isinstance(first_failure, int): + return httpx.Response(first_failure, request=request, json={"message": "down"}) + raise httpx.ConnectError("refused", request=request) + + client = build_langfuse_client( + public_key="pk", + secret_key="sk", + base_url="http://127.0.0.1:1", + httpx_client=httpx.Client(transport=httpx.MockTransport(flaky)), + ) + + started = monotonic() + assert client.get_prompt("greeting").compile() == "hello" + assert len(requests) == 2 + assert monotonic() - started < 0.5 + assert client.get_prompt("greeting").compile() == "hello", "the retried prompt is cached like any other" + assert len(requests) == 2 + + +def test_prompt_fetch_error_carries_status_and_body_but_no_upstream_headers(): + """The proxy forwards an exception's ``headers`` to its client and prints ``str(e)``; the generated + ``ApiError`` carries Langfuse's response headers in both.""" + upstream_headers = {"server": "langfuse-edge", "set-cookie": "session=abc; HttpOnly", "x-upstream-internal": "1"} + + def not_found(request: httpx.Request) -> httpx.Response: + return httpx.Response(404, request=request, headers=upstream_headers, json={"message": "Prompt not found"}) + + client = build_langfuse_client( + public_key="pk", + secret_key="sk", + base_url="http://127.0.0.1:1", + httpx_client=httpx.Client(transport=httpx.MockTransport(not_found)), + ) + + with pytest.raises(Exception, match="Prompt not found") as caught: + client.get_prompt("missing") + + error = caught.value + assert getattr(error, "headers", None) is None + assert getattr(error, "status_code", None) == 404 + assert not any(header in str(error) for header in upstream_headers) + assert error.__cause__ is None and error.__suppress_context__, "the header-bearing ApiError must not ride along" + + +_TEXT_PROMPT_BODY: Final[dict[str, object]] = { + "type": "text", + "name": "n", + "version": 1, + "config": {}, + "labels": ["production"], + "tags": [], + "prompt": "hello", +} + + +@pytest.mark.parametrize( + ("name", "encoded"), + [ + ("what?", "what%3F"), + ("folder/greeting", "folder%2Fgreeting"), + ("my-prompt?label=staging", "my-prompt%3Flabel%3Dstaging"), + ("100% sure#1", "100%25%20sure%231"), + ], + ids=["question-mark", "folder-slash", "query-injection", "percent-space-hash"], +) +def test_prompt_name_is_url_encoded_into_the_request_path(name: str, encoded: str): + """The v2 client quoted the name before building the path and the v4 SDK's ``get_prompt`` does too; the + generated client alone puts the raw name into the URL, so ``what?`` fetched prompt ``what`` and + ``a/b`` left the prompts route.""" + requests: list[httpx.Request] = [] + + def record(request: httpx.Request) -> httpx.Response: + requests.append(request) + return httpx.Response(200, request=request, json=_TEXT_PROMPT_BODY) + + client = build_langfuse_client( + public_key="pk", + secret_key="sk", + base_url="http://127.0.0.1:1", + httpx_client=httpx.Client(transport=httpx.MockTransport(record)), + ) + + client.get_prompt(name, label="staging") + + (request,) = requests + assert request.url.raw_path == f"/api/public/v2/prompts/{encoded}?label=staging".encode() + + +def test_rest_client_reports_the_project_id_and_a_passing_auth_check(): + requests: list[httpx.Request] = [] + client = build_langfuse_client( + public_key="pk-project-test", + secret_key="sk", + base_url="http://127.0.0.1:1", + httpx_client=_recording_transport(requests, status=200), + ) + assert client.project_id() == "proj-under-test" + assert client.auth_check() is None + + +def test_rest_client_leaves_a_host_applications_langfuse_client_alone(): + """The SDK hands every ``Langfuse()`` built for one public key the same resource bundle, so a + litellm-built SDK client used to make a host application's client fetch through litellm's + host, secret and httpx client. litellm now speaks REST directly and registers nothing.""" + from langfuse import Langfuse + + requests: list[httpx.Request] = [] + litellm_client = build_langfuse_client( + public_key="pk-shared-with-host", + secret_key="sk-litellm", + base_url="http://litellm.example", + httpx_client=_recording_transport(requests, status=200), + ) + assert litellm_client.project_id() == "proj-under-test" + + host_requests: list[httpx.Request] = [] + host = Langfuse( + public_key="pk-shared-with-host", + secret_key="sk-host", + base_url="http://host.example", + httpx_client=_recording_transport(host_requests, status=200), + tracing_enabled=False, + ) + try: + assert host.auth_check() is True + finally: + host.shutdown() + + assert [request.url.host for request in requests] == ["litellm.example"] + assert host_requests[-1].url.host == "host.example" + assert host_requests[-1].headers["authorization"] == "Basic " + b64encode(b"pk-shared-with-host:sk-host").decode() + + +def test_rest_client_does_not_take_over_the_process_tracer_provider(): + provider_before = otel_trace.get_tracer_provider() + build_langfuse_client( + public_key="pk-sdk-global-test", secret_key="sk", base_url="http://127.0.0.1:1", httpx_client=None + ) + assert otel_trace.get_tracer_provider() is provider_before + + +def _finished_span(): + provider = TracerProvider() + span = provider.get_tracer("t").start_span("generation") + span.end() + return span + + +def _exporter_over(responses, *, delays=(0.5, 1.5), timeout=5.0): + """A LangfuseSpanExporter whose litellm HTTPHandler talks to a scripted transport instead of the network.""" + from litellm.llms.custom_httpx.http_handler import HTTPHandler + + seen = [] + script = list(responses) + + def transport(request: httpx.Request) -> httpx.Response: + seen.append(request) + step = script.pop(0) + if isinstance(step, Exception): + raise step + return httpx.Response(step, request=request) + + handler = HTTPHandler(client=httpx.Client(transport=httpx.MockTransport(transport))) + exporter = LangfuseSpanExporter( + handler=handler, + endpoint="https://lf.internal.example/api/public/otel/v1/traces", + headers=MappingProxyType({"Authorization": "Basic cGs6c2s=", "Content-Type": "application/x-protobuf"}), + timeout=timeout, + delays=delays, + ) + return exporter, seen + + +def test_exporter_posts_the_otlp_batch_through_litellm_http_handler(monkeypatch): + """Traces travel through litellm's own handler, so litellm's TLS and proxy settings apply to them.""" + from opentelemetry.proto.collector.trace.v1.trace_service_pb2 import ExportTraceServiceRequest + + slept = [] + monkeypatch.setattr("litellm.integrations.langfuse.langfuse_sdk.sleep", slept.append) + exporter, seen = _exporter_over([200]) + span = _finished_span() + + assert exporter.export((span,)) is SpanExportResult.SUCCESS + + (request,) = seen + assert request.method == "POST" + assert str(request.url) == "https://lf.internal.example/api/public/otel/v1/traces" + assert request.headers["Authorization"] == "Basic cGs6c2s=" + assert request.headers["Content-Type"] == "application/x-protobuf" + decoded = ExportTraceServiceRequest() + decoded.ParseFromString(request.content) + exported = decoded.resource_spans[0].scope_spans[0].spans[0] + assert exported.name == "generation" + assert exported.span_id == span.context.span_id.to_bytes(8, "big") + assert slept == [] + + +@pytest.mark.parametrize( + "failure", + [httpx.ReadTimeout("stalled"), httpx.ConnectError("refused"), 503, 429, 408, 501, 507, 599], + ids=["read-timeout", "connect-error", "http-503", "http-429", "http-408", "http-501", "http-507", "http-599"], +) +def test_exporter_retries_a_failed_round_trip_and_then_succeeds(monkeypatch, failure): + """A stalled or restarting destination used to drop the batch outright; v2 backed off and re-sent every 5xx.""" + slept = [] + monkeypatch.setattr("litellm.integrations.langfuse.langfuse_sdk.sleep", slept.append) + exporter, seen = _exporter_over([failure, failure, 200], delays=(0.5, 1.5, 2.5)) + + assert exporter.export((_finished_span(),)) is SpanExportResult.SUCCESS + assert len(seen) == 3 + assert len({request.content for request in seen}) == 1 + assert slept == [0.5, 1.5] + + +def test_exporter_gives_up_after_the_last_delay(monkeypatch): + slept = [] + monkeypatch.setattr("litellm.integrations.langfuse.langfuse_sdk.sleep", slept.append) + exporter, seen = _exporter_over([httpx.ConnectError("refused")] * 3, delays=(1.0, 2.0)) + + assert exporter.export((_finished_span(),)) is SpanExportResult.FAILURE + assert len(seen) == 3 + assert slept == [1.0, 2.0] + + +def _exporter_with_body_cap(max_bytes: int, *, deliveries: list[int]): + """A destination that answers 413 to any body over ``max_bytes``, the way an ingress with a body limit does.""" + from litellm.llms.custom_httpx.http_handler import HTTPHandler + + def transport(request: httpx.Request) -> httpx.Response: + if len(request.content) > max_bytes: + return httpx.Response(413, request=request) + deliveries.append(len(request.content)) + return httpx.Response(200, request=request) + + return LangfuseSpanExporter( + handler=HTTPHandler(client=httpx.Client(transport=httpx.MockTransport(transport))), + endpoint="https://lf.internal.example/api/public/otel/v1/traces", + headers=MappingProxyType({}), + timeout=5.0, + delays=(), + ) + + +def test_exporter_splits_a_batch_the_destination_finds_too_large(monkeypatch): + """One 413 used to drop every span in the batch; the v2 consumer sized its batches by bytes before posting.""" + monkeypatch.setattr("litellm.integrations.langfuse.langfuse_sdk.sleep", lambda _: None) + spans = tuple(_finished_span() for _ in range(8)) + whole = _encode(spans) + assert whole is not None + deliveries: list[int] = [] + exporter = _exporter_with_body_cap(len(whole) // 2, deliveries=deliveries) + + assert exporter.export(spans) is SpanExportResult.SUCCESS + assert len(deliveries) >= 2 + assert all(size <= len(whole) // 2 for size in deliveries) + assert ( + sum(deliveries) >= len(whole) - 8 * 8 + ) # each half repeats the resource and scope envelope, spans are not lost + + +def test_exporter_drops_only_the_single_span_that_alone_exceeds_the_cap(monkeypatch, caplog): + monkeypatch.setattr("litellm.integrations.langfuse.langfuse_sdk.sleep", lambda _: None) + provider = TracerProvider() + huge = provider.get_tracer("t").start_span("generation", attributes={"body": "x" * 4000}) + huge.end() + small = tuple(_finished_span() for _ in range(3)) + single_small = _encode(small[:1]) + assert single_small is not None + deliveries: list[int] = [] + exporter = _exporter_with_body_cap(len(single_small) * 3, deliveries=deliveries) + + with caplog.at_level(logging.ERROR, logger="LiteLLM"): + result = exporter.export((*small, huge)) + + assert result is SpanExportResult.FAILURE + assert len(deliveries) >= 1 and all(size <= len(single_small) * 3 for size in deliveries) + assert "single" in caplog.text and "too large" in caplog.text + + +def _decoded_attributes(body: bytes) -> dict[str, str]: + decoded = ExportTraceServiceRequest() + decoded.ParseFromString(body) + return { + attribute.key: attribute.value.string_value + for attribute in decoded.resource_spans[0].scope_spans[0].spans[0].attributes + } + + +def _generation_span(**attributes: str): + provider = TracerProvider() + span = provider.get_tracer("t").start_span("generation", attributes=attributes) + span.end() + return span + + +def test_exporter_truncates_a_single_oversized_span_the_way_v2_did_instead_of_dropping_it(monkeypatch, caplog): + """v2 replaced the largest of input, output and metadata with a marker and still delivered the observation; a + vision request over a self-hosted ingress cap used to lose the whole generation, model and usage included.""" + monkeypatch.setattr("litellm.integrations.langfuse.langfuse_sdk.sleep", lambda _: None) + span = _generation_span( + **{ + "langfuse.observation.input": "data:image/png;base64," + "A" * 6000, + "langfuse.trace.input": "data:image/png;base64," + "A" * 200, + "langfuse.observation.output": "o" * 1000, + "langfuse.observation.metadata.team": "m" * 100, + "langfuse.observation.model.name": "gpt-4o", + } + ) + bodies: list[bytes] = [] + + def transport(request: httpx.Request) -> httpx.Response: + if len(request.content) > 2000: + return httpx.Response(413, request=request) + bodies.append(request.content) + return httpx.Response(200, request=request) + + exporter = LangfuseSpanExporter( + handler=HTTPHandler(client=httpx.Client(transport=httpx.MockTransport(transport))), + endpoint="https://lf.internal.example/api/public/otel/v1/traces", + headers=MappingProxyType({}), + timeout=5.0, + delays=(), + ) + with caplog.at_level(logging.WARNING, logger="LiteLLM"): + result = exporter.export((span,)) + + assert result is SpanExportResult.SUCCESS + delivered = _decoded_attributes(bodies[-1]) + assert delivered["langfuse.observation.input"] == "" + assert delivered["langfuse.trace.input"] == "" + assert delivered["langfuse.observation.output"] == "o" * 1000 + assert delivered["langfuse.observation.metadata.team"] == "m" * 100 + assert delivered["langfuse.observation.model.name"] == "gpt-4o" + assert "dropping it" not in caplog.text and "truncated" in caplog.text + + +def test_exporter_truncates_largest_first_and_drops_only_when_nothing_is_left(monkeypatch, caplog): + """Langfuse stores a bare ``langfuse.observation.metadata`` string as nothing, so the metadata marker travels + under a flattened key the way every other metadata value does.""" + monkeypatch.setattr("litellm.integrations.langfuse.langfuse_sdk.sleep", lambda _: None) + span = _generation_span( + **{ + "langfuse.observation.input": "i" * 3000, + "langfuse.observation.output": "o" * 2000, + "langfuse.observation.metadata.a": "m" * 500, + "langfuse.trace.metadata.b": "m" * 500, + } + ) + posted: list[dict[str, str]] = [] + + def always_too_large(request: httpx.Request) -> httpx.Response: + posted.append(_decoded_attributes(request.content)) + return httpx.Response(413, request=request) + + exporter = LangfuseSpanExporter( + handler=HTTPHandler(client=httpx.Client(transport=httpx.MockTransport(always_too_large))), + endpoint="https://lf.internal.example/api/public/otel/v1/traces", + headers=MappingProxyType({}), + timeout=5.0, + delays=(), + ) + with caplog.at_level(logging.ERROR, logger="LiteLLM"): + assert exporter.export((span,)) is SpanExportResult.FAILURE + + marker = "" + assert [sorted(key for key, value in body.items() if value == marker) for body in posted] == [ + [], + ["langfuse.observation.input"], + ["langfuse.observation.input", "langfuse.observation.output"], + [ + "langfuse.observation.input", + "langfuse.observation.metadata.truncated", + "langfuse.observation.output", + "langfuse.trace.metadata.truncated", + ], + ] + assert "langfuse.observation.metadata.a" not in posted[-1] and "langfuse.trace.metadata.b" not in posted[-1] + assert "dropping it" in caplog.text + + +@pytest.mark.parametrize("status", [400, 401, 403, 404, 422, 499]) +def test_exporter_does_not_retry_a_rejected_batch(monkeypatch, status): + """Bad credentials or a bad payload will not get better on the next attempt, so retrying only delays the flush.""" + slept = [] + monkeypatch.setattr("litellm.integrations.langfuse.langfuse_sdk.sleep", slept.append) + exporter, seen = _exporter_over([status, 200]) + + assert exporter.export((_finished_span(),)) is SpanExportResult.FAILURE + assert len(seen) == 1 + assert slept == [] + + +def test_exporter_names_the_server_floor_when_the_otlp_route_is_missing(monkeypatch, caplog): + """A Langfuse server too old to serve the OTLP route answers 404; a bare status leaves the operator guessing.""" + monkeypatch.setattr("litellm.integrations.langfuse.langfuse_sdk.sleep", lambda _: None) + exporter, _ = _exporter_over([404]) + + with caplog.at_level(logging.ERROR, logger="LiteLLM"): + assert exporter.export((_finished_span(),)) is SpanExportResult.FAILURE + + assert "HTTP 404" in caplog.text and "3.63.0" in caplog.text + + +def _finished_span_named(name: object): + provider = TracerProvider() + span = provider.get_tracer("t").start_span("placeholder") + span._name = name # pyright: ignore[reportAttributeAccessIssue, reportPrivateUsage] # the SDK only stores str + span.end() + return span + + +def test_exporter_drops_a_span_the_encoder_rejects_and_still_posts_the_rest(monkeypatch, caplog): + """One span the OTLP encoder cannot serialize used to raise out of ``export`` and lose every span in the batch.""" + from opentelemetry.proto.collector.trace.v1.trace_service_pb2 import ExportTraceServiceRequest + + monkeypatch.setattr("litellm.integrations.langfuse.langfuse_sdk.sleep", lambda _: None) + exporter, seen = _exporter_over([200]) + + with caplog.at_level(logging.ERROR, logger="LiteLLM"): + result = exporter.export((_finished_span(), _finished_span_named(12345), _finished_span())) + + assert result is SpanExportResult.SUCCESS + (request,) = seen + decoded = ExportTraceServiceRequest() + decoded.ParseFromString(request.content) + assert [span.name for span in decoded.resource_spans[0].scope_spans[0].spans] == ["generation", "generation"] + assert "dropped 1 span(s)" in caplog.text + + +def test_exporter_reports_failure_when_no_span_of_the_batch_can_be_encoded(monkeypatch): + exporter, seen = _exporter_over([200]) + + assert exporter.export((_finished_span_named(12345),)) is SpanExportResult.FAILURE + assert seen == [] + + +def test_built_exporter_uses_the_shared_litellm_handler_and_langfuse_headers(monkeypatch): + """No private requests session or TLS adapter: the channel is the same handler the rest of litellm uses.""" + from litellm.llms.custom_httpx.http_handler import _get_httpx_client + + monkeypatch.delenv("LANGFUSE_TIMEOUT", raising=False) + monkeypatch.delenv("LANGFUSE_MAX_RETRIES", raising=False) + default = _build_span_exporter(public_key="pk", secret_key="sk", base_url="https://lf.internal.example") + assert default.handler is _get_httpx_client() + assert default.timeout == 20 + assert len(default.delays) == 3 + + monkeypatch.setenv("LANGFUSE_TIMEOUT", "7.5") + monkeypatch.setenv("LANGFUSE_MAX_RETRIES", "1") + exporter = _build_span_exporter(public_key="pk", secret_key="sk", base_url="https://lf.internal.example") + assert exporter.endpoint == "https://lf.internal.example/api/public/otel/v1/traces" + assert exporter.timeout == 7.5 + assert exporter.delays == (1.0,) + assert exporter.headers["Authorization"] == "Basic " + b64encode(b"pk:sk").decode() + assert exporter.headers["x-langfuse-public-key"] == "pk" + assert exporter.headers["x-langfuse-sdk-version"] == installed_langfuse_version() + assert exporter.headers["x-langfuse-ingestion-version"] == "4" + + +def test_large_retry_count_builds_an_exporter_with_capped_backoff(monkeypatch): + """``LANGFUSE_MAX_RETRIES=1025`` constructed a v2 client; here ``2.0**1024`` would raise ``OverflowError`` + and take the whole callback down at init.""" + monkeypatch.setenv("LANGFUSE_MAX_RETRIES", "1025") + exporter = _build_span_exporter(public_key="pk", secret_key="sk", base_url="https://lf.internal.example") + assert 3 < len(exporter.delays) <= 1025 + assert exporter.delays[:4] == (1.0, 2.0, 4.0, 8.0) + assert max(exporter.delays) == exporter.delays[-1] <= 64.0 + + +def test_absurd_retry_count_is_clamped_instead_of_allocating_one_delay_per_retry(monkeypatch, caplog): + """A retry count with twelve digits must not turn callback init into a multi-gigabyte tuple allocation.""" + monkeypatch.setenv("LANGFUSE_MAX_RETRIES", "999999999999") + with caplog.at_level(logging.WARNING, logger="LiteLLM"): + exporter = _build_span_exporter(public_key="pk", secret_key="sk", base_url="https://lf.internal.example") + assert 3 < len(exporter.delays) <= 1025 + assert exporter.delays[-1] <= 64.0 + assert any("LANGFUSE_MAX_RETRIES=999999999999" in record.getMessage() for record in caplog.records) + + caplog.clear() + monkeypatch.setenv("LANGFUSE_MAX_RETRIES", "5") + with caplog.at_level(logging.WARNING, logger="LiteLLM"): + modest = _build_span_exporter(public_key="pk", secret_key="sk", base_url="https://lf.internal.example") + assert len(modest.delays) == 5 + assert not any("LANGFUSE_MAX_RETRIES" in record.getMessage() for record in caplog.records) + + +def test_enable_langfuse_debug_logging_makes_deliveries_visible_on_the_langfuse_logger(caplog): + """``LANGFUSE_DEBUG`` turned on the v2 SDK's own logger; it has to do the same for litellm's export channel.""" + exporter, _ = _exporter_over([200]) + langfuse_logger = logging.getLogger("langfuse") + level_before = langfuse_logger.level + try: + with caplog.at_level(logging.INFO, logger="langfuse"): + assert exporter.export((_finished_span(),)) is SpanExportResult.SUCCESS + assert "Exported" not in caplog.text + enable_langfuse_debug_logging() + assert langfuse_logger.level == logging.DEBUG + exporter_after, _ = _exporter_over([200]) + assert exporter_after.export((_finished_span(),)) is SpanExportResult.SUCCESS + assert "Exported" in caplog.text and "lf.internal.example" in caplog.text + finally: + langfuse_logger.setLevel(level_before) + + +@pytest.mark.parametrize( + ("base_url", "export_path", "expected"), + [ + ("https://lf.internal.example/", None, "https://lf.internal.example/api/public/otel/v1/traces"), + ("https://lf.internal.example", "/otel/traces", "https://lf.internal.example/otel/traces"), + ("https://lf.internal.example/", "/otel/traces", "https://lf.internal.example/otel/traces"), + ("https://lf.internal.example", "otel/traces", "https://lf.internal.example/otel/traces"), + ( + "https://lf.internal.example", + "//elsewhere.example/otel", + "https://lf.internal.example/elsewhere.example/otel", + ), + ( + "https://lf.internal.example", + "https://elsewhere.example/otel", + "https://lf.internal.example/https://elsewhere.example/otel", + ), + ], + ids=[ + "default", + "leading-slash", + "both-slashes", + "no-slash", + "scheme-relative-stays-on-host", + "absolute-stays-on-host", + ], +) +def test_export_endpoint_never_doubles_the_slash_or_leaves_the_configured_host( + monkeypatch, base_url, export_path, expected +): + if export_path is None: + monkeypatch.delenv("LANGFUSE_OTEL_TRACES_EXPORT_PATH", raising=False) + else: + monkeypatch.setenv("LANGFUSE_OTEL_TRACES_EXPORT_PATH", export_path) + + exporter = _build_span_exporter(public_key="pk", secret_key="sk", base_url=base_url) + + assert exporter.endpoint == expected + + +class _RecordingPromptsApi: + """Answers ``prompts.get`` with a text prompt that names the label it was asked for.""" + + def __init__(self) -> None: + self.prompts = self + self.requests: list[tuple[str, int | None, str | None]] = [] # mutable-ok: test-side call log + + def get(self, name: str, *, version: int | None, label: str | None, request_options: RequestOptions): + from langfuse.api import Prompt_Text + + assert request_options.get("max_retries") == 0, "a prompt fetch must not sleep through the client's retries" + self.requests.append((name, version, label)) + return Prompt_Text( + name=name, + version=version or 1, + config={}, + labels=[label or "production"], + tags=[], + prompt=f"label={label!r}", + ) + + +class _BlockingPromptsApi(_RecordingPromptsApi): + """Every fetch after the first blocks until the test releases it, and may be told to fail.""" + + def __init__(self) -> None: + super().__init__() + self.release = threading.Event() + self.fail_refresh = False + + def get(self, name: str, *, version: int | None, label: str | None, request_options: RequestOptions): + is_refresh = bool(self.requests) + prompt = super().get(name, version=version, label=label, request_options=request_options) + if is_refresh: + assert self.release.wait(5), "refresh was never released" + if self.fail_refresh: + raise RuntimeError("langfuse is down") + return prompt + + +def _wait_until(predicate, timeout: float = 5.0) -> None: + for _ in range(int(timeout / 0.01)): + if predicate(): + return + sleep(0.01) + raise AssertionError("condition not met in time") + + +def test_stale_prompt_is_served_at_once_while_the_refresh_runs_elsewhere(): + """``get_prompt`` runs on the proxy's event loop; a stale entry used to refetch inline and block every + request on the REST round trip. The stale prompt is returned immediately and refreshed off-thread.""" + api = _BlockingPromptsApi() + client = LangfuseApiClient(api, prompt_cache_ttl_seconds=0) # pyright: ignore[reportArgumentType] # duck-typed prompts API + + first = client.get_prompt("greeting") + started = monotonic() + stale = client.get_prompt("greeting") + + assert stale is first, "the stale prompt must come back without waiting on the refresh" + assert monotonic() - started < 1.0, "the stale read waited on the blocked refresh" + _wait_until(lambda: len(api.requests) == 2) + api.release.set() + _wait_until(lambda: client.get_prompt("greeting") is not first) + + +def test_a_failed_background_refresh_keeps_the_stale_prompt_in_service(caplog): + api = _BlockingPromptsApi() + api.fail_refresh = True + client = LangfuseApiClient(api, prompt_cache_ttl_seconds=0) # pyright: ignore[reportArgumentType] # duck-typed prompts API + + first = client.get_prompt("greeting") + with caplog.at_level(logging.WARNING, logger="LiteLLM"): + started = monotonic() + assert client.get_prompt("greeting") is first + assert monotonic() - started < 1.0, "the stale read waited on the blocked refresh" + api.release.set() + _wait_until(lambda: "refresh failed" in caplog.text) + assert client.get_prompt("greeting") is first + + +def test_only_one_refresh_runs_for_a_stale_prompt_under_concurrent_reads(): + api = _BlockingPromptsApi() + client = LangfuseApiClient(api, prompt_cache_ttl_seconds=0.3) # pyright: ignore[reportArgumentType] # duck-typed prompts API + + first = client.get_prompt("greeting") + sleep(0.3) + for _ in range(20): + assert client.get_prompt("greeting") is first + _wait_until(lambda: len(api.requests) == 2) + api.release.set() + _wait_until(lambda: client.get_prompt("greeting") is not first) + assert len(api.requests) == 2 + + +def test_prompt_cache_keeps_a_missing_label_apart_from_the_label_named_none(): + """A prompt labelled ``"None"`` and the unlabelled default are different prompts in Langfuse + and must not answer each other's requests from the cache.""" + api = _RecordingPromptsApi() + client = LangfuseApiClient(api, prompt_cache_ttl_seconds=60) # pyright: ignore[reportArgumentType] # duck-typed prompts API + + unlabelled = client.get_prompt("greeting") + named_none = client.get_prompt("greeting", label="None") + cached_unlabelled = client.get_prompt("greeting") + + assert unlabelled.prompt == "label=None" + assert named_none.prompt == "label='None'" + assert cached_unlabelled is unlabelled + assert api.requests == [("greeting", None, None), ("greeting", None, "None")] diff --git a/tests/test_litellm/integrations/test_langfuse.py b/tests/test_litellm/integrations/test_langfuse.py index 37860ae8445..3e6e130cac5 100644 --- a/tests/test_litellm/integrations/test_langfuse.py +++ b/tests/test_litellm/integrations/test_langfuse.py @@ -1,6 +1,8 @@ import datetime import json -import sys +import logging +import threading +import time import types import unittest from typing import Final, Optional @@ -11,6 +13,7 @@ import pytest import litellm from litellm.integrations.langfuse import langfuse as langfuse_module from litellm.integrations.langfuse.langfuse import LangFuseLogger +from litellm.integrations.langfuse.langfuse_sdk import resolve_trace_id # Import LangfuseUsageDetails directly from the module where it's defined @@ -33,58 +36,20 @@ class TestLangfuseUsageDetails(unittest.TestCase): ) self.env_patcher.start() - # Create mock objects - self.mock_langfuse_client = MagicMock() - # Mock the client attribute to prevent errors during logger initialization - self.mock_langfuse_client.client = MagicMock() - self.mock_langfuse_trace = MagicMock() - self.mock_langfuse_generation = MagicMock() - self.mock_langfuse_generation.trace_id = "test-trace-id" - - # Mock span method for trace (used by log_provider_specific_information_as_span and _log_guardrail_information_as_span) - self.mock_langfuse_span = MagicMock() - self.mock_langfuse_span.end = MagicMock() - self.mock_langfuse_trace.span.return_value = self.mock_langfuse_span - - # Setup the trace and generation chain - self.mock_langfuse_trace.generation.return_value = self.mock_langfuse_generation - self.last_trace_kwargs = {} - - def _trace_side_effect(*args, **kwargs): - self.last_trace_kwargs = kwargs - return self.mock_langfuse_trace - - self.mock_langfuse_client.trace.side_effect = _trace_side_effect - - # Mock the langfuse module that's imported locally in methods - self.langfuse_module_patcher = patch.dict( - "sys.modules", {"langfuse": MagicMock()} - ) - self.mock_langfuse_module = self.langfuse_module_patcher.start() - - # Create a mock for the langfuse module with version - self.mock_langfuse = MagicMock() - self.mock_langfuse.version = MagicMock() - self.mock_langfuse.version.__version__ = ( - "3.0.0" # Set a version that supports all features + from opentelemetry.sdk.trace import TracerProvider + from opentelemetry.sdk.trace.export import SimpleSpanProcessor + from opentelemetry.sdk.trace.export.in_memory_span_exporter import ( + InMemorySpanExporter, ) - # Mock the Langfuse class - self.mock_langfuse_class = MagicMock() - self.mock_langfuse_class.return_value = self.mock_langfuse_client + self.span_exporter = InMemorySpanExporter() + self.real_provider = TracerProvider() + self.real_provider.add_span_processor(SimpleSpanProcessor(self.span_exporter)) - # Set up the sys.modules['langfuse'] mock - sys.modules["langfuse"] = self.mock_langfuse - sys.modules["langfuse"].Langfuse = self.mock_langfuse_class - - # Create a fresh logger instance for each test + # the host above is unreachable, so the REST client is cheap to build + # and each test swaps in the export channel it wants self.logger = LangFuseLogger() - # Explicitly set the Langfuse client to our mock - self.logger.Langfuse = self.mock_langfuse_client - # Ensure langfuse_sdk_version is set correctly for _supports_* methods - self.logger.langfuse_sdk_version = "3.0.0" - # Add the log_event_on_langfuse method to the instance def log_event_on_langfuse( self, @@ -113,23 +78,46 @@ class TestLangfuseUsageDetails(unittest.TestCase): ) # Bind the method to the instance - self.logger.log_event_on_langfuse = types.MethodType( - log_event_on_langfuse, self.logger - ) + self.logger.log_event_on_langfuse = types.MethodType(log_event_on_langfuse, self.logger) def tearDown(self): # Clean up logger instance to prevent state leakage if hasattr(self, "logger"): - # Reset logger's Langfuse client to break any references - self.logger.Langfuse = None - # Delete logger instance to ensure complete cleanup del self.logger # Restore global Langfuse client counter to prevent cross-test pollution litellm.initialized_langfuse_clients = self._original_langfuse_clients_count self.env_patcher.stop() - self.langfuse_module_patcher.stop() # patch.dict automatically restores sys.modules + + def use_real_langfuse_client(self): + """Point the logger at an export channel whose spans land in memory.""" + from opentelemetry.sdk.trace.export.in_memory_span_exporter import ( + InMemorySpanExporter, + ) + + from litellm.integrations.langfuse.langfuse_sdk import build_langfuse_tracing + + self.span_exporter = InMemorySpanExporter() + self.logger.tracing = build_langfuse_tracing( + exporter=self.span_exporter, + environment=None, + release=None, + sample_rate=1.0, + flush_interval_millis=10, + ) + self.real_provider = self.logger.tracing.provider + return self.logger.tracing + + def exported_generation(self): + self.logger.tracing.flush() + spans = [s for s in self.span_exporter.get_finished_spans()] + assert spans, "no spans were exported" + return spans[-1] + + @staticmethod + def span_trace_id(span): + return format(span.context.trace_id, "032x") def test_langfuse_usage_details_type(self): """Test that LangfuseUsageDetails TypedDict is properly defined with the correct fields""" @@ -260,21 +248,7 @@ class TestLangfuseUsageDetails(unittest.TestCase): Test that _log_langfuse_v2 correctly handles None values in the usage object by converting them to 0, preventing validation errors. """ - # Reset the mock to ensure clean state; clear side_effect so return_value takes effect - self.mock_langfuse_client.reset_mock(side_effect=True) - self.mock_langfuse_trace.reset_mock(side_effect=True) - self.mock_langfuse_generation.reset_mock(side_effect=True) - - # Re-setup the trace and generation chain with clean state - self.mock_langfuse_generation.trace_id = "test-trace-id" - mock_span = MagicMock() - mock_span.end = MagicMock() - self.mock_langfuse_trace.span.return_value = mock_span - self.mock_langfuse_trace.generation.return_value = self.mock_langfuse_generation - - # Ensure trace returns our mock - self.mock_langfuse_client.trace.return_value = self.mock_langfuse_trace - self.logger.Langfuse = self.mock_langfuse_client + self.use_real_langfuse_client() with ( patch( @@ -282,7 +256,6 @@ class TestLangfuseUsageDetails(unittest.TestCase): side_effect=lambda generation_params, **kwargs: generation_params, create=True, ) as mock_add_prompt_params, - patch.object(self.logger, "_supports_prompt", return_value=True), ): # Create a mock response object with usage information containing None values response_obj = MagicMock() @@ -332,29 +305,12 @@ class TestLangfuseUsageDetails(unittest.TestCase): except Exception as e: self.fail(f"_log_langfuse_v2 raised an exception: {e}") - # Verify that trace was called first - self.mock_langfuse_client.trace.assert_called() - - # Check the arguments passed to the mocked langfuse generation call - self.mock_langfuse_trace.generation.assert_called_once() - call_args, call_kwargs = self.mock_langfuse_trace.generation.call_args - - # Inspect the usage and usage_details dictionaries - usage_arg = call_kwargs.get("usage") - usage_details_arg = call_kwargs.get("usage_details") - - self.assertIsNotNone(usage_arg) - self.assertIsNotNone(usage_details_arg) - - # Verify that None values were converted to 0 - self.assertEqual(usage_arg["prompt_tokens"], 0) - self.assertEqual(usage_arg["completion_tokens"], 0) - - self.assertEqual(usage_details_arg["input"], 0) - self.assertEqual(usage_details_arg["output"], 0) - self.assertEqual(usage_details_arg["total"], 0) - self.assertEqual(usage_details_arg["cache_creation_input_tokens"], 0) - self.assertEqual(usage_details_arg["cache_read_input_tokens"], 0) + usage_details = json.loads(self.exported_generation().attributes["langfuse.observation.usage_details"]) + assert usage_details["input"] == 0 + assert usage_details["output"] == 0 + assert usage_details["total"] == 0 + assert usage_details["cache_creation_input_tokens"] == 0 + assert usage_details["cache_read_input_tokens"] == 0 mock_add_prompt_params.assert_called_once() @@ -407,7 +363,7 @@ class TestLangfuseUsageDetails(unittest.TestCase): def test_log_langfuse_v2_uses_standard_trace_id_when_available(self): payload = self._build_standard_logging_payload(trace_id="std-trace-id") kwargs = self._build_langfuse_kwargs(payload) - self.last_trace_kwargs = {} + self.use_real_langfuse_client() with patch( "litellm.integrations.langfuse.langfuse._add_prompt_to_generation_params", @@ -429,12 +385,12 @@ class TestLangfuseUsageDetails(unittest.TestCase): litellm_call_id="call-id-xyz", ) - assert self.last_trace_kwargs.get("id") == "std-trace-id" + assert self.span_trace_id(self.exported_generation()) == resolve_trace_id("std-trace-id") def test_log_langfuse_v2_defaults_to_call_id_without_standard_trace_id(self): payload = self._build_standard_logging_payload() kwargs = self._build_langfuse_kwargs(payload) - self.last_trace_kwargs = {} + self.use_real_langfuse_client() with patch( "litellm.integrations.langfuse.langfuse._add_prompt_to_generation_params", @@ -456,7 +412,7 @@ class TestLangfuseUsageDetails(unittest.TestCase): litellm_call_id="call-id-xyz", ) - assert self.last_trace_kwargs.get("id") == "call-id-xyz" + assert self.span_trace_id(self.exported_generation()) == resolve_trace_id("call-id-xyz") def test_log_langfuse_v2_uses_litellm_trace_id_fallback_over_call_id(self): """ @@ -468,7 +424,7 @@ class TestLangfuseUsageDetails(unittest.TestCase): payload = self._build_standard_logging_payload() # no trace_id kwargs = self._build_langfuse_kwargs(payload) kwargs["litellm_trace_id"] = "trace-id-from-kwargs" - self.last_trace_kwargs = {} + self.use_real_langfuse_client() with patch( "litellm.integrations.langfuse.langfuse._add_prompt_to_generation_params", @@ -491,7 +447,7 @@ class TestLangfuseUsageDetails(unittest.TestCase): ) # litellm_trace_id should be preferred over litellm_call_id - assert self.last_trace_kwargs.get("id") == "trace-id-from-kwargs" + assert self.span_trace_id(self.exported_generation()) == resolve_trace_id("trace-id-from-kwargs") CANARY = "sk-lf-canary-SECRET-d4e5f6" @@ -521,24 +477,51 @@ class TestLangfuseUsageDetails(unittest.TestCase): } def _emitted_payload_text(self): - """Every blob this logger handed to the langfuse SDK, as one searchable string.""" + """Every attribute this logger exported to langfuse, as one searchable string.""" import json - blobs = [self.last_trace_kwargs] - if self.mock_langfuse_trace.generation.call_args is not None: - blobs.append(self.mock_langfuse_trace.generation.call_args.kwargs) - blobs.extend(call.kwargs for call in self.mock_langfuse_trace.span.call_args_list) - return json.dumps(blobs, default=repr) + self.logger.tracing.flush() + return json.dumps( + [dict(span.attributes or {}) for span in self.span_exporter.get_finished_spans()], + default=repr, + ) - def _drive_with_canary(self, extra_metadata=None, hidden_params=None): + def exported_generation_metadata(self): + """The generation's metadata as langfuse receives it, one attribute per key. + + v4 serializes each value onto the span, so they are decoded back here to + keep these assertions about what litellm emitted rather than about the + SDK's wire encoding. + """ + import json + + prefix = "langfuse.observation.metadata." + + def decoded(raw): + try: + return json.loads(raw) + except (TypeError, ValueError): + return raw + + return { + key[len(prefix) :]: decoded(value) + for key, value in (self.exported_generation().attributes or {}).items() + if key.startswith(prefix) + } + + def exported_spans_named(self, name): + self.logger.tracing.flush() + return [span for span in self.span_exporter.get_finished_spans() if span.name == name] + + def _drive_with_canary(self, extra_metadata=None, hidden_params=None, guardrail_information=None): metadata = {**self._canary_request_metadata(), **(extra_metadata or {})} payload = self._build_standard_logging_payload(trace_id="canary-trace-id") if hidden_params is not None: payload["hidden_params"] = hidden_params + if guardrail_information is not None: + payload["guardrail_information"] = guardrail_information kwargs = {**self._build_langfuse_kwargs(payload), "response_cost": 0.25} - self.last_trace_kwargs = {} - self.mock_langfuse_trace.generation.reset_mock() - self.mock_langfuse_trace.span.reset_mock() + self.use_real_langfuse_client() with patch( "litellm.integrations.langfuse.langfuse._add_prompt_to_generation_params", @@ -559,7 +542,7 @@ class TestLangfuseUsageDetails(unittest.TestCase): level="INFO", litellm_call_id="canary-call-id", ) - return self.mock_langfuse_trace.generation.call_args.kwargs["metadata"] + return self.exported_generation_metadata() def test_team_callback_credentials_never_reach_langfuse(self): """ @@ -583,10 +566,13 @@ class TestLangfuseUsageDetails(unittest.TestCase): debug_langfuse dumps request metadata into the trace as a second emit site. It must be sourced from the allowlisted payload too. """ - self._drive_with_canary(extra_metadata={"debug_langfuse": True}) + import json + + self._drive_with_canary(extra_metadata={"debug_langfuse": True}) + dumped = json.loads(self.exported_generation().attributes["langfuse.trace.metadata.metadata_passed_to_litellm"]) - dumped = self.last_trace_kwargs["metadata"]["metadata_passed_to_litellm"] assert "user_api_key_auth" not in dumped + assert dumped["first_custom"] == "keep-first" assert self.CANARY not in self._emitted_payload_text() def test_raw_request_metadata_reaches_the_emitted_blob_through_no_key(self): @@ -610,18 +596,68 @@ class TestLangfuseUsageDetails(unittest.TestCase): """ self._drive_with_canary(hidden_params={"vertex_ai_grounding_metadata": ["ground-a", "ground-b"]}) - span_inputs = [call.kwargs.get("input") for call in self.mock_langfuse_trace.span.call_args_list] + span_inputs = [ + span.attributes.get("langfuse.observation.input") + for span in self.exported_spans_named("vertex_ai_grounding_metadata") + ] assert span_inputs == ["ground-a", "ground-b"] assert self.CANARY not in self._emitted_payload_text() + def test_only_the_generation_claims_the_trace_root(self): + """ + Langfuse derives trace name and I/O from the root observation, and with several + roots the one with the latest start wins. A post-call guardrail starts after the + model call, so it must nest under the generation instead of being a root itself, or + the trace shows the guardrail's request instead of the model's. + """ + self._drive_with_canary( + hidden_params={"vertex_ai_grounding_metadata": ["ground-a"]}, + guardrail_information=[ + { + "guardrail_name": "pii-post", + "guardrail_mode": "post_call", + "guardrail_request": {"texts": ["post-call scan"]}, + "guardrail_response": {"flagged": False}, + "start_time": 1704110402.0, + "end_time": 1704110403.0, + } + ], + ) + + [generation] = [span for span in self.span_exporter.get_finished_spans() if span.name.startswith("litellm-")] + [guardrail] = self.exported_spans_named("guardrail") + [grounding] = self.exported_spans_named("vertex_ai_grounding_metadata") + assert generation.parent is None + assert generation.attributes["langfuse.trace.name"] == "canary-trace" + for child in (guardrail, grounding): + assert child.parent.span_id == generation.context.span_id + assert child.context.trace_id == generation.context.trace_id + assert "langfuse.trace.name" not in child.attributes + + def test_generation_is_exported_when_a_child_span_fails(self): + """v2 buffered the generation in one call, so a bad guardrail entry could not lose it; + the OTel generation is open until ``end()`` and must still be ended when a child raises.""" + self._drive_with_canary( + guardrail_information=[ + { + "guardrail_name": "pii-post", + "guardrail_mode": "post_call", + "start_time": "not-a-timestamp", + "end_time": 1704110403.0, + } + ], + ) + + [generation] = [span for span in self.span_exporter.get_finished_spans() if span.name.startswith("litellm-")] + assert generation.attributes["langfuse.trace.name"] == "canary-trace" + assert self.exported_spans_named("guardrail") == [] + def test_caller_cannot_spoof_an_allowlisted_identity_field(self): """ Request metadata never reaches the blob, so a caller naming user_api_key_alias cannot have their value emitted in place of the proxy-resolved one. """ - generation_metadata = self._drive_with_canary( - extra_metadata={"user_api_key_alias": "spoofed-by-caller"} - ) + generation_metadata = self._drive_with_canary(extra_metadata={"user_api_key_alias": "spoofed-by-caller"}) assert generation_metadata["user_api_key_alias"] == "canary-alias" @@ -636,7 +672,7 @@ class TestLangfuseUsageDetails(unittest.TestCase): payload["metadata"]["requester_metadata"] = {"litellm_response_cost": "caller-value", "api_base": "caller"} kwargs = {**self._build_langfuse_kwargs(payload), "response_cost": 0.25} metadata = self._canary_request_metadata() - self.mock_langfuse_trace.generation.reset_mock() + self.use_real_langfuse_client() with patch( "litellm.integrations.langfuse.langfuse._add_prompt_to_generation_params", @@ -658,10 +694,48 @@ class TestLangfuseUsageDetails(unittest.TestCase): litellm_call_id="canary-call-id", ) - generation_metadata = self.mock_langfuse_trace.generation.call_args.kwargs["metadata"] + generation_metadata = self.exported_generation_metadata() assert generation_metadata["litellm_response_cost"] == 0.25 assert generation_metadata["api_base"] == "https://real-api-base" + def test_generation_metadata_carries_the_call_id_and_response_id(self): + """ + v2's generation id was ``time-_``, so a generation could + be found from the provider response id. v4 hashes that string onto 16 hex chars, + which leaves nothing searchable unless both ids are emitted as metadata. + """ + payload = self._build_standard_logging_payload(trace_id="canary-trace-id") + kwargs = {**self._build_langfuse_kwargs(payload), "response_cost": 0.25} + metadata = self._canary_request_metadata() + self.use_real_langfuse_client() + + with patch( + "litellm.integrations.langfuse.langfuse._add_prompt_to_generation_params", + side_effect=lambda generation_params, **kw: generation_params, + create=True, + ): + self.logger._log_langfuse_v2( + user_id="user-1", + metadata=metadata, + litellm_params={"metadata": metadata}, + output=None, + start_time=datetime.datetime(2024, 1, 1, 12, 0, 0), + end_time=datetime.datetime(2024, 1, 1, 12, 0, 1), + kwargs=kwargs, + optional_params={}, + input=None, + response_obj=litellm.ModelResponse( + id="chatcmpl-canary-response", choices=[{"message": {"role": "assistant", "content": "OK"}}] + ), + level="DEFAULT", + litellm_call_id="canary-call-id", + ) + + generation_metadata = self.exported_generation_metadata() + assert generation_metadata["litellm_call_id"] == "canary-call-id" + assert generation_metadata["response_id"] == "chatcmpl-canary-response" + assert "chatcmpl-canary-response" in self._emitted_payload_text() + def test_denied_steering_keys_and_enrichments(self): """ endpoint is a plain string, so without the deny-list it would ride the @@ -726,8 +800,9 @@ class TestLangfuseUsageDetails(unittest.TestCase): """ self._drive_with_canary() - assert self.last_trace_kwargs.get("session_id") == "canary-session" - assert self.last_trace_kwargs.get("name") == "canary-trace" + generation = self.exported_generation() + assert generation.attributes["session.id"] == "canary-session" + assert generation.attributes["langfuse.trace.name"] == "canary-trace" def test_failure_trace_survives_a_missing_standard_logging_object(self): """ @@ -746,8 +821,7 @@ class TestLangfuseUsageDetails(unittest.TestCase): "messages": [], "litellm_trace_id": "trace-id-failure", } - self.last_trace_kwargs = {} - self.mock_langfuse_trace.generation.reset_mock() + self.use_real_langfuse_client() with patch( "litellm.integrations.langfuse.langfuse._add_prompt_to_generation_params", @@ -771,9 +845,12 @@ class TestLangfuseUsageDetails(unittest.TestCase): import json - assert trace_id == "trace-id-failure" - assert self.last_trace_kwargs.get("id") == "trace-id-failure" - generation_metadata = self.mock_langfuse_trace.generation.call_args.kwargs["metadata"] + # Must use litellm_trace_id, not litellm_call_id. v4 addresses a trace by a + # 32-hex id, so the callback returns the resolved form, which is what makes + # the alerting deep link point at a trace langfuse can actually open + assert trace_id == resolve_trace_id("trace-id-failure") + assert self.span_trace_id(self.exported_generation()) == trace_id + generation_metadata = self.exported_generation_metadata() assert "user_api_key_auth" not in generation_metadata assert self.CANARY not in self._emitted_payload_text() assert "first_custom" not in generation_metadata @@ -790,7 +867,7 @@ class TestLangfuseUsageDetails(unittest.TestCase): """ payload = self._build_standard_logging_payload(trace_id="std-trace-123") kwargs = self._build_langfuse_kwargs(payload) - self.last_trace_kwargs = {} + self.use_real_langfuse_client() with patch( "litellm.integrations.langfuse.langfuse._add_prompt_to_generation_params", @@ -813,9 +890,9 @@ class TestLangfuseUsageDetails(unittest.TestCase): ) # session_id should be set for Langfuse session grouping - assert self.last_trace_kwargs.get("session_id") == "my-session-abc" + assert self.exported_generation().attributes["session.id"] == "my-session-abc" # trace_id should remain the standard trace_id, NOT the session_id - assert self.last_trace_kwargs.get("id") == "std-trace-123" + assert self.span_trace_id(self.exported_generation()) == resolve_trace_id("std-trace-123") def test_log_langfuse_v2_session_id_preserved_for_error_level(self): """ @@ -825,7 +902,7 @@ class TestLangfuseUsageDetails(unittest.TestCase): """ payload = self._build_standard_logging_payload(trace_id="std-trace-err") kwargs = self._build_langfuse_kwargs(payload) - self.last_trace_kwargs = {} + self.use_real_langfuse_client() with patch( "litellm.integrations.langfuse.langfuse._add_prompt_to_generation_params", @@ -848,11 +925,11 @@ class TestLangfuseUsageDetails(unittest.TestCase): ) # session_id must be preserved even for ERROR level logs - assert self.last_trace_kwargs.get("session_id") == "error-session-xyz" + assert self.exported_generation().attributes["session.id"] == "error-session-xyz" # trace_id should be the standard trace_id, not the session_id - assert self.last_trace_kwargs.get("id") == "std-trace-err" + assert self.span_trace_id(self.exported_generation()) == resolve_trace_id("std-trace-err") # status_message should be set for error traces - assert self.last_trace_kwargs.get("status_message") is not None + assert self.exported_generation().attributes["langfuse.observation.level"] == "ERROR" def test_log_langfuse_v2_explicit_trace_id_takes_priority_over_session_id(self): """ @@ -861,7 +938,7 @@ class TestLangfuseUsageDetails(unittest.TestCase): """ payload = self._build_standard_logging_payload() kwargs = self._build_langfuse_kwargs(payload) - self.last_trace_kwargs = {} + self.use_real_langfuse_client() with patch( "litellm.integrations.langfuse.langfuse._add_prompt_to_generation_params", @@ -892,9 +969,9 @@ class TestLangfuseUsageDetails(unittest.TestCase): ) # Explicit trace_id must take priority - assert self.last_trace_kwargs.get("id") == "explicit-trace-id-777" + assert self.span_trace_id(self.exported_generation()) == resolve_trace_id("explicit-trace-id-777") # session_id must still be set for session grouping - assert self.last_trace_kwargs.get("session_id") == "session-999" + assert self.exported_generation().attributes["session.id"] == "session-999" def test_failure_handler_langfuse_kwargs_excludes_original_response(): @@ -942,12 +1019,10 @@ def test_failure_handler_langfuse_kwargs_excludes_original_response(): try: # Mock LangFuseHandler to return our capturing mock logger - with patch( - "litellm.litellm_core_utils.litellm_logging.LangFuseHandler" - ) as mock_handler_class: - mock_handler_class.get_langfuse_logger_for_request.return_value = ( - mock_langfuse_logger - ) + with ( + patch("litellm.litellm_core_utils.litellm_logging.LangFuseHandler") as mock_handler_class + ): # test-quality-ok: route the request to the capturing logger; the real handler builds live clients + mock_handler_class.get_langfuse_logger_for_request.return_value = mock_langfuse_logger # Call the actual failure_handler test_exception = Exception("TestError: model not found") @@ -959,23 +1034,19 @@ def test_failure_handler_langfuse_kwargs_excludes_original_response(): ) # Verify log_event_on_langfuse was actually called - assert ( - mock_langfuse_logger.log_event_on_langfuse.called - ), "log_event_on_langfuse was not called" + assert mock_langfuse_logger.log_event_on_langfuse.called, "log_event_on_langfuse was not called" # Verify original_response is NOT in the kwargs passed to Langfuse langfuse_kwargs = captured_kwargs.get("kwargs", {}) - assert ( - "original_response" not in langfuse_kwargs - ), "original_response should be excluded from kwargs passed to Langfuse" + assert "original_response" not in langfuse_kwargs, ( + "original_response should be excluded from kwargs passed to Langfuse" + ) # Verify session_id metadata is preserved in the kwargs - langfuse_metadata = langfuse_kwargs.get("litellm_params", {}).get( - "metadata", {} + langfuse_metadata = langfuse_kwargs.get("litellm_params", {}).get("metadata", {}) + assert langfuse_metadata.get("session_id") == "test-session-failure", ( + "session_id should be preserved in kwargs passed to Langfuse" ) - assert ( - langfuse_metadata.get("session_id") == "test-session-failure" - ), "session_id should be preserved in kwargs passed to Langfuse" # Verify level is ERROR assert captured_kwargs.get("level") == "ERROR" @@ -1017,9 +1088,9 @@ async def test_async_log_failure_event_logs_to_langfuse(): "generation_id": "mock-gen", } - with patch( - "litellm.integrations.langfuse.langfuse_prompt_management.LangFuseHandler" - ) as mock_handler: + with ( + patch("litellm.integrations.langfuse.langfuse_prompt_management.LangFuseHandler") as mock_handler + ): # test-quality-ok: route the request to the capturing logger; the real handler builds live clients mock_handler.get_langfuse_logger_for_request.return_value = mock_logger kwargs = { @@ -1044,9 +1115,7 @@ async def test_async_log_failure_event_logs_to_langfuse(): ) # Verify log_event_on_langfuse was called - assert ( - mock_logger.log_event_on_langfuse.called - ), "log_event_on_langfuse was not called for failure event" + assert mock_logger.log_event_on_langfuse.called, "log_event_on_langfuse was not called for failure event" call_kwargs = mock_logger.log_event_on_langfuse.call_args[1] assert call_kwargs["level"] == "ERROR" assert call_kwargs["status_message"] == "API error: model not found" @@ -1086,9 +1155,9 @@ async def test_async_log_failure_event_works_without_standard_logging_object(): "generation_id": "mock-gen", } - with patch( - "litellm.integrations.langfuse.langfuse_prompt_management.LangFuseHandler" - ) as mock_handler: + with ( + patch("litellm.integrations.langfuse.langfuse_prompt_management.LangFuseHandler") as mock_handler + ): # test-quality-ok: route the request to the capturing logger; the real handler builds live clients mock_handler.get_langfuse_logger_for_request.return_value = mock_logger kwargs = { @@ -1119,6 +1188,77 @@ async def test_async_log_failure_event_works_without_standard_logging_object(): assert "InternalServerError" in call_kwargs["status_message"] +class _OtlpReceiver: + """A local HTTP server that records the paths of every POST it gets, standing in for Langfuse.""" + + def __init__(self) -> None: + from http.server import BaseHTTPRequestHandler, HTTPServer + + self.received: list[str] = [] + received = self.received + + class _Handler(BaseHTTPRequestHandler): + def do_POST(self): + received.append(self.path) + self.rfile.read(int(self.headers.get("Content-Length") or 0)) + self.send_response(200) + self.send_header("Content-Length", "0") + self.end_headers() + + def log_message(self, *args): + pass + + self.server = HTTPServer(("127.0.0.1", 0), _Handler) + threading.Thread(target=self.server.serve_forever, daemon=True).start() + + @property + def url(self) -> str: + return f"http://127.0.0.1:{self.server.server_port}" + + def close(self) -> None: + self.server.shutdown() + + +def _log_one_completion(logger: LangFuseLogger) -> None: + now = datetime.datetime.now() + logger.log_event_on_langfuse( + kwargs={ + "call_type": "completion", + "litellm_params": {"metadata": {}, "proxy_server_request": {"headers": {}}}, + "messages": [{"role": "user", "content": "hi"}], + "optional_params": {}, + }, + response_obj=litellm.ModelResponse(choices=[{"message": {"role": "assistant", "content": "yo"}}]), + start_time=now, + end_time=now, + ) + logger.flush() + + +def test_mock_mode_makes_no_network_calls(monkeypatch): + """LANGFUSE_MOCK promises full execution without egress. + + The mock intercepts httpx, but v4 ships observations over its own OTLP + exporter, so nothing stops a real request to the configured host without an + exporter that drops them. + """ + receiver = _OtlpReceiver() + monkeypatch.setenv("LANGFUSE_MOCK", "true") + monkeypatch.setenv("LANGFUSE_HOST", receiver.url) + monkeypatch.setenv("LANGFUSE_PUBLIC_KEY", "pk-mock-egress") + monkeypatch.setenv("LANGFUSE_SECRET_KEY", "sk-mock-egress") + + try: + logger = LangFuseLogger() + assert logger.is_mock_mode is True + _log_one_completion(logger) + time.sleep(1) + finally: + receiver.close() + + assert receiver.received == [], f"mock mode sent real requests: {receiver.received}" + + def test_max_langfuse_clients_limit(): """ Test that the max langfuse clients limit is respected when initializing multiple clients @@ -1154,7 +1294,7 @@ def test_max_langfuse_clients_limit(): assert litellm.initialized_langfuse_clients == 2 # Third client should fail with exception - with pytest.raises(Exception, match='Max langfuse clients reached') as exc_info: + with pytest.raises(Exception, match="Max langfuse clients reached") as exc_info: logger3 = LangFuseLogger( langfuse_public_key="test_key_3", langfuse_secret="test_secret_3", @@ -1170,73 +1310,76 @@ def test_max_langfuse_clients_limit(): litellm.initialized_langfuse_clients = original_initialized_langfuse_clients -class _RecordingLangfuse: - last_parameters: Optional[dict] = None - - def __init__(self, environment=None, **parameters): - type(self).last_parameters = {"environment": environment, **parameters} - self.client = MagicMock() +_UNREACHABLE_HOST: Final = "http://127.0.0.1:1" -class _RecordingLangfuseWithoutEnvironment: - last_parameters: Optional[dict] = None - - def __init__(self, **parameters): - type(self).last_parameters = parameters - self.client = MagicMock() - - -def _build_langfuse_logger(monkeypatch) -> LangFuseLogger: +def _build_langfuse_logger(monkeypatch, **overrides) -> LangFuseLogger: monkeypatch.setenv("LANGFUSE_MOCK", "false") monkeypatch.setattr(litellm, "initialized_langfuse_clients", 0) - with patch("langfuse.Langfuse", _RecordingLangfuse): - return LangFuseLogger( - langfuse_public_key="pk-lit5228", - langfuse_secret="sk-lit5228", - langfuse_host="https://test.langfuse.com", - ) + return LangFuseLogger( + **{ + "langfuse_public_key": "pk-lit5228", + "langfuse_secret": "sk-lit5228", + "langfuse_host": _UNREACHABLE_HOST, + **overrides, + } + ) -def test_langfuse_environment_is_passed_to_sdk_client(monkeypatch): - monkeypatch.setenv("LANGFUSE_MOCK", "false") +def _exported_environment(logger: LangFuseLogger): + from langfuse import LangfuseOtelSpanAttributes + + return logger.tracing.provider.resource.attributes.get(LangfuseOtelSpanAttributes.ENVIRONMENT) + + +def test_langfuse_environment_lands_on_every_exported_span(monkeypatch): monkeypatch.delenv("LANGFUSE_TRACING_ENVIRONMENT", raising=False) - monkeypatch.setattr(litellm, "initialized_langfuse_clients", 0) - with patch("langfuse.Langfuse", _RecordingLangfuse): - logger = LangFuseLogger( - langfuse_public_key="pk-env", - langfuse_secret="sk-env", - langfuse_host="https://test.langfuse.com", - langfuse_environment="staging", - ) + logger = _build_langfuse_logger(monkeypatch, langfuse_public_key="pk-env", langfuse_environment="staging") assert logger.langfuse_environment == "staging" - assert _RecordingLangfuse.last_parameters["environment"] == "staging" + assert _exported_environment(logger) == "staging" def test_langfuse_environment_falls_back_to_deployment_env_var(monkeypatch): - monkeypatch.setenv("LANGFUSE_MOCK", "false") monkeypatch.setenv("LANGFUSE_TRACING_ENVIRONMENT", "deployment-wide") - monkeypatch.setattr(litellm, "initialized_langfuse_clients", 0) - with patch("langfuse.Langfuse", _RecordingLangfuse): - logger = LangFuseLogger( - langfuse_public_key="pk-env", - langfuse_secret="sk-env", - langfuse_host="https://test.langfuse.com", - ) + logger = _build_langfuse_logger(monkeypatch, langfuse_public_key="pk-env") assert logger.langfuse_environment == "deployment-wide" - assert _RecordingLangfuse.last_parameters["environment"] == "deployment-wide" + assert _exported_environment(logger) == "deployment-wide" -def test_langfuse_environment_omitted_for_old_sdk_versions(monkeypatch): - monkeypatch.setenv("LANGFUSE_MOCK", "false") - monkeypatch.setattr(litellm, "initialized_langfuse_clients", 0) - with patch("langfuse.Langfuse", _RecordingLangfuseWithoutEnvironment): - LangFuseLogger( - langfuse_public_key="pk-env", - langfuse_secret="sk-env", - langfuse_host="https://test.langfuse.com", - langfuse_environment="staging", - ) - assert "environment" not in _RecordingLangfuseWithoutEnvironment.last_parameters +def _exported_release(logger: LangFuseLogger): + from langfuse import LangfuseOtelSpanAttributes + + return logger.tracing.provider.resource.attributes.get(LangfuseOtelSpanAttributes.RELEASE) + + +@pytest.mark.parametrize("platform_var", ["GITHUB_SHA", "CI_COMMIT_SHA", "RENDER_GIT_COMMIT", "SOURCE_VERSION"]) +def test_release_falls_back_to_the_deploy_platforms_commit_variable(monkeypatch, platform_var): + """Deployments that never set ``LANGFUSE_RELEASE`` still got a release on every trace from the v2 SDK, which + read the CI or hosting platform's commit variable; dropping that silently blanked their release filter.""" + from litellm.integrations.langfuse.langfuse_sdk import _COMMON_RELEASE_ENVS + + for name in ("LANGFUSE_RELEASE", *_COMMON_RELEASE_ENVS): + monkeypatch.delenv(name, raising=False) + monkeypatch.setenv(platform_var, "deadbeef") + logger = _build_langfuse_logger(monkeypatch, langfuse_public_key=f"pk-release-{platform_var}") + assert logger.langfuse_release == "deadbeef" + assert _exported_release(logger) == "deadbeef" + + +def test_explicit_langfuse_release_wins_over_the_platform_commit(monkeypatch): + monkeypatch.setenv("LANGFUSE_RELEASE", "v9") + monkeypatch.setenv("GITHUB_SHA", "deadbeef") + logger = _build_langfuse_logger(monkeypatch, langfuse_public_key="pk-release-explicit") + assert _exported_release(logger) == "v9" + + +def test_non_string_generation_name_is_exported_as_its_text(monkeypatch): + """v2 coerced ``generation_name`` through pydantic; a raw int would now fail OTLP encoding and lose the batch.""" + rig = _steering_logger() + + _, _, span = _emit(rig, metadata={"generation_name": 12345}) + + assert span.name == "12345" def test_dynamic_langfuse_environment_triggers_dynamic_logger(): @@ -1247,13 +1390,11 @@ def test_dynamic_langfuse_environment_triggers_dynamic_logger(): assert LangFuseHandler._dynamic_langfuse_credentials_are_passed(params) is True - config = LangFuseHandler.get_dynamic_langfuse_logging_config( - standard_callback_dynamic_params=params - ) + config = LangFuseHandler.get_dynamic_langfuse_logging_config(standard_callback_dynamic_params=params) assert config["langfuse_environment"] == "team-a-env" -def test_langfuse_sdk_client_survives_httpx_cache_eviction(monkeypatch): +def test_langfuse_rest_client_survives_httpx_cache_eviction(monkeypatch): import gc import weakref @@ -1263,21 +1404,20 @@ def test_langfuse_sdk_client_survives_httpx_cache_eviction(monkeypatch): monkeypatch.setattr(litellm, "in_memory_llm_clients_cache", LLMClientCache()) logger = _build_langfuse_logger(monkeypatch) - sdk_client = _RecordingLangfuse.last_parameters["httpx_client"] cached_handler = _get_httpx_client() handler_ref = weakref.ref(cached_handler) - assert sdk_client is logger.langfuse_client - assert sdk_client is cached_handler.client + assert logger.langfuse_client is cached_handler.client litellm.in_memory_llm_clients_cache = LLMClientCache() del cached_handler gc.collect() assert litellm.in_memory_llm_clients_cache.get_cache("httpx_client") is None - assert handler_ref() is not None, "logger must keep the handler that owns the client it handed the SDK" - assert not sdk_client.is_closed + assert handler_ref() is not None, "logger must keep the handler that owns the client behind its REST API" + assert not logger.langfuse_client.is_closed + assert logger.api_client.auth_check() is not None def test_langfuse_logger_reuses_the_shared_cached_client(monkeypatch): @@ -1301,20 +1441,100 @@ def test_langfuse_logger_reuses_the_shared_cached_client(monkeypatch): _LANGFUSE_REDACTED = "redacted-by-litellm" -def _steering_logger() -> LangFuseLogger: - """``__new__`` skips the SDK and network setup in ``__init__``.""" - logger = LangFuseLogger.__new__(LangFuseLogger) - logger.Langfuse = MagicMock() - logger.langfuse_sdk_version = "2.60.0" - return logger - - -def _emit(logger: LangFuseLogger, *, metadata=None, headers=None): - """``log_event_on_langfuse`` is the entry point that folds ``langfuse_*`` headers into metadata.""" - now = datetime.datetime.now() - response_obj = litellm.ModelResponse( - choices=[{"message": {"role": "assistant", "content": "the-output"}}] +def _steering_logger(): + """``__new__`` skips the network setup in ``__init__``; spans land in memory.""" + from opentelemetry.sdk.trace.export.in_memory_span_exporter import ( + InMemorySpanExporter, ) + + from litellm.integrations.langfuse.langfuse import installed_langfuse_version + from litellm.integrations.langfuse.langfuse_sdk import build_langfuse_client, build_langfuse_tracing + + exporter = InMemorySpanExporter() + logger = LangFuseLogger.__new__(LangFuseLogger) + logger.tracing = build_langfuse_tracing( + exporter=exporter, environment=None, release=None, sample_rate=1.0, flush_interval_millis=10 + ) + logger.api_client = build_langfuse_client( + public_key="pk-steering-test", secret_key="sk-steering-test", base_url=_UNREACHABLE_HOST, httpx_client=None + ) + logger.langfuse_sdk_version = installed_langfuse_version() + return logger, exporter + + +def test_log_event_keeps_exporting_after_the_dynamic_cache_evicts_the_logger(): + """Per-key loggers are evicted from ``DynamicLoggingCache`` while a callback may still hold them. + + v2 lost that callback's events to a shut-down client; the export channel is shared per + credential set and outlives any one logger, so the events still land. + """ + from litellm.litellm_core_utils.specialty_caches.dynamic_logging_cache import LangfuseInMemoryCache + + logger, exporter = _steering_logger() + cache = LangfuseInMemoryCache() + cache.set_cache("langfuse-evicted", logger) + litellm.initialized_langfuse_clients += 1 + before = litellm.initialized_langfuse_clients + cache._remove_key("langfuse-evicted") + + now = datetime.datetime.now() + returned = logger.log_event_on_langfuse( + kwargs={ + "call_type": "completion", + "litellm_params": {"metadata": {}}, + "messages": [{"role": "user", "content": "the-input"}], + "optional_params": {}, + }, + response_obj=litellm.ModelResponse(choices=[{"message": {"role": "assistant", "content": "the-output"}}]), + start_time=now, + end_time=now, + ) + + assert litellm.initialized_langfuse_clients == before - 1 + assert _span_trace_id(_exported_span(logger, exporter)) == returned["trace_id"] + + +def _exported_span(logger, exporter): + logger.flush() + return exporter.get_finished_spans()[-1] + + +_TRACE_FIELD_KEYS = { + "user.id": "user_id", + "session.id": "session_id", + "langfuse.version": "version", + "langfuse.release": "release", +} + + +def _trace_params(span): + """The trace-level fields of the exported span, keyed as v2's ``trace_params`` were.""" + prefix = "langfuse.trace." + attributes = span.attributes or {} + return { + **{ + key[len(prefix) :]: value + for key, value in attributes.items() + if key.startswith(prefix) and not key.startswith(prefix + "metadata.") + }, + **{name: attributes[key] for key, name in _TRACE_FIELD_KEYS.items() if key in attributes}, + } + + +def _span_trace_id(span): + return format(span.context.trace_id, "032x") + + +def _emit(rig, *, metadata=None, headers=None): + """``log_event_on_langfuse`` is the entry point that folds ``langfuse_*`` headers into metadata. + + Both the trace-level and the observation fields are read back off the span litellm exported. + """ + logger, exporter = rig + exporter.clear() + + now = datetime.datetime.now() + response_obj = litellm.ModelResponse(choices=[{"message": {"role": "assistant", "content": "the-output"}}]) logger.log_event_on_langfuse( kwargs={ "call_type": "completion", @@ -1329,10 +1549,14 @@ def _emit(logger: LangFuseLogger, *, metadata=None, headers=None): start_time=now, end_time=now, ) - return ( - logger.Langfuse.trace.call_args.kwargs, - logger.Langfuse.trace.return_value.generation.call_args.kwargs, - ) + prefix = "langfuse.observation." + span = _exported_span(logger, exporter) + generation_params = { + key[len(prefix) :]: value + for key, value in (span.attributes or {}).items() + if key.startswith(prefix) and not key.startswith(prefix + "metadata.") + } + return _trace_params(span), generation_params, span @pytest.mark.parametrize("level", ["DEFAULT", "ERROR"]) @@ -1458,8 +1682,9 @@ def test_session_header_trace_provenance(headers, metadata, expected_id, level): redact_credential_headers, ) - logger: Final = _steering_logger() + logger, exporter = _steering_logger() for turn in range(2): + exporter.clear() call_id = f"call-{turn}" request_headers = Headers(headers) data = LiteLLMProxyRequestSetup.add_litellm_metadata_from_request_headers( @@ -1489,17 +1714,19 @@ def test_session_header_trace_provenance(headers, metadata, expected_id, level): level=level, status_message="provider error" if level == "ERROR" else None, ) - trace_params = logger.Langfuse.trace.call_args.kwargs - assert trace_params["id"] == (call_id if expected_id == "call" else expected_id) - assert result["trace_id"] == trace_params["id"] + span = _exported_span(logger, exporter) + assert _span_trace_id(span) == resolve_trace_id(call_id if expected_id == "call" else expected_id) + assert result["trace_id"] == _span_trace_id(span) if expected_id != "existing-trace": - assert trace_params["session_id"] == headers.get("langfuse_session_id", original_metadata.get("session_id")) + assert span.attributes.get("session.id") == headers.get( + "langfuse_session_id", original_metadata.get("session_id") + ) steering = {key[len("langfuse_") :]: value for key, value in headers.items() if key.startswith("langfuse_")} assert data["metadata"] == {**original_metadata, **steering} def test_session_header_trace_without_call_id_keeps_session_alias(): - logger: Final = _steering_logger() + logger, exporter = _steering_logger() now: Final = datetime.datetime.now() result: Final = logger.log_event_on_langfuse( @@ -1518,8 +1745,8 @@ def test_session_header_trace_without_call_id_keeps_session_alias(): end_time=now, ) - assert logger.Langfuse.trace.call_args.kwargs["id"] == "session-7125" - assert result["trace_id"] == "session-7125" + assert _span_trace_id(_exported_span(logger, exporter)) == resolve_trace_id("session-7125") + assert result["trace_id"] == resolve_trace_id("session-7125") def test_every_proxy_session_header_shape_is_classified_as_a_session_alias(): @@ -1553,7 +1780,7 @@ def test_every_proxy_session_header_shape_is_classified_as_a_session_alias(): ) def test_sdk_caller_without_request_headers_keeps_its_trace(proxy_server_request): """A direct SDK caller has no request headers, so a session-shaped trace id stays the caller's.""" - logger: Final = _steering_logger() + logger, exporter = _steering_logger() now: Final = datetime.datetime.now() result: Final = logger.log_event_on_langfuse( @@ -1572,8 +1799,8 @@ def test_sdk_caller_without_request_headers_keeps_its_trace(proxy_server_request end_time=now, ) - assert logger.Langfuse.trace.call_args.kwargs["id"] == "session-7125" - assert result["trace_id"] == "session-7125" + assert _span_trace_id(_exported_span(logger, exporter)) == resolve_trace_id("session-7125") + assert result["trace_id"] == resolve_trace_id("session-7125") def test_session_header_classifier_survives_non_string_header_keys(): @@ -1587,38 +1814,38 @@ def test_session_header_classifier_survives_non_string_header_keys(): def test_mask_input_header_false_keeps_the_prompt(): - logger = _steering_logger() + rig = _steering_logger() - trace_params, generation_params = _emit(logger, headers={"langfuse_mask_input": "false"}) + trace_params, generation_params, _ = _emit(rig, headers={"langfuse_mask_input": "false"}) - assert trace_params["input"] == {"messages": [{"role": "user", "content": "the-input"}]} - assert generation_params["input"] == {"messages": [{"role": "user", "content": "the-input"}]} + assert "input" not in trace_params + assert json.loads(generation_params["input"]) == {"messages": [{"role": "user", "content": "the-input"}]} def test_mask_input_header_true_redacts_the_prompt(): - logger = _steering_logger() + rig = _steering_logger() - trace_params, generation_params = _emit(logger, headers={"langfuse_mask_input": "true"}) + trace_params, generation_params, _ = _emit(rig, headers={"langfuse_mask_input": "true"}) - assert trace_params["input"] == _LANGFUSE_REDACTED + assert "input" not in trace_params assert generation_params["input"] == _LANGFUSE_REDACTED def test_mask_output_header_false_keeps_the_completion(): - logger = _steering_logger() + rig = _steering_logger() - trace_params, generation_params = _emit(logger, headers={"langfuse_mask_output": "false"}) + trace_params, generation_params, _ = _emit(rig, headers={"langfuse_mask_output": "false"}) - assert trace_params["output"] != _LANGFUSE_REDACTED - assert generation_params["output"] != _LANGFUSE_REDACTED + assert "output" not in trace_params + assert "the-output" in generation_params["output"] def test_mask_output_header_true_redacts_the_completion(): - logger = _steering_logger() + rig = _steering_logger() - trace_params, generation_params = _emit(logger, headers={"langfuse_mask_output": "true"}) + trace_params, generation_params, _ = _emit(rig, headers={"langfuse_mask_output": "true"}) - assert trace_params["output"] == _LANGFUSE_REDACTED + assert "output" not in trace_params assert generation_params["output"] == _LANGFUSE_REDACTED @@ -1632,30 +1859,31 @@ def test_mask_output_header_true_redacts_the_completion(): ], ) def test_mask_input_from_the_request_body_is_unchanged(mask_input, expect_redacted): - logger = _steering_logger() + rig = _steering_logger() - trace_params, _ = _emit(logger, metadata={"mask_input": mask_input}) + _, generation_params, _ = _emit(rig, metadata={"mask_input": mask_input}) - assert (trace_params["input"] == _LANGFUSE_REDACTED) is expect_redacted + assert (generation_params["input"] == _LANGFUSE_REDACTED) is expect_redacted @pytest.mark.parametrize("flag", [True, "true"]) -def test_update_trace_keys_header_applies_every_key_when_enabled(flag): - logger = _steering_logger() +def test_update_trace_keys_header_applies_every_key_when_enabled(flag, monkeypatch): + rig = _steering_logger() - with patch.object(litellm, "langfuse_enable_update_trace_keys", flag): - trace_params, _ = _emit( - logger, - headers={ - "langfuse_existing_trace_id": "trace-1", - "langfuse_update_trace_keys": "trace_release, trace_tail", - "langfuse_trace_release": "v1.2.3", - "langfuse_trace_tail": "last", - }, - ) + monkeypatch.setattr(litellm, "langfuse_enable_update_trace_keys", flag) + trace_params, _, span = _emit( + rig, + headers={ + "langfuse_existing_trace_id": "trace-1", + "langfuse_update_trace_keys": "trace_release, trace_tail", + "langfuse_trace_release": "v1.2.3", + "langfuse_trace_tail": "last", + }, + ) assert trace_params["release"] == "v1.2.3" - assert trace_params["tail"] == "last" + assert span.attributes["langfuse.release"] == "v1.2.3" + assert not [key for key in span.attributes if key.endswith("tail")] def test_update_trace_keys_is_off_by_default(): @@ -1664,10 +1892,10 @@ def test_update_trace_keys_is_off_by_default(): user_api_key_auth and have the resolved auth object, including team callback credentials, serialized onto the trace. It stays inert until an operator opts in. """ - logger = _steering_logger() + rig = _steering_logger() - trace_params, _ = _emit( - logger, + trace_params, _, span = _emit( + rig, metadata={ "existing_trace_id": "trace-1", "update_trace_keys": ["user_api_key_auth", "trace_release"], @@ -1678,41 +1906,185 @@ def test_update_trace_keys_is_off_by_default(): assert "user_api_key_auth" not in trace_params assert "release" not in trace_params - assert "sk-canary" not in json.dumps(trace_params, default=repr) + assert "sk-canary" not in json.dumps(dict(span.attributes or {}), default=repr) -def test_update_trace_keys_input_and_output_are_gated_too(): - logger = _steering_logger() +def test_update_trace_keys_input_and_output_are_gated_too(monkeypatch): + rig = _steering_logger() - off, _ = _emit(logger, metadata={"existing_trace_id": "trace-1", "update_trace_keys": ["input", "output"]}) - with patch.object(litellm, "langfuse_enable_update_trace_keys", True): - on, _ = _emit(logger, metadata={"existing_trace_id": "trace-1", "update_trace_keys": ["input", "output"]}) + off, _, _ = _emit(rig, metadata={"existing_trace_id": "trace-1", "update_trace_keys": ["input", "output"]}) + monkeypatch.setattr(litellm, "langfuse_enable_update_trace_keys", True) + on, _, _ = _emit(rig, metadata={"existing_trace_id": "trace-1", "update_trace_keys": ["input", "output"]}) assert "input" not in off and "output" not in off assert "input" in on and "output" in on -def test_update_trace_keys_from_the_request_body_list_applies_when_enabled(): - logger = _steering_logger() +def test_update_trace_keys_input_output_reach_the_trace_even_under_a_parent(monkeypatch): + """With a real parent the generation is not the trace root, so trace-level + I/O must be stamped explicitly; v2 updated the trace object directly.""" + rig = _steering_logger() - with patch.object(litellm, "langfuse_enable_update_trace_keys", True): - trace_params, _ = _emit( - logger, - metadata={ - "existing_trace_id": "trace-1", - "update_trace_keys": ["trace_release"], - "trace_release": "v1.2.3", - }, - ) + monkeypatch.setattr(litellm, "langfuse_enable_update_trace_keys", True) + _, _, span = _emit( + rig, + metadata={ + "existing_trace_id": "trace-1", + "parent_observation_id": "b" * 16, + "update_trace_keys": ["input", "output"], + }, + ) + + assert "the-input" in str(span.attributes["langfuse.trace.input"]) + assert "the-output" in str(span.attributes["langfuse.trace.output"]) + + +def test_a_fresh_trace_under_a_callers_parent_still_carries_its_own_input_and_output(): + """Langfuse copies I/O onto a trace only from its root observation; a caller's ``parent_observation_id`` + makes the generation a child, so the trace-level fields v2 set on ``trace(...)`` must be stamped.""" + rig = _steering_logger() + + _, _, span = _emit(rig, metadata={"parent_observation_id": "0123456789abcdef"}) + + assert span.parent is not None + assert "the-input" in str(span.attributes["langfuse.trace.input"]) + assert "the-output" in str(span.attributes["langfuse.trace.output"]) + + +def test_a_failed_call_under_a_callers_parent_stamps_the_error_as_the_trace_output(): + """The ERROR branch used to write a trace-level ``status_message``, a field the v4 trace schema does not + have, and skip ``output``; the generation's parent is the caller's, so nothing else fills the trace.""" + logger, exporter = _steering_logger() + now = datetime.datetime.now() + + logger.log_event_on_langfuse( + kwargs={ + "call_type": "completion", + "litellm_params": {"metadata": {"parent_observation_id": "0123456789abcdef"}}, + "messages": [{"role": "user", "content": "the-input"}], + "optional_params": {}, + }, + response_obj=None, + start_time=now, + end_time=now, + level="ERROR", + status_message="provider said no", + ) + span = _exported_span(logger, exporter) + + assert span.parent is not None + assert "provider said no" in str(span.attributes["langfuse.trace.output"]) + assert span.attributes["langfuse.observation.status_message"] == "provider said no" + + +def test_a_fresh_trace_root_leaves_the_duplicate_io_to_langfuse(): + rig = _steering_logger() + + _, _, span = _emit(rig, metadata={"trace_id": "a" * 32}) + + assert span.parent is None + assert "langfuse.trace.input" not in (span.attributes or {}) + assert "the-input" in str(span.attributes["langfuse.observation.input"]) + + +def test_existing_trace_id_appends_without_claiming_trace_root(): + """Langfuse copies a root observation's name and I/O onto the trace, so a + continuation that claimed root would rename the trace after every request; + v2 only ever touched the keys in ``update_trace_keys``.""" + rig = _steering_logger() + + _, _, span = _emit(rig, metadata={"existing_trace_id": "trace-1", "trace_name": "second-call"}) + + assert span.parent is not None + assert "langfuse.trace.name" not in (span.attributes or {}) + + +def test_a_fresh_trace_still_claims_root_so_its_generation_names_it(): + rig = _steering_logger() + + _, _, span = _emit(rig, metadata={"trace_id": "a" * 32, "trace_name": "first-call"}) + + assert span.parent is None + assert span.attributes["langfuse.trace.name"] == "first-call" + + +def test_trace_io_is_not_stamped_when_update_trace_keys_does_not_ask(monkeypatch): + rig = _steering_logger() + + monkeypatch.setattr(litellm, "langfuse_enable_update_trace_keys", True) + _, _, span = _emit( + rig, + metadata={ + "existing_trace_id": "trace-1", + "parent_observation_id": "b" * 16, + "update_trace_keys": ["trace_release"], + }, + ) + + assert "langfuse.trace.input" not in (span.attributes or {}) + assert "langfuse.trace.output" not in (span.attributes or {}) + + +def test_update_trace_keys_from_the_request_body_list_applies_when_enabled(monkeypatch): + rig = _steering_logger() + + monkeypatch.setattr(litellm, "langfuse_enable_update_trace_keys", True) + trace_params, _, span = _emit( + rig, + metadata={ + "existing_trace_id": "trace-1", + "update_trace_keys": ["trace_release"], + "trace_release": "v1.2.3", + }, + ) assert trace_params["release"] == "v1.2.3" + assert span.attributes["langfuse.release"] == "v1.2.3" + + +def test_update_trace_keys_trace_metadata_reaches_the_trace_and_stays_off_the_generation(monkeypatch): + rig = _steering_logger() + + monkeypatch.setattr(litellm, "langfuse_enable_update_trace_keys", True) + _, _, span = _emit( + rig, + metadata={ + "existing_trace_id": "trace-1", + "parent_observation_id": "b" * 16, + "update_trace_keys": ["trace_metadata"], + "trace_metadata": {"step": 2, "note": "x" * 300}, + }, + ) + + assert span.attributes["langfuse.trace.metadata.step"] == 2 + assert span.attributes["langfuse.trace.metadata.note"] == "x" * 300 + assert "langfuse.observation.metadata.step" not in span.attributes + + +def test_non_mapping_trace_metadata_does_not_lose_the_event(): + """A caller who passes ``trace_metadata`` as a string still gets a generation, and the string is not spread.""" + rig = _steering_logger() + + trace_params, generation_params, span = _emit(rig, metadata={"trace_metadata": "just-a-note"}) + + assert json.loads(generation_params["output"])["content"] == "the-output" + assert trace_params["name"] == "litellm-completion" + assert not any(key.startswith("langfuse.trace.metadata.") for key in span.attributes or {}) + + +def test_trace_metadata_is_not_propagated_when_absent(): + rig = _steering_logger() + + _, _, span = _emit(rig, metadata={"trace_name": "plain"}) + + assert not any(key.startswith("langfuse.trace.metadata.") for key in span.attributes or {}) def test_update_trace_keys_matches_whole_keys_not_substrings(): - logger = _steering_logger() + rig = _steering_logger() - trace_params, _ = _emit( - logger, + trace_params, _, _ = _emit( + rig, headers={"langfuse_existing_trace_id": "trace-1", "langfuse_update_trace_keys": "my_input"}, ) @@ -1720,25 +2092,12 @@ def test_update_trace_keys_matches_whole_keys_not_substrings(): def test_langfuse_environment_is_coerced_and_validated(monkeypatch): - monkeypatch.setenv("LANGFUSE_MOCK", "false") monkeypatch.delenv("LANGFUSE_TRACING_ENVIRONMENT", raising=False) - monkeypatch.setattr(litellm, "initialized_langfuse_clients", 0) - with patch("langfuse.Langfuse", _RecordingLangfuse): - logger = LangFuseLogger( - langfuse_public_key="pk-env", - langfuse_secret="sk-env", - langfuse_host="https://test.langfuse.com", - langfuse_environment=123, # non-string: must coerce, not crash - ) + logger = _build_langfuse_logger(monkeypatch, langfuse_public_key="pk-env", langfuse_environment=123) assert logger.langfuse_environment == "123" with pytest.raises(ValueError, match="langfuse_environment"): - LangFuseLogger( - langfuse_public_key="pk-env", - langfuse_secret="sk-env", - langfuse_host="https://test.langfuse.com", - langfuse_environment="Production", - ) + _build_langfuse_logger(monkeypatch, langfuse_public_key="pk-env", langfuse_environment="Production") def test_langfuse_empty_environment_falls_back_and_is_not_dynamic(monkeypatch): @@ -1748,15 +2107,7 @@ def test_langfuse_empty_environment_falls_back_and_is_not_dynamic(monkeypatch): monkeypatch.setenv("LANGFUSE_TRACING_ENVIRONMENT", "production") # '' falls back to the deployment env var at init - monkeypatch.setenv("LANGFUSE_MOCK", "false") - monkeypatch.setattr(litellm, "initialized_langfuse_clients", 0) - with patch("langfuse.Langfuse", _RecordingLangfuse): - logger = LangFuseLogger( - langfuse_public_key="pk-env", - langfuse_secret="sk-env", - langfuse_host="https://test.langfuse.com", - langfuse_environment="", - ) + logger = _build_langfuse_logger(monkeypatch, langfuse_public_key="pk-env", langfuse_environment="") assert logger.langfuse_environment == "production" # env-only params that add nothing do not select a dynamic logger @@ -1799,3 +2150,322 @@ def test_langfuse_deployment_environment_fallback_never_raises(monkeypatch, env_ langfuse_host="https://test.langfuse.com", ) assert logger.langfuse_environment == expected + + +def test_continued_trace_keeps_the_generation_version(): + """v2 set ``version`` on the generation even when the trace was not being updated.""" + rig = _steering_logger() + + _, _, span = _emit(rig, metadata={"existing_trace_id": "b" * 32, "version": "gen-7"}) + + assert span.attributes["langfuse.version"] == "gen-7" + + +def test_new_trace_version_takes_precedence_over_the_generation_version(): + """v4 has one ``langfuse.version`` per span, so unlike v2's separate trace and generation fields only one + value can survive; ``trace_version`` wins, matching the v4 SDK, whose propagated attributes overwrite a span's own.""" + rig = _steering_logger() + + captured_trace_params, _, span = _emit(rig, metadata={"trace_version": "trace-1", "version": "gen-7"}) + + assert captured_trace_params["version"] == "trace-1" + assert span.attributes["langfuse.version"] == "trace-1" + + +def test_log_event_returns_the_v2_dict_shape_for_the_alerting_trace_id_cache(): + """litellm_logging only caches the langfuse trace id off a dict with a ``trace_id`` key. + + Slack alerting builds its trace URL from that cache, so a different return + shape silently breaks alert links. + """ + rig = _steering_logger() + logger, _ = rig + + returned = logger.log_event_on_langfuse( + kwargs={ + "call_type": "completion", + "litellm_params": {"metadata": {"trace_id": "c" * 32}}, + "messages": [{"role": "user", "content": "the-input"}], + "optional_params": {}, + }, + response_obj=litellm.ModelResponse(choices=[{"message": {"role": "assistant", "content": "the-output"}}]), + start_time=datetime.datetime.now(), + end_time=datetime.datetime.now(), + ) + + assert isinstance(returned, dict) + assert returned["trace_id"] == "c" * 32 + assert returned["generation_id"] + + +def test_parse_langfuse_debug_only_enables_on_true_strings(): + """v4 treats any truthy value as debug=on, so the raw env string "false" would enable debug.""" + assert langfuse_module.parse_langfuse_debug("true") is True + assert langfuse_module.parse_langfuse_debug("True") is True + assert langfuse_module.parse_langfuse_debug("1") is True + assert langfuse_module.parse_langfuse_debug("false") is False + assert langfuse_module.parse_langfuse_debug("False") is False + assert langfuse_module.parse_langfuse_debug("") is False + assert langfuse_module.parse_langfuse_debug(None) is False + + +@pytest.mark.parametrize( + ("raw", "expected"), + [(None, 1), ("", 1), ("3", 3), ("0", 1), ("-5", 1), ("abc", 1)], + ids=["unset", "empty", "valid", "zero", "negative", "text"], +) +def test_flush_interval_env_falls_back_instead_of_failing_the_first_request(monkeypatch, raw, expected, caplog): + """The batch scheduler rejects a non-positive delay; v2's consumer thread accepted 0, so the value must + not raise out of the lazily built logger and take Langfuse logging down for the worker.""" + if raw is None: + monkeypatch.delenv("LANGFUSE_FLUSH_INTERVAL", raising=False) + else: + monkeypatch.setenv("LANGFUSE_FLUSH_INTERVAL", raw) + with caplog.at_level(logging.WARNING, logger="LiteLLM"): + assert LangFuseLogger._get_langfuse_flush_interval(1) == expected # pyright: ignore[reportPrivateUsage] # the parser under test + assert ("LANGFUSE_FLUSH_INTERVAL" in caplog.text) is (raw in ("0", "-5", "abc")) + + +def test_zero_flush_interval_still_builds_a_working_export_channel(monkeypatch): + receiver = _OtlpReceiver() + monkeypatch.setenv("LANGFUSE_PUBLIC_KEY", "pk-flush-zero-test") + monkeypatch.setenv("LANGFUSE_SECRET_KEY", "sk-flush-zero-test") + monkeypatch.delenv("LANGFUSE_MOCK", raising=False) + monkeypatch.setenv("LANGFUSE_FLUSH_INTERVAL", "0") + monkeypatch.setattr(litellm, "initialized_langfuse_clients", litellm.initialized_langfuse_clients) + + try: + logger = LangFuseLogger(langfuse_host=receiver.url) + _log_one_completion(logger) + finally: + receiver.close() + + assert receiver.received == ["/api/public/otel/v1/traces"] + + +def test_langfuse_debug_env_string_false_stays_off(monkeypatch): + """LANGFUSE_DEBUG=false must not reach the v4 client as a truthy string. + + The v4 client does ``if debug:`` and then mutates root logging via + ``logging.basicConfig``, so the unparsed string "false" turns debug ON. + """ + monkeypatch.setenv("LANGFUSE_PUBLIC_KEY", "pk-debug-parse-test") + monkeypatch.setenv("LANGFUSE_SECRET_KEY", "sk-debug-parse-test") + monkeypatch.setenv("LANGFUSE_MOCK", "true") + monkeypatch.setenv("LANGFUSE_DEBUG", "false") + monkeypatch.setattr(litellm, "initialized_langfuse_clients", litellm.initialized_langfuse_clients) + + assert LangFuseLogger().langfuse_debug is False + + +def test_langfuse_debug_env_true_turns_on_the_langfuse_logger(monkeypatch): + """``LANGFUSE_DEBUG=true`` reached the v2 client as ``debug=`` and switched the SDK's logger to DEBUG; + a parsed flag that nothing reads would make the variable a silent no-op.""" + monkeypatch.setenv("LANGFUSE_PUBLIC_KEY", "pk-debug-wire-test") + monkeypatch.setenv("LANGFUSE_SECRET_KEY", "sk-debug-wire-test") + monkeypatch.setenv("LANGFUSE_MOCK", "true") + monkeypatch.setenv("LANGFUSE_DEBUG", "true") + monkeypatch.setattr(litellm, "initialized_langfuse_clients", litellm.initialized_langfuse_clients) + langfuse_logger = logging.getLogger("langfuse") + level_before = langfuse_logger.level + langfuse_logger.setLevel(logging.WARNING) + try: + assert LangFuseLogger().langfuse_debug is True + assert langfuse_logger.level == logging.DEBUG + finally: + langfuse_logger.setLevel(level_before) + + +def test_explicit_langfuse_host_beats_the_v4_base_url_env(monkeypatch): + """Per-key/per-team ``langfuse_host`` must win over LANGFUSE_BASE_URL. + + v4 resolves ``base_url or $LANGFUSE_BASE_URL or host``, so a stray env var + could silently redirect every tenant's traces to one server. The proof is a + real round trip: the observation lands on the configured host. + """ + receiver = _OtlpReceiver() + monkeypatch.setenv("LANGFUSE_PUBLIC_KEY", "pk-base-url-test") + monkeypatch.setenv("LANGFUSE_SECRET_KEY", "sk-base-url-test") + monkeypatch.delenv("LANGFUSE_MOCK", raising=False) + monkeypatch.setenv("LANGFUSE_BASE_URL", "http://127.0.0.1:1") + monkeypatch.setenv("LANGFUSE_FLUSH_INTERVAL", "1") + monkeypatch.setattr(litellm, "initialized_langfuse_clients", litellm.initialized_langfuse_clients) + + try: + logger = LangFuseLogger(langfuse_host=receiver.url) + _log_one_completion(logger) + finally: + receiver.close() + + assert logger.langfuse_host == receiver.url + assert receiver.received == ["/api/public/otel/v1/traces"] + + +def test_resolve_credentials_falls_back_to_langfuse_base_url(monkeypatch): + """v4's canonical env var works when LANGFUSE_HOST is unset, but never beats it.""" + monkeypatch.setenv("LANGFUSE_BASE_URL", "https://from-base-url.example") + monkeypatch.delenv("LANGFUSE_HOST", raising=False) + + _, _, host = langfuse_module.resolve_langfuse_credentials() + assert host == "https://from-base-url.example" + + monkeypatch.setenv("LANGFUSE_HOST", "https://from-host.example") + _, _, host = langfuse_module.resolve_langfuse_credentials() + assert host == "https://from-host.example" + + _, _, host = langfuse_module.resolve_langfuse_credentials(langfuse_host="https://explicit.example") + assert host == "https://explicit.example" + + +def test_version_gate_rejects_v5_prereleases(): + """ "5.0.0rc1" sorts below "5", so a plain version comparison would admit it.""" + langfuse_module.raise_if_unsupported_langfuse_version("4.7") + with pytest.raises(ImportError): + langfuse_module.raise_if_unsupported_langfuse_version("5.0.0rc1") + with pytest.raises(ImportError): + langfuse_module.raise_if_unsupported_langfuse_version("5.0.0") + + +def test_old_sdk_fails_with_the_upgrade_message_before_the_otel_module_is_imported(monkeypatch): + """On a v2 install `langfuse_sdk` itself fails to import, so the version gate must run first + or the caller is told the package is missing when it only needs upgrading.""" + import sys + + monkeypatch.setattr(langfuse_module, "installed_langfuse_version", lambda: "2.59.7") + monkeypatch.setitem(sys.modules, "litellm.integrations.langfuse.langfuse_sdk", None) + + with pytest.raises(ImportError) as raised: + _build_langfuse_logger(monkeypatch, langfuse_public_key="pk-old-sdk") + + assert "2.59.7" in str(raised.value) + assert "langfuse_otel" in str(raised.value) + assert "not installed" not in str(raised.value) + + +def test_missing_sdk_is_reported_as_not_installed(monkeypatch): + from importlib.metadata import PackageNotFoundError + + def not_installed() -> str: + raise PackageNotFoundError("langfuse") + + monkeypatch.setattr(langfuse_module, "installed_langfuse_version", not_installed) + + with pytest.raises(Exception, match="Langfuse not installed"): + _build_langfuse_logger(monkeypatch, langfuse_public_key="pk-no-sdk") + + +@pytest.mark.parametrize("raw", ["abc", "2.5", ""], ids=["text", "fraction", "empty"]) +def test_prompt_cache_ttl_typo_is_named_before_the_sdk_is_imported(monkeypatch, raw): + """The v4 SDK evaluates ``int(LANGFUSE_PROMPT_CACHE_DEFAULT_TTL_SECONDS)`` at import, so without this + gate every request failed with a bare ``invalid literal for int()`` that never named the variable.""" + import sys + + monkeypatch.setenv("LANGFUSE_PROMPT_CACHE_DEFAULT_TTL_SECONDS", raw) + monkeypatch.setitem(sys.modules, "litellm.integrations.langfuse.langfuse_sdk", None) + + with pytest.raises(ValueError, match="LANGFUSE_PROMPT_CACHE_DEFAULT_TTL_SECONDS") as raised: + _build_langfuse_logger(monkeypatch, langfuse_public_key="pk-ttl-typo") + + assert repr(raw) in str(raised.value) + + +@pytest.mark.parametrize("raw", ["5", " -3 ", "+0"], ids=["whole", "negative", "signed-zero"]) +def test_whole_second_prompt_cache_ttl_passes_the_gate(monkeypatch, raw): + monkeypatch.setenv("LANGFUSE_PROMPT_CACHE_DEFAULT_TTL_SECONDS", raw) + assert langfuse_module.raise_if_unusable_prompt_cache_ttl() is None + + +def test_stopped_logger_hands_its_export_channel_back(monkeypatch): + """`DynamicLoggingCache` calls `stop()` on expiry; the channel must be retired once every + logger that held it has stopped, or each credential rotation leaks a batch export thread.""" + from litellm.integrations.langfuse.langfuse_sdk import acquire_langfuse_tracing, release_langfuse_tracing + + logger = _build_langfuse_logger(monkeypatch, langfuse_public_key="pk-stop-releases") + + def acquire_same_credentials(): + return acquire_langfuse_tracing( + public_key="pk-stop-releases", + secret_key="sk-lit5228", + base_url=_UNREACHABLE_HOST, + environment=logger.langfuse_environment, + release=logger.langfuse_release, + flush_interval=logger.langfuse_flush_interval, + mock_mode=False, + ) + + logger.stop() + reacquired = acquire_same_credentials() + assert reacquired is logger.tracing, "the channel stays up while another logger still holds it" + + release_langfuse_tracing(reacquired, grace_seconds=0.0) + assert acquire_same_credentials() is not logger.tracing, "stop() did not give the logger's hold back" + + +def test_logger_that_fails_to_build_takes_no_slot_and_no_channel(monkeypatch): + """Each failed retry for the same dynamic credentials would otherwise eat a client slot and a + holder on the channel, so fixing the configuration could not bring Langfuse logging back.""" + from litellm.integrations.langfuse.langfuse_sdk import acquire_langfuse_tracing, release_langfuse_tracing + + monkeypatch.setenv("LANGFUSE_TIMEOUT", "5.5") + probe = _build_langfuse_logger(monkeypatch, langfuse_public_key="pk-failed-build") + + def acquire_same_credentials(): + return acquire_langfuse_tracing( + public_key="pk-failed-build", + secret_key="sk-lit5228", + base_url=_UNREACHABLE_HOST, + environment=probe.langfuse_environment, + release=probe.langfuse_release, + flush_interval=probe.langfuse_flush_interval, + mock_mode=False, + ) + + monkeypatch.setenv("LANGFUSE_TIMEOUT", "not-a-number") + with pytest.raises(ValueError, match="not-a-number"): + _build_langfuse_logger(monkeypatch, langfuse_public_key="pk-failed-build") + assert litellm.initialized_langfuse_clients == 0 + + monkeypatch.setenv("LANGFUSE_TIMEOUT", "5.5") + release_langfuse_tracing(probe.tracing, grace_seconds=0.0) + assert acquire_same_credentials() is not probe.tracing, "the failed build left a holder on the channel" + + +def test_int_steering_values_reach_langfuse_as_strings(): + """Langfuse models user, session and version as strings; v2's pydantic coerced ints for the caller.""" + rig = _steering_logger() + + _, _, span = _emit(rig, metadata={"trace_user_id": 12345, "session_id": 67, "trace_version": 3}) + + assert span.attributes["user.id"] == "12345" + assert span.attributes["session.id"] == "67" + assert span.attributes["langfuse.version"] == "3" + + +def test_long_steering_values_are_neither_capped_nor_dropped(): + """v2 sent ids of any length; the SDK's 200 character rule belongs to baggage propagation, which litellm no longer uses.""" + rig = _steering_logger() + long_user: Final = "u" * 250 + + _, _, span = _emit(rig, metadata={"trace_user_id": long_user}) + + assert span.attributes["user.id"] == long_user + + +def test_returned_generation_id_names_the_exported_observation(): + """v4 derives observation ids from the OTel span, so a pre-computed id would name nothing.""" + logger, exporter = _steering_logger() + + returned = logger.log_event_on_langfuse( + kwargs={ + "call_type": "completion", + "litellm_params": {"metadata": {"trace_id": "d" * 32}}, + "messages": [{"role": "user", "content": "the-input"}], + "optional_params": {}, + }, + response_obj=litellm.ModelResponse(choices=[{"message": {"role": "assistant", "content": "the-output"}}]), + start_time=datetime.datetime.now(), + end_time=datetime.datetime.now(), + ) + + span = _exported_span(logger, exporter) + assert returned["generation_id"] == format(span.context.span_id, "016x") diff --git a/tests/test_litellm/litellm_core_utils/specialty_caches/test_dynamic_logging_cache.py b/tests/test_litellm/litellm_core_utils/specialty_caches/test_dynamic_logging_cache.py index 5ed9dca68fd..3ab710d8023 100644 --- a/tests/test_litellm/litellm_core_utils/specialty_caches/test_dynamic_logging_cache.py +++ b/tests/test_litellm/litellm_core_utils/specialty_caches/test_dynamic_logging_cache.py @@ -24,10 +24,7 @@ class TestLangfuseInMemoryCache: # Create a mock LangFuseLogger class class MockLangFuseLogger: - def __init__(self): - self.Langfuse = MagicMock() - self.Langfuse.flush = MagicMock() - self.Langfuse.shutdown = MagicMock() + pass mock_logger = MockLangFuseLogger() @@ -50,29 +47,72 @@ class TestLangfuseInMemoryCache: assert litellm.initialized_langfuse_clients == initial_count - 1 @patch("litellm.initialized_langfuse_clients", 3) - def test_langfuse_client_shutdown_called_on_eviction(self): - """Test that langfuse client shutdown is called to close the thread.""" + def test_evicted_logger_releases_its_hold_on_the_shared_export_channel(self): + """Export channels are shared per credential set: eviction gives this logger's hold back + while a sibling logger keeps exporting, and the channel is retired once the last hold goes.""" + from litellm.integrations.langfuse.langfuse import LangFuseLogger + from litellm.integrations.langfuse.langfuse_sdk import acquire_langfuse_tracing, release_langfuse_tracing - # Create a mock LangFuseLogger class - class MockLangFuseLogger: - def __init__(self): - self.Langfuse = MagicMock() - self.Langfuse.flush = MagicMock() - self.Langfuse.shutdown = MagicMock() + def acquire(): + return acquire_langfuse_tracing( + public_key="pk-eviction-test", + secret_key="sk", + base_url="http://127.0.0.1:1", + environment=None, + release=None, + flush_interval=1.0, + mock_mode=True, + ) - mock_logger = MockLangFuseLogger() + logger = LangFuseLogger.__new__(LangFuseLogger) + logger.api_client = MagicMock() + logger.api_client.get_prompt.return_value = "prompt-after-eviction" + logger.tracing = acquire() + sibling = acquire() + self.cache.cache_dict["test_key"] = logger + self.cache.ttl_dict["test_key"] = time.time() + 100 - # Patch the LangFuseLogger import to return our mock class - with patch( - "litellm.integrations.langfuse.langfuse.LangFuseLogger", MockLangFuseLogger - ): - # Add the mock logger to cache - self.cache.cache_dict["test_key"] = mock_logger - self.cache.ttl_dict["test_key"] = time.time() + 100 + self.cache._remove_key("test_key") - # Remove the key (this should trigger cleanup) - self.cache._remove_key("test_key") + assert litellm.initialized_langfuse_clients == 2 + assert logger.api_client.get_prompt("greeting") == "prompt-after-eviction" + with sibling.tracer.start_as_current_span("still-open"): + pass + assert sibling.flush(1000) is True - # Verify flush and shutdown were called - mock_logger.Langfuse.flush.assert_called_once() - mock_logger.Langfuse.shutdown.assert_called_once() + release_langfuse_tracing(sibling, grace_seconds=0.0) + assert acquire() is not logger.tracing, "eviction did not release the evicted logger's hold" + + @patch("litellm.initialized_langfuse_clients", 3) + def test_second_evictor_of_the_same_entry_releases_nothing(self): + """Two callers can expire the same entry at once (a request thread and the reaper). Only the one that + claims the entry may give its slot and channel hold back, or a sibling logger loses its channel.""" + from litellm.integrations.langfuse.langfuse import LangFuseLogger + from litellm.integrations.langfuse.langfuse_sdk import acquire_langfuse_tracing, release_langfuse_tracing + + def acquire(): + return acquire_langfuse_tracing( + public_key="pk-double-eviction-test", + secret_key="sk", + base_url="http://127.0.0.1:1", + environment=None, + release=None, + flush_interval=1.0, + mock_mode=True, + ) + + logger = LangFuseLogger.__new__(LangFuseLogger) + logger.api_client = MagicMock() + logger.tracing = acquire() + sibling = acquire() + self.cache.cache_dict["test_key"] = logger + self.cache.ttl_dict["test_key"] = time.time() + 100 + + self.cache._remove_key("test_key") + self.cache._remove_key("test_key") + + assert litellm.initialized_langfuse_clients == 2 + assert "test_key" not in self.cache.cache_dict and "test_key" not in self.cache.ttl_dict + assert acquire() is sibling, "the second evictor took the sibling logger's hold on the channel" + release_langfuse_tracing(sibling) + release_langfuse_tracing(sibling, grace_seconds=0.0) diff --git a/tests/test_litellm/proxy/health_endpoints/test_health_endpoints.py b/tests/test_litellm/proxy/health_endpoints/test_health_endpoints.py index ee4c468a460..3ec5176159d 100644 --- a/tests/test_litellm/proxy/health_endpoints/test_health_endpoints.py +++ b/tests/test_litellm/proxy/health_endpoints/test_health_endpoints.py @@ -4307,3 +4307,22 @@ async def test_health_services_endpoint_pointfive_blocks_non_admin(monkeypatch, assert str(raised.value.code) == "403" logger_class.assert_not_called() + + +@pytest.mark.asyncio +async def test_health_services_endpoint_langfuse_missing_keys_errors(monkeypatch): + """v2 raised out of ``auth_check`` and the endpoint printed the server's answer; the v4 check + returns the failure as a value, and the endpoint has to error with that reason rather than a + generic credentials message that reads the same for an outage and a bad key.""" + import litellm.integrations.langfuse.langfuse as langfuse_module + from litellm.integrations.langfuse.langfuse_sdk import AuthCheckFailure + + logger_class = MagicMock() + logger_class.return_value.api_client.auth_check.return_value = AuthCheckFailure( + "connection refused by lf.internal.example" + ) + monkeypatch.setattr(langfuse_module, "LangFuseLogger", logger_class) + + with pytest.raises(ProxyException, match="auth_check failed") as raised: + await health_services_endpoint(service="langfuse") + assert "connection refused by lf.internal.example" in str(raised.value.message) diff --git a/tests/test_litellm/proxy/management_endpoints/test_callback_management_endpoints.py b/tests/test_litellm/proxy/management_endpoints/test_callback_management_endpoints.py index 5c5cfd0814d..8f19cf6329c 100644 --- a/tests/test_litellm/proxy/management_endpoints/test_callback_management_endpoints.py +++ b/tests/test_litellm/proxy/management_endpoints/test_callback_management_endpoints.py @@ -81,35 +81,29 @@ class TestCallbackManagementEndpoints: # Setup test client client = TestClient(app) - # Initialize Langfuse logger and add to callbacks - with patch("litellm.integrations.langfuse.langfuse.Langfuse") as mock_langfuse: - # Mock the Langfuse client initialization - mock_langfuse_client = MagicMock() - mock_langfuse.return_value = mock_langfuse_client + # Add string representation to callback lists (this is how the system typically works) + litellm.success_callback.append("langfuse") + litellm._async_success_callback.append("langfuse") - # Add string representation to callback lists (this is how the system typically works) - litellm.success_callback.append("langfuse") - litellm._async_success_callback.append("langfuse") + # Make request to list callbacks endpoint + response = client.get( + "/callbacks/list", headers={"Authorization": "Bearer sk-1234"} + ) - # Make request to list callbacks endpoint - response = client.get( - "/callbacks/list", headers={"Authorization": "Bearer sk-1234"} - ) + # Verify response + assert response.status_code == 200 - # Verify response - assert response.status_code == 200 + response_data = response.json() - response_data = response.json() + # Verify langfuse appears in success callbacks + assert "langfuse" in response_data["success"] + assert response_data["failure"] == [] + assert response_data["success_and_failure"] == [] - # Verify langfuse appears in success callbacks - assert "langfuse" in response_data["success"] - assert response_data["failure"] == [] - assert response_data["success_and_failure"] == [] - - # Verify the response structure is correct - assert isinstance(response_data["success"], list) - assert isinstance(response_data["failure"], list) - assert isinstance(response_data["success_and_failure"], list) + # Verify the response structure is correct + assert isinstance(response_data["success"], list) + assert isinstance(response_data["failure"], list) + assert isinstance(response_data["success_and_failure"], list) def test_alist_callbacks_with_datadog_logger(self): """Test /callbacks/list endpoint with DataDog logger configuration""" diff --git a/tests/test_litellm/proxy/proxy_server/test_lifecycle.py b/tests/test_litellm/proxy/proxy_server/test_lifecycle.py index 36d2e16d261..6feb37e9867 100644 --- a/tests/test_litellm/proxy/proxy_server/test_lifecycle.py +++ b/tests/test_litellm/proxy/proxy_server/test_lifecycle.py @@ -132,6 +132,62 @@ async def test_proxy_shutdown_event_disconnects_prisma_and_resets(monkeypatch): } +@pytest.mark.asyncio +async def test_proxy_shutdown_flushes_every_langfuse_export_channel(monkeypatch): + """A generation finished just before a graceful restart is still queued in its batch + processor, so shutdown must flush every acquired export channel.""" + from litellm.integrations.langfuse import langfuse_sdk + + flushed = MagicMock(return_value=True) + monkeypatch.setattr(langfuse_sdk, "flush_langfuse_tracing", flushed) + monkeypatch.setattr(ps, "prisma_client", None, raising=False) + monkeypatch.setattr(ps, "jwt_handler", MagicMock(close=AsyncMock()), raising=False) + monkeypatch.setattr(ps, "db_writer_client", None, raising=False) + + import litellm + + monkeypatch.setattr(litellm, "cache", None, raising=False) + monkeypatch.setattr(litellm, "success_callback", [], raising=False) + + await proxy_shutdown_event() + + assert flushed.call_count == 1 + + +@pytest.mark.asyncio +async def test_proxy_shutdown_flushes_langfuse_off_the_event_loop_and_logs_a_timeout(monkeypatch, caplog): + """The flush blocks on OTLP exports for up to its deadline, so it must run on a worker thread + with the shutdown deadline, and a channel that misses it is reported instead of ignored.""" + import threading + + from litellm.constants import LANGFUSE_SHUTDOWN_FLUSH_TIMEOUT_MILLIS + from litellm.integrations.langfuse import langfuse_sdk + + ran_on = MagicMock() + + def flushed(timeout_millis: int) -> bool: + ran_on(threading.current_thread(), timeout_millis) + return False + + monkeypatch.setattr(langfuse_sdk, "flush_langfuse_tracing", flushed) + monkeypatch.setattr(ps, "prisma_client", None, raising=False) + monkeypatch.setattr(ps, "jwt_handler", MagicMock(close=AsyncMock()), raising=False) + monkeypatch.setattr(ps, "db_writer_client", None, raising=False) + + import litellm + + monkeypatch.setattr(litellm, "cache", None, raising=False) + monkeypatch.setattr(litellm, "success_callback", [], raising=False) + + with caplog.at_level("WARNING", logger="LiteLLM Proxy"): + await proxy_shutdown_event() + + (flush_thread, timeout_millis), _ = ran_on.call_args + assert flush_thread is not threading.main_thread() + assert timeout_millis == LANGFUSE_SHUTDOWN_FLUSH_TIMEOUT_MILLIS + assert any("Langfuse shutdown flush incomplete" in record.getMessage() for record in caplog.records) + + @pytest.mark.asyncio async def test_proxy_shutdown_drains_gateway_requests_before_disconnecting(monkeypatch): """ diff --git a/uv.lock b/uv.lock index 8f63ca2b564..c235171ecb2 100644 --- a/uv.lock +++ b/uv.lock @@ -4256,21 +4256,22 @@ wheels = [ [[package]] name = "langfuse" -version = "2.59.7" +version = "4.15.2" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "anyio" }, { name = "backoff" }, { name = "httpx" }, - { name = "idna" }, + { name = "opentelemetry-api" }, + { name = "opentelemetry-exporter-otlp-proto-http" }, + { name = "opentelemetry-sdk" }, { name = "packaging" }, { name = "pydantic" }, - { name = "requests" }, + { name = "typing-extensions" }, { name = "wrapt" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/d5/0e/8390bd3a4ad92ecb1ba0462ec8b7c7d328b2e2f31ae0e734bf2f50dbdc96/langfuse-2.59.7.tar.gz", hash = "sha256:f631981705177bf53d030d191397da9b864b99729a7273448afed10d76f78e23", size = 146608, upload-time = "2025-03-03T16:30:59.926Z" } +sdist = { url = "https://files.pythonhosted.org/packages/97/30/6a64dcf84de2f2eb4d03adbfd22cc7bdc95ce67e3e56cd6288405087fb8a/langfuse-4.15.2.tar.gz", hash = "sha256:7f818f38cc22daba88fdcec62d2addcee4e18d1af4529978b6d07501e86b6946", size = 391727, upload-time = "2026-09-09T16:01:25.73Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/b7/f3/420518b9003c997cdcb0a86473bf0c111181578a95565823c333cb58eb7b/langfuse-2.59.7-py3-none-any.whl", hash = "sha256:2c6890f5b842257173eb54d08f2890c7fd7617859a48b3914ef73f13a6514473", size = 260468, upload-time = "2025-03-03T16:30:57.426Z" }, + { url = "https://files.pythonhosted.org/packages/02/de/e59da18cb5ca9cb8515a254199bd96ace8cf918788d13ded66aa2693e007/langfuse-4.15.2-py3-none-any.whl", hash = "sha256:98c27a3c06e18c4497045f2d4decce2c716ef11cb215cf5b27bbea6ee0877115", size = 705824, upload-time = "2026-09-09T16:01:23.696Z" }, ] [[package]] @@ -4794,7 +4795,7 @@ requires-dist = [ { name = "jinja2", specifier = ">=3.1.6,<4.0" }, { name = "jsonschema", specifier = ">=4.0.0,<5.0" }, { name = "keyring", marker = "extra == 'cli'", specifier = ">=25.6.0,<26.0" }, - { name = "langfuse", marker = "extra == 'proxy-runtime'", specifier = ">=2.59.7,<3.0" }, + { name = "langfuse", marker = "extra == 'proxy-runtime'", specifier = ">=4.7,<5.0" }, { name = "litellm-enterprise", marker = "extra == 'proxy'", editable = "enterprise" }, { name = "litellm-proxy-extras", marker = "extra == 'proxy'", editable = "litellm-proxy-extras" }, { name = "llm-sandbox", marker = "extra == 'proxy-runtime'", specifier = ">=0.3.39,<1.0" }, @@ -4806,10 +4807,10 @@ requires-dist = [ { name = "numpydoc", marker = "extra == 'utils'", specifier = ">=1.8.0,<2.0" }, { name = "nvidia-riva-client", marker = "extra == 'stt-nvidia-riva'", specifier = ">=2.15.0" }, { name = "openai", specifier = ">=2.20.0,<3.0.0" }, - { name = "opentelemetry-api", marker = "extra == 'proxy-runtime'", specifier = "==1.28.0" }, - { name = "opentelemetry-exporter-otlp", marker = "extra == 'proxy-runtime'", specifier = "==1.28.0" }, - { name = "opentelemetry-instrumentation-fastapi", marker = "extra == 'proxy-runtime'", specifier = "==0.49b0" }, - { name = "opentelemetry-sdk", marker = "extra == 'proxy-runtime'", specifier = "==1.28.0" }, + { name = "opentelemetry-api", marker = "extra == 'proxy-runtime'", specifier = "==1.33.1" }, + { name = "opentelemetry-exporter-otlp", marker = "extra == 'proxy-runtime'", specifier = "==1.33.1" }, + { name = "opentelemetry-instrumentation-fastapi", marker = "extra == 'proxy-runtime'", specifier = "==0.54b1" }, + { name = "opentelemetry-sdk", marker = "extra == 'proxy-runtime'", specifier = "==1.33.1" }, { name = "orjson", marker = "extra == 'proxy'", specifier = ">=3.11.6,<4.0" }, { name = "packaging", specifier = ">=24.0" }, { name = "polars", marker = "extra == 'proxy'", specifier = ">=1.38.1,<2.0" }, @@ -4889,7 +4890,7 @@ ci = [ { name = "pytest-codspeed", specifier = "==4.3.0" }, { name = "pytest-retry", specifier = "==1.7.0" }, { name = "tenacity", specifier = "==8.5.0" }, - { name = "traceloop-sdk", specifier = "==0.33.12" }, + { name = "traceloop-sdk", specifier = "==0.34.0" }, ] dev = [ { name = "basedpyright", specifier = "==1.39.7" }, @@ -4899,14 +4900,14 @@ dev = [ { name = "fastapi-offline", specifier = "==1.7.6" }, { name = "hypothesis", specifier = "==6.165.10" }, { name = "keyring", specifier = "==25.7.0" }, - { name = "langfuse", specifier = "==2.59.7" }, + { name = "langfuse", specifier = ">=4.7,<5.0" }, { name = "mypy", specifier = "==1.20.1" }, { name = "numpy", specifier = ">=1.26.0,<3.0" }, { name = "openapi-core", specifier = "==0.22.0" }, - { name = "opentelemetry-api", specifier = "==1.28.0" }, - { name = "opentelemetry-exporter-otlp", specifier = "==1.28.0" }, - { name = "opentelemetry-instrumentation-fastapi", specifier = "==0.49b0" }, - { name = "opentelemetry-sdk", specifier = "==1.28.0" }, + { name = "opentelemetry-api", specifier = "==1.33.1" }, + { name = "opentelemetry-exporter-otlp", specifier = "==1.33.1" }, + { name = "opentelemetry-instrumentation-fastapi", specifier = "==0.54b1" }, + { name = "opentelemetry-sdk", specifier = "==1.33.1" }, { name = "parameterized", specifier = "==0.9.0" }, { name = "psycopg", specifier = "==3.3.3" }, { name = "psycopg-binary", specifier = "==3.3.3" }, @@ -4949,10 +4950,10 @@ proxy-dev = [ { name = "a2a-sdk", specifier = "==1.1.0" }, { name = "azure-identity", specifier = "==1.25.2" }, { name = "hypercorn", specifier = "==0.17.3" }, - { name = "opentelemetry-api", specifier = "==1.28.0" }, - { name = "opentelemetry-exporter-otlp", specifier = "==1.28.0" }, - { name = "opentelemetry-instrumentation-fastapi", specifier = "==0.49b0" }, - { name = "opentelemetry-sdk", specifier = "==1.28.0" }, + { name = "opentelemetry-api", specifier = "==1.33.1" }, + { name = "opentelemetry-exporter-otlp", specifier = "==1.33.1" }, + { name = "opentelemetry-instrumentation-fastapi", specifier = "==0.54b1" }, + { name = "opentelemetry-sdk", specifier = "==1.33.1" }, { name = "prisma", specifier = "==0.11.0" }, { name = "prometheus-client", specifier = "==0.20.0" }, ] @@ -6152,45 +6153,45 @@ wheels = [ [[package]] name = "opentelemetry-api" -version = "1.28.0" +version = "1.33.1" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "deprecated" }, { name = "importlib-metadata" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/79/36/260eaea0f74fdd0c0d8f22ed3a3031109ea1c85531f94f4fde266c29e29a/opentelemetry_api-1.28.0.tar.gz", hash = "sha256:578610bcb8aa5cdcb11169d136cc752958548fb6ccffb0969c1036b0ee9e5353", size = 62803, upload-time = "2024-11-05T19:14:45.497Z" } +sdist = { url = "https://files.pythonhosted.org/packages/9a/8d/1f5a45fbcb9a7d87809d460f09dc3399e3fbd31d7f3e14888345e9d29951/opentelemetry_api-1.33.1.tar.gz", hash = "sha256:1c6055fc0a2d3f23a50c7e17e16ef75ad489345fd3df1f8b8af7c0bbf8a109e8", size = 65002, upload-time = "2025-05-16T18:52:41.146Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/22/e4/3b25d8b856791c04d8a62b1257b5fc09dc41a057800db06885af8ddcdce1/opentelemetry_api-1.28.0-py3-none-any.whl", hash = "sha256:8457cd2c59ea1bd0988560f021656cecd254ad7ef6be4ba09dbefeca2409ce52", size = 64314, upload-time = "2024-11-05T19:14:21.659Z" }, + { url = "https://files.pythonhosted.org/packages/05/44/4c45a34def3506122ae61ad684139f0bbc4e00c39555d4f7e20e0e001c8a/opentelemetry_api-1.33.1-py3-none-any.whl", hash = "sha256:4db83ebcf7ea93e64637ec6ee6fabee45c5cbe4abd9cf3da95c43828ddb50b83", size = 65771, upload-time = "2025-05-16T18:52:17.419Z" }, ] [[package]] name = "opentelemetry-exporter-otlp" -version = "1.28.0" +version = "1.33.1" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "opentelemetry-exporter-otlp-proto-grpc" }, { name = "opentelemetry-exporter-otlp-proto-http" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/eb/16/14e3fc163930ea68f0980a4cdd4ae5796e60aeb898965990e13263d64baf/opentelemetry_exporter_otlp-1.28.0.tar.gz", hash = "sha256:31ae7495831681dd3da34ac457f6970f147465ae4b9aae3a888d7a581c7cd868", size = 6170, upload-time = "2024-11-05T19:14:47.349Z" } +sdist = { url = "https://files.pythonhosted.org/packages/b1/3f/c8ad4f1c3aaadcea2b0f1b4d7970e7b7898c145699769a789f3435143f69/opentelemetry_exporter_otlp-1.33.1.tar.gz", hash = "sha256:4d050311ea9486e3994575aa237e32932aad58330a31fba24fdba5c0d531cf04", size = 6189, upload-time = "2025-05-16T18:52:43.176Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/c2/82/3f521b3c1f2a411ed60a24a8c9f486c1beeaf8c6c55337c87d3ae1642151/opentelemetry_exporter_otlp-1.28.0-py3-none-any.whl", hash = "sha256:1fd02d70f2c1b7ac5579c81e78de4594b188d3317c8ceb69e8b53900fb7b40fd", size = 7024, upload-time = "2024-11-05T19:14:24.534Z" }, + { url = "https://files.pythonhosted.org/packages/4d/32/b9add70dd4e845654fc9fcd1401a705477743880be6c3e62acb1ad0d8662/opentelemetry_exporter_otlp-1.33.1-py3-none-any.whl", hash = "sha256:9bcf1def35b880b55a49e31ebd63910edac14b294fd2ab884953c4deaff5b300", size = 7045, upload-time = "2025-05-16T18:52:21.022Z" }, ] [[package]] name = "opentelemetry-exporter-otlp-proto-common" -version = "1.28.0" +version = "1.33.1" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "opentelemetry-proto" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/c2/8d/5d411084ac441052f4c9bae03a1aec65ae5d16b439fea7b9c5ac3842c013/opentelemetry_exporter_otlp_proto_common-1.28.0.tar.gz", hash = "sha256:5fa0419b0c8e291180b0fc8430a20dd44a3f3236f8e0827992145914f273ec4f", size = 18505, upload-time = "2024-11-05T19:14:48.204Z" } +sdist = { url = "https://files.pythonhosted.org/packages/7a/18/a1ec9dcb6713a48b4bdd10f1c1e4d5d2489d3912b80d2bcc059a9a842836/opentelemetry_exporter_otlp_proto_common-1.33.1.tar.gz", hash = "sha256:c57b3fa2d0595a21c4ed586f74f948d259d9949b58258f11edb398f246bec131", size = 20828, upload-time = "2025-05-16T18:52:43.795Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/e1/72/3c44aabc74db325aaba09361b6a0d80f6d601f0ff86ecea8ee655c9538fc/opentelemetry_exporter_otlp_proto_common-1.28.0-py3-none-any.whl", hash = "sha256:467e6437d24e020156dffecece8c0a4471a8a60f6a34afeda7386df31a092410", size = 18403, upload-time = "2024-11-05T19:14:25.798Z" }, + { url = "https://files.pythonhosted.org/packages/09/52/9bcb17e2c29c1194a28e521b9d3f2ced09028934c3c52a8205884c94b2df/opentelemetry_exporter_otlp_proto_common-1.33.1-py3-none-any.whl", hash = "sha256:b81c1de1ad349785e601d02715b2d29d6818aed2c809c20219f3d1f20b038c36", size = 18839, upload-time = "2025-05-16T18:52:22.447Z" }, ] [[package]] name = "opentelemetry-exporter-otlp-proto-grpc" -version = "1.28.0" +version = "1.33.1" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "deprecated" }, @@ -6201,14 +6202,14 @@ dependencies = [ { name = "opentelemetry-proto" }, { name = "opentelemetry-sdk" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/43/4d/f215162e58041afb4bdf5dbd0d8faf0b7fc9bf7b3d3fc0e44e06f9e7e869/opentelemetry_exporter_otlp_proto_grpc-1.28.0.tar.gz", hash = "sha256:47a11c19dc7f4289e220108e113b7de90d59791cb4c37fc29f69a6a56f2c3735", size = 26237, upload-time = "2024-11-05T19:14:49.026Z" } +sdist = { url = "https://files.pythonhosted.org/packages/d8/5f/75ef5a2a917bd0e6e7b83d3fb04c99236ee958f6352ba3019ea9109ae1a6/opentelemetry_exporter_otlp_proto_grpc-1.33.1.tar.gz", hash = "sha256:345696af8dc19785fac268c8063f3dc3d5e274c774b308c634f39d9c21955728", size = 22556, upload-time = "2025-05-16T18:52:44.76Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/1d/b5/afabc8106abc0f9cfeecf5b3e682622b3e04bba1d9b967dbfcd91b9c4ebe/opentelemetry_exporter_otlp_proto_grpc-1.28.0-py3-none-any.whl", hash = "sha256:edbdc53e7783f88d4535db5807cb91bd7b1ec9e9b9cdbfee14cd378f29a3b328", size = 18532, upload-time = "2024-11-05T19:14:26.853Z" }, + { url = "https://files.pythonhosted.org/packages/ba/ec/6047e230bb6d092c304511315b13893b1c9d9260044dd1228c9d48b6ae0e/opentelemetry_exporter_otlp_proto_grpc-1.33.1-py3-none-any.whl", hash = "sha256:7e8da32c7552b756e75b4f9e9c768a61eb47dee60b6550b37af541858d669ce1", size = 18591, upload-time = "2025-05-16T18:52:23.772Z" }, ] [[package]] name = "opentelemetry-exporter-otlp-proto-http" -version = "1.28.0" +version = "1.33.1" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "deprecated" }, @@ -6219,14 +6220,14 @@ dependencies = [ { name = "opentelemetry-sdk" }, { name = "requests" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/f1/2a/555f2845928086cd51aa6941c7a546470805b68ed631ec139ce7d841763d/opentelemetry_exporter_otlp_proto_http-1.28.0.tar.gz", hash = "sha256:d83a9a03a8367ead577f02a64127d827c79567de91560029688dd5cfd0152a8e", size = 15051, upload-time = "2024-11-05T19:14:49.813Z" } +sdist = { url = "https://files.pythonhosted.org/packages/60/48/e4314ac0ed2ad043c07693d08c9c4bf5633857f5b72f2fefc64fd2b114f6/opentelemetry_exporter_otlp_proto_http-1.33.1.tar.gz", hash = "sha256:46622d964a441acb46f463ebdc26929d9dec9efb2e54ef06acdc7305e8593c38", size = 15353, upload-time = "2025-05-16T18:52:45.522Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/b2/ce/80d5adabbf7ab4a0ca7b5e0f4039b24d273be370c3ba85fc05b13794411c/opentelemetry_exporter_otlp_proto_http-1.28.0-py3-none-any.whl", hash = "sha256:e8f3f7961b747edb6b44d51de4901a61e9c01d50debd747b120a08c4996c7e7b", size = 17228, upload-time = "2024-11-05T19:14:28.613Z" }, + { url = "https://files.pythonhosted.org/packages/63/ba/5a4ad007588016fe37f8d36bf08f325fe684494cc1e88ca8fa064a4c8f57/opentelemetry_exporter_otlp_proto_http-1.33.1-py3-none-any.whl", hash = "sha256:ebd6c523b89a2ecba0549adb92537cc2bf647b4ee61afbbd5a4c6535aa3da7cf", size = 17733, upload-time = "2025-05-16T18:52:25.137Z" }, ] [[package]] name = "opentelemetry-instrumentation" -version = "0.49b0" +version = "0.54b1" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "opentelemetry-api" }, @@ -6234,14 +6235,14 @@ dependencies = [ { name = "packaging" }, { name = "wrapt" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/de/6b/6c25b15063c92a011cf3f68375971e2c58a9c764690847edc97df2d94eeb/opentelemetry_instrumentation-0.49b0.tar.gz", hash = "sha256:398a93e0b9dc2d11cc8627e1761665c506fe08c6b2df252a2ab3ade53d751c46", size = 26478, upload-time = "2024-11-05T19:21:41.402Z" } +sdist = { url = "https://files.pythonhosted.org/packages/c3/fd/5756aea3fdc5651b572d8aef7d94d22a0a36e49c8b12fcb78cb905ba8896/opentelemetry_instrumentation-0.54b1.tar.gz", hash = "sha256:7658bf2ff914b02f246ec14779b66671508125c0e4227361e56b5ebf6cef0aec", size = 28436, upload-time = "2025-05-16T19:03:22.223Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/93/61/e0d21e958d6072ce25c4f5e26a1d22835fc86f80836660adf6badb6038ce/opentelemetry_instrumentation-0.49b0-py3-none-any.whl", hash = "sha256:68364d73a1ff40894574cbc6138c5f98674790cae1f3b0865e21cf702f24dcb3", size = 30694, upload-time = "2024-11-05T19:20:38.584Z" }, + { url = "https://files.pythonhosted.org/packages/f4/89/0790abc5d9c4fc74bd3e03cb87afe2c820b1d1a112a723c1163ef32453ee/opentelemetry_instrumentation-0.54b1-py3-none-any.whl", hash = "sha256:a4ae45f4a90c78d7006c51524f57cd5aa1231aef031eae905ee34d5423f5b198", size = 31019, upload-time = "2025-05-16T19:02:15.611Z" }, ] [[package]] name = "opentelemetry-instrumentation-alephalpha" -version = "0.33.12" +version = "0.34.0" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "opentelemetry-api" }, @@ -6249,14 +6250,14 @@ dependencies = [ { name = "opentelemetry-semantic-conventions" }, { name = "opentelemetry-semantic-conventions-ai" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/47/32/15048d7773f6018abcd5b85f5c346b44fad8322031f6b4ea5a6c5ada304a/opentelemetry_instrumentation_alephalpha-0.33.12.tar.gz", hash = "sha256:b474ac634cd1e12b30c8863a925320a01043af8c0f46fd58288e587073d6ddec", size = 3727, upload-time = "2024-11-13T20:27:50.425Z" } +sdist = { url = "https://files.pythonhosted.org/packages/64/12/b962c7fd3d29bc4ffe70f41fab8054d0221ebfecc28a344aef6fc749be67/opentelemetry_instrumentation_alephalpha-0.34.0.tar.gz", hash = "sha256:ed6647505963d53aed63b0b2ca84c989ca94ccc215ad19355a7de33e0b10f0ac", size = 3688, upload-time = "2024-12-12T21:02:01.771Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/d3/77/e483e2fa14fddc87324b242d59992cfbd2d590563352aa044d11e1d200ed/opentelemetry_instrumentation_alephalpha-0.33.12-py3-none-any.whl", hash = "sha256:b3c7e3dd99121f5c52d7c7a3a82dd2d7a9ba7360f63ac6fcbdca187f58756e16", size = 5116, upload-time = "2024-11-13T20:27:12.818Z" }, + { url = "https://files.pythonhosted.org/packages/ef/1b/d37c9af6319ad64b182f77aec1154f5fab25b9123c9e04fe1a6d19e19e7e/opentelemetry_instrumentation_alephalpha-0.34.0-py3-none-any.whl", hash = "sha256:4e05e1b12edf30597e3cb6163d2e63f938fd3b061a3251940ac12783d1103ce6", size = 5101, upload-time = "2024-12-12T21:01:12.317Z" }, ] [[package]] name = "opentelemetry-instrumentation-anthropic" -version = "0.33.12" +version = "0.34.0" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "opentelemetry-api" }, @@ -6264,14 +6265,14 @@ dependencies = [ { name = "opentelemetry-semantic-conventions" }, { name = "opentelemetry-semantic-conventions-ai" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/40/0a/cba0a6ac1832e3002158b5a9451268aebfe0150c7b8355068d1f2cea148b/opentelemetry_instrumentation_anthropic-0.33.12.tar.gz", hash = "sha256:0bc1fd9d4cf2feec4fe9f80c0bdfcbfab33ed9cf0edea850b6c198a8679b01ff", size = 8711, upload-time = "2024-11-13T20:27:52.005Z" } +sdist = { url = "https://files.pythonhosted.org/packages/d1/56/57bbdb8907e14793d9831b220a6561a29033204acc60ce2ebc6387d29ad5/opentelemetry_instrumentation_anthropic-0.34.0.tar.gz", hash = "sha256:ab4336723de8cc3327aeacfab6e2fa085101f92614a402ee2822f8fb557ba7a6", size = 8693, upload-time = "2024-12-12T21:02:02.731Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/ba/46/ba2dc8d18b04acae3d34facd8fe1e5e0cdc9fe64292d45eca9d1d4a8a298/opentelemetry_instrumentation_anthropic-0.33.12-py3-none-any.whl", hash = "sha256:b31618d12a429045db14ed982a142a25df0f0f1dbf03d756e8d597f25b9a053d", size = 11024, upload-time = "2024-11-13T20:27:14.622Z" }, + { url = "https://files.pythonhosted.org/packages/5c/8e/ef2782ecd3e2b03fb792f42ade5fea3c549ba28e6ebefdcf95a4c14412df/opentelemetry_instrumentation_anthropic-0.34.0-py3-none-any.whl", hash = "sha256:8fc397802033636eb74967ffc6a85344e575ea615b5de502386b0a004b07ba68", size = 11005, upload-time = "2024-12-12T21:01:13.846Z" }, ] [[package]] name = "opentelemetry-instrumentation-asgi" -version = "0.49b0" +version = "0.54b1" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "asgiref" }, @@ -6280,14 +6281,14 @@ dependencies = [ { name = "opentelemetry-semantic-conventions" }, { name = "opentelemetry-util-http" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/e8/55/693c3d0938ba5fead5c3aa4ac7022a992b4ff99a8e9979800d0feb843ff4/opentelemetry_instrumentation_asgi-0.49b0.tar.gz", hash = "sha256:959fd9b1345c92f20c6ef1d42f92ef6a76b3c3083fbc4104d59da6859b15b083", size = 24117, upload-time = "2024-11-05T19:21:46.769Z" } +sdist = { url = "https://files.pythonhosted.org/packages/20/f7/a3377f9771947f4d3d59c96841d3909274f446c030dbe8e4af871695ddee/opentelemetry_instrumentation_asgi-0.54b1.tar.gz", hash = "sha256:ab4df9776b5f6d56a78413c2e8bbe44c90694c67c844a1297865dc1bd926ed3c", size = 24230, upload-time = "2025-05-16T19:03:30.234Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/2c/0b/7900c782a1dfaa584588d724bc3bbdf8405a32497537dd96b3fcbf8461b9/opentelemetry_instrumentation_asgi-0.49b0-py3-none-any.whl", hash = "sha256:722a90856457c81956c88f35a6db606cc7db3231046b708aae2ddde065723dbe", size = 16326, upload-time = "2024-11-05T19:20:46.176Z" }, + { url = "https://files.pythonhosted.org/packages/20/24/7a6f0ae79cae49927f528ecee2db55a5bddd87b550e310ce03451eae7491/opentelemetry_instrumentation_asgi-0.54b1-py3-none-any.whl", hash = "sha256:84674e822b89af563b283a5283c2ebb9ed585d1b80a1c27fb3ac20b562e9f9fc", size = 16338, upload-time = "2025-05-16T19:02:22.808Z" }, ] [[package]] name = "opentelemetry-instrumentation-bedrock" -version = "0.33.12" +version = "0.34.0" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "anthropic" }, @@ -6296,14 +6297,14 @@ dependencies = [ { name = "opentelemetry-semantic-conventions" }, { name = "opentelemetry-semantic-conventions-ai" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/95/5a/346c17fca4dd929ce6be8cf402cd3580bb6e4da42ca8eadd2b6f2b4907e4/opentelemetry_instrumentation_bedrock-0.33.12.tar.gz", hash = "sha256:6f5a3f7044edff020d62b3e94f0ea543da4e5c23b7cdb72642692952843b0003", size = 7690, upload-time = "2024-11-13T20:27:53.497Z" } +sdist = { url = "https://files.pythonhosted.org/packages/aa/79/c384051d3e234ffb5f995ecb2245aef54083dc4919258601d9449c8c47bd/opentelemetry_instrumentation_bedrock-0.34.0.tar.gz", hash = "sha256:07f0ed84fa6d9e93c8cefee48ce171c59961c44708fcc11ec21fc1fbcdfb314d", size = 7695, upload-time = "2024-12-12T21:02:04.602Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/d1/04/d93857519edd693e72e6d9ba08a6f0feda2ca21a08e3bc02cbfe242495f6/opentelemetry_instrumentation_bedrock-0.33.12-py3-none-any.whl", hash = "sha256:f9749898c52643d5027b45ac92bf4d3fd39b83adfaf68705a0ed9b4f04b8afae", size = 8982, upload-time = "2024-11-13T20:27:15.935Z" }, + { url = "https://files.pythonhosted.org/packages/56/2c/6d3e353d69407b308a254713728a613651bbe34138956f4f6b0104a5cc0a/opentelemetry_instrumentation_bedrock-0.34.0-py3-none-any.whl", hash = "sha256:1e521e33721e0fbcde2c2cb7cf788e2b8926063846800db777678be583bf1420", size = 8966, upload-time = "2024-12-12T21:01:16.457Z" }, ] [[package]] name = "opentelemetry-instrumentation-chromadb" -version = "0.33.12" +version = "0.34.0" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "opentelemetry-api" }, @@ -6311,14 +6312,14 @@ dependencies = [ { name = "opentelemetry-semantic-conventions" }, { name = "opentelemetry-semantic-conventions-ai" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/62/05/ae78dd08c30203009815b35bce9b524458d73174e7dc4924a431e7b0b65b/opentelemetry_instrumentation_chromadb-0.33.12.tar.gz", hash = "sha256:eb4c591d398963504f82c20879030ea3694f10065ee62450da761c9b6e1792e7", size = 4598, upload-time = "2024-11-13T20:27:54.38Z" } +sdist = { url = "https://files.pythonhosted.org/packages/29/8e/0846e9c8846eee6f782767a1ee2f760ed5ca53cc95035189706c63027d58/opentelemetry_instrumentation_chromadb-0.34.0.tar.gz", hash = "sha256:ed0b4842db9bd35a0cff138d88d84d63a1529038ac11cf37eeba1dd294d4a2e8", size = 4596, upload-time = "2024-12-12T21:02:06.825Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/cc/2f/3fabec28e538fc0671c5c149e979e6f36f823078aa8c43ab0f69392185e1/opentelemetry_instrumentation_chromadb-0.33.12-py3-none-any.whl", hash = "sha256:2413426c3bf1f3714a95318e934f090fa778ab7b3d7bdd2cc8ee068cda216a06", size = 6322, upload-time = "2024-11-13T20:27:18.789Z" }, + { url = "https://files.pythonhosted.org/packages/9a/b6/132c1cdd8dea4f0e4e1cab910dabdacd9802fb3a8e802e0c825bf6e9691f/opentelemetry_instrumentation_chromadb-0.34.0-py3-none-any.whl", hash = "sha256:d95df8285405a23b82c3b6d0c1b7c439ec86793d21b3a23e51965853d3e9c4a6", size = 6303, upload-time = "2024-12-12T21:01:17.711Z" }, ] [[package]] name = "opentelemetry-instrumentation-cohere" -version = "0.33.12" +version = "0.34.0" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "opentelemetry-api" }, @@ -6326,14 +6327,14 @@ dependencies = [ { name = "opentelemetry-semantic-conventions" }, { name = "opentelemetry-semantic-conventions-ai" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/fe/96/f9cfc4f27c20deabfca237cb26734f060525b43e6993f753fad4ee0eded1/opentelemetry_instrumentation_cohere-0.33.12.tar.gz", hash = "sha256:4ea626d096fdf4c64e04a63b437e36f72a4341f818034ee6dc73ba1dba9ab341", size = 4235, upload-time = "2024-11-13T20:27:55.358Z" } +sdist = { url = "https://files.pythonhosted.org/packages/b2/bc/d38c64d0e0f92fb8b8bcde241024dd5d4810c2fbe379fdfbbbb32dee957f/opentelemetry_instrumentation_cohere-0.34.0.tar.gz", hash = "sha256:80e27c6f86a73a2c0e89aa3c9ca1a37ff58a01b4c0eb7f249d7ae66568730477", size = 4227, upload-time = "2024-12-12T21:02:08.177Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/bf/08/dce2b7926ace0204ce7946563348e1ff755873e387833484791e4ed391c8/opentelemetry_instrumentation_cohere-0.33.12-py3-none-any.whl", hash = "sha256:3bee3f7f7105259c85145be8c3b68612421860c95ad170f4d03144a3b8c07418", size = 5589, upload-time = "2024-11-13T20:27:21.317Z" }, + { url = "https://files.pythonhosted.org/packages/e3/bb/5efa301486ad236777d15b515158224cb17ca4e1f138e1480ce8a9d5c369/opentelemetry_instrumentation_cohere-0.34.0-py3-none-any.whl", hash = "sha256:6238c84948d809ea5feb1ce603de2c8f9d72d7b8286d9f9115edf33b91202011", size = 5576, upload-time = "2024-12-12T21:01:20.273Z" }, ] [[package]] name = "opentelemetry-instrumentation-fastapi" -version = "0.49b0" +version = "0.54b1" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "opentelemetry-api" }, @@ -6342,14 +6343,14 @@ dependencies = [ { name = "opentelemetry-semantic-conventions" }, { name = "opentelemetry-util-http" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/fe/bf/8e6d2a4807360f2203192017eb4845f5628dbeaf0597adf3d141cc5c24e1/opentelemetry_instrumentation_fastapi-0.49b0.tar.gz", hash = "sha256:6d14935c41fd3e49328188b6a59dd4c37bd17a66b01c15b0c64afa9714a1f905", size = 19230, upload-time = "2024-11-05T19:21:59.361Z" } +sdist = { url = "https://files.pythonhosted.org/packages/98/3b/9a262cdc1a4defef0e52afebdde3e8add658cc6f922e39e9dcee0da98349/opentelemetry_instrumentation_fastapi-0.54b1.tar.gz", hash = "sha256:1fcad19cef0db7092339b571a59e6f3045c9b58b7fd4670183f7addc459d78df", size = 19325, upload-time = "2025-05-16T19:03:45.359Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/b1/f4/0895b9410c10abf987c90dee1b7688a8f2214a284fe15e575648f6a1473a/opentelemetry_instrumentation_fastapi-0.49b0-py3-none-any.whl", hash = "sha256:646e1b18523cbe6860ae9711eb2c7b9c85466c3c7697cd6b8fb5180d85d3fe6e", size = 12101, upload-time = "2024-11-05T19:21:01.805Z" }, + { url = "https://files.pythonhosted.org/packages/df/9c/6b2b0f9d6c5dea7528ae0bf4e461dd765b0ae35f13919cd452970bb0d0b3/opentelemetry_instrumentation_fastapi-0.54b1-py3-none-any.whl", hash = "sha256:fb247781cfa75fd09d3d8713c65e4a02bd1e869b00e2c322cc516d4b5429860c", size = 12125, upload-time = "2025-05-16T19:02:41.172Z" }, ] [[package]] name = "opentelemetry-instrumentation-google-generativeai" -version = "0.33.12" +version = "0.34.0" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "opentelemetry-api" }, @@ -6357,14 +6358,14 @@ dependencies = [ { name = "opentelemetry-semantic-conventions" }, { name = "opentelemetry-semantic-conventions-ai" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/b0/39/d33585303893fec6d4e828b794b8b26188658e3bd905098e568031eb0698/opentelemetry_instrumentation_google_generativeai-0.33.12.tar.gz", hash = "sha256:9d09cd39afecf70063733b3f2f15200b7dc28addfa6384947a9514557f18d64b", size = 4302, upload-time = "2024-11-13T20:27:56.179Z" } +sdist = { url = "https://files.pythonhosted.org/packages/30/c8/4620090d09b3d450ac7069ad84b366b34c2488df290c7fc0af6582178812/opentelemetry_instrumentation_google_generativeai-0.34.0.tar.gz", hash = "sha256:b0ecc9cb840277d4040277158c4d77a48c171a64ca556c54ddc1c5ce5105ebd8", size = 4288, upload-time = "2024-12-12T21:02:10.891Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/11/9a/622ca1552d05b5b948c1f0e78d8456464697118c63a58d4f5aa01c105d45/opentelemetry_instrumentation_google_generativeai-0.33.12-py3-none-any.whl", hash = "sha256:0dcd71c38331c47663d7ba6237ddfe02c14e3d1e3a47524c1437e3ee56cd0036", size = 5889, upload-time = "2024-11-13T20:27:22.37Z" }, + { url = "https://files.pythonhosted.org/packages/37/6e/e20b5fce0020a1f3de78227610a7d018764e9b26e8766c4f948493c2485e/opentelemetry_instrumentation_google_generativeai-0.34.0-py3-none-any.whl", hash = "sha256:eb42d8d48e3d13e03363932b69f424d27e8d9a53c8cbd23f190c4a294a881edc", size = 5879, upload-time = "2024-12-12T21:01:22.735Z" }, ] [[package]] name = "opentelemetry-instrumentation-groq" -version = "0.33.12" +version = "0.34.0" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "opentelemetry-api" }, @@ -6372,14 +6373,14 @@ dependencies = [ { name = "opentelemetry-semantic-conventions" }, { name = "opentelemetry-semantic-conventions-ai" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/fa/39/71c87d595a312e2cfef83b006070e5d56c73895c59b596336a20aa43e79c/opentelemetry_instrumentation_groq-0.33.12.tar.gz", hash = "sha256:1460901e66c87b47198d639fb22ec25552281cdf7cafe13ae9605447661d6871", size = 5687, upload-time = "2024-11-13T20:28:00.703Z" } +sdist = { url = "https://files.pythonhosted.org/packages/af/1d/443944e52fc37a5e564525134dd86bee6a3f2db7be1c08f6459f056965ad/opentelemetry_instrumentation_groq-0.34.0.tar.gz", hash = "sha256:0c9162ce1a7b5b5a613dbf50f5f2ee8d5e6e175e0cc1758d53d71cb22c7aac1b", size = 5670, upload-time = "2024-12-12T21:02:12.256Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/f7/24/631269741eabb0b028a313f15063871b28e700ba27feece154a3dd71f62d/opentelemetry_instrumentation_groq-0.33.12-py3-none-any.whl", hash = "sha256:4d239c73d689c046ab2c90a25b78d6c7406cef1e26f04633bc148464b66cc74c", size = 7270, upload-time = "2024-11-13T20:27:23.508Z" }, + { url = "https://files.pythonhosted.org/packages/a3/81/beb464fdd0d3f568b589b45629f74e0fb1a1e518a9df2f575bb68ea2096a/opentelemetry_instrumentation_groq-0.34.0-py3-none-any.whl", hash = "sha256:0f74c8b0df2984b27aadabebf3bed4443c0db45fbd851f956299207be12bb207", size = 7252, upload-time = "2024-12-12T21:01:24.069Z" }, ] [[package]] name = "opentelemetry-instrumentation-haystack" -version = "0.33.12" +version = "0.34.0" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "opentelemetry-api" }, @@ -6387,14 +6388,14 @@ dependencies = [ { name = "opentelemetry-semantic-conventions" }, { name = "opentelemetry-semantic-conventions-ai" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/9c/06/067e4b2db2bc29d0a7e3a6cc8676d5f1971b0ecbaf7e5fa0c1e478e092af/opentelemetry_instrumentation_haystack-0.33.12.tar.gz", hash = "sha256:3d45df14aff1f2321066e55ecce632653d67c36249d3eaccbefa189f0daaba05", size = 4663, upload-time = "2024-11-13T20:28:01.551Z" } +sdist = { url = "https://files.pythonhosted.org/packages/c3/61/2aa5d850c1891fd99636a4ad724489ed792ac4aa560be75ab34af0ee26eb/opentelemetry_instrumentation_haystack-0.34.0.tar.gz", hash = "sha256:29739e9429a1a327dc72f743a0b37a3b7f26a742ac762791a75b1bc2f3ba43ff", size = 4645, upload-time = "2024-12-12T21:02:13.193Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/dc/ba/b8872dce7eb67bd6589d4cccfc97554851dad719067b24c7697a5ffef69a/opentelemetry_instrumentation_haystack-0.33.12-py3-none-any.whl", hash = "sha256:d2a3041a58e1027d8728e1a24430f7b01e8c27e9c04c22fa4204c590b12a95f6", size = 7513, upload-time = "2024-11-13T20:27:24.671Z" }, + { url = "https://files.pythonhosted.org/packages/58/15/682dfc4717e4ddbb668fdcb5a12a8b22a2f6c9402d78c26690528722e8e5/opentelemetry_instrumentation_haystack-0.34.0-py3-none-any.whl", hash = "sha256:2ae56f4abc7a2bafad7b2b3ec8e218edf2aa0daaa6570c692076c24682ee78ce", size = 7495, upload-time = "2024-12-12T21:01:25.723Z" }, ] [[package]] name = "opentelemetry-instrumentation-lancedb" -version = "0.33.12" +version = "0.34.0" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "opentelemetry-api" }, @@ -6402,14 +6403,14 @@ dependencies = [ { name = "opentelemetry-semantic-conventions" }, { name = "opentelemetry-semantic-conventions-ai" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/38/52/16eef8e5c92627a82904f0112715ee95b2a6ee74ec958ec69f7db803a8d5/opentelemetry_instrumentation_lancedb-0.33.12.tar.gz", hash = "sha256:0aa9f6319374f532e2087949c15674f4d8036591ba70f91f9ce6996ea34508c3", size = 3198, upload-time = "2024-11-13T20:28:02.455Z" } +sdist = { url = "https://files.pythonhosted.org/packages/05/00/ad6383e2981308146e282da4d977ed61dd63321c6ac72751aa1c9eb26d74/opentelemetry_instrumentation_lancedb-0.34.0.tar.gz", hash = "sha256:5d081f36335d7b5dd3a8ae3b0fac0b895f4284941e3521f32332d3393b3b1178", size = 3185, upload-time = "2024-12-12T21:02:14.193Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/26/1d/218b74341471aa370999f7dbee09de44e32d8f822d69b6ca6d95f400be36/opentelemetry_instrumentation_lancedb-0.33.12-py3-none-any.whl", hash = "sha256:e1cdd55ef38d939d8af924478486e66c0cf65a7e5ac19c82f20ad5d06e682b9d", size = 4794, upload-time = "2024-11-13T20:27:25.825Z" }, + { url = "https://files.pythonhosted.org/packages/5b/65/db706f845a5ab861ee59feb6eb394843e27bf0f025bc1438e44a7af19f19/opentelemetry_instrumentation_lancedb-0.34.0-py3-none-any.whl", hash = "sha256:b8284453cb3d98fbe83bd286448eca4edbb779fc79ffc58bdb3a344137d82719", size = 4780, upload-time = "2024-12-12T21:01:26.96Z" }, ] [[package]] name = "opentelemetry-instrumentation-langchain" -version = "0.33.12" +version = "0.34.0" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "opentelemetry-api" }, @@ -6417,14 +6418,14 @@ dependencies = [ { name = "opentelemetry-semantic-conventions" }, { name = "opentelemetry-semantic-conventions-ai" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/73/60/4fb638bc69cc63bbf7aad81a08650c99bd343a67f49c532261190e7ee7e4/opentelemetry_instrumentation_langchain-0.33.12.tar.gz", hash = "sha256:ff607742c76a1844211648415fa35da9eac22a42da2a9732c673bd13f2973994", size = 8518, upload-time = "2024-11-13T20:28:03.247Z" } +sdist = { url = "https://files.pythonhosted.org/packages/ce/b4/a8bafbc727a874eb26e788b1fe667db85c7dbcb6d685a6b9da07f6ba231b/opentelemetry_instrumentation_langchain-0.34.0.tar.gz", hash = "sha256:2a25bc07ff8719d30b9a01acf29305c7de5418683c14334ad7ddef4608222911", size = 8508, upload-time = "2024-12-12T21:02:15.138Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/b1/fe/215a5b5b52360c94b2223f4dfb339665a60838d8f53edd37dd1c40897de4/opentelemetry_instrumentation_langchain-0.33.12-py3-none-any.whl", hash = "sha256:7406ab7116fa43343f53602f7b530f9bb1552e20ad750dbfc5aa1761c027c2d3", size = 9749, upload-time = "2024-11-13T20:27:27.623Z" }, + { url = "https://files.pythonhosted.org/packages/a1/3f/01f5d6e5fc3e34e068b6ad650bde73facb9748df74de305a33702ad06820/opentelemetry_instrumentation_langchain-0.34.0-py3-none-any.whl", hash = "sha256:373c69adcf18e9d37cd47d96fad78c57959c3f8af7034aff50553103fdbf0ba8", size = 9734, upload-time = "2024-12-12T21:01:28.244Z" }, ] [[package]] name = "opentelemetry-instrumentation-llamaindex" -version = "0.33.12" +version = "0.34.0" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "inflection" }, @@ -6433,27 +6434,27 @@ dependencies = [ { name = "opentelemetry-semantic-conventions" }, { name = "opentelemetry-semantic-conventions-ai" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/df/e7/9b9d43c7b78eea5ecf95b378ed4c12850f8745ff5b46ac5fa9042c89c941/opentelemetry_instrumentation_llamaindex-0.33.12.tar.gz", hash = "sha256:7a278dfe21fbba7dd1b8fe824c9baee0bfb3b4f7ccd71aae5f677412be45587e", size = 9285, upload-time = "2024-11-13T20:28:04.082Z" } +sdist = { url = "https://files.pythonhosted.org/packages/06/fe/b73490ee120672c81f78209a787feb1a5fbf19f2ec0657cf9b85277597ae/opentelemetry_instrumentation_llamaindex-0.34.0.tar.gz", hash = "sha256:f84eaa198873e856401fd8382f86d4f099e8edd712369579f4b74c24e0404933", size = 9274, upload-time = "2024-12-12T21:02:17.055Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/d6/6a/aef813dff690cf06c62a86bf3f723ab1331b55f7c3dfd56690e93ac993df/opentelemetry_instrumentation_llamaindex-0.33.12-py3-none-any.whl", hash = "sha256:7f0d0700015f1e1576cf2de211a4062ae2d0ea899c298f4d7d44ce2a37226135", size = 16372, upload-time = "2024-11-13T20:27:28.724Z" }, + { url = "https://files.pythonhosted.org/packages/71/38/01d81a1bae3965031d612afe162a4fdee40d181b46d0f2aab7d2ac49d015/opentelemetry_instrumentation_llamaindex-0.34.0-py3-none-any.whl", hash = "sha256:0058a44a584ccb9046bed3d5da7bb64160c51d46f0a9946d2bb6517ffcd29fd0", size = 16354, upload-time = "2024-12-12T21:01:32.806Z" }, ] [[package]] name = "opentelemetry-instrumentation-logging" -version = "0.49b0" +version = "0.54b1" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "opentelemetry-api" }, { name = "opentelemetry-instrumentation" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/c8/80/1d15f8afebc2b67ed47bfe45ee97c042808441586617d5aea8df1f1cbd96/opentelemetry_instrumentation_logging-0.49b0.tar.gz", hash = "sha256:d8058216b06c029785113a71428c6edbb3f0e3b9f69ee917050cb98cd8137fb2", size = 9731, upload-time = "2024-11-05T19:22:05.252Z" } +sdist = { url = "https://files.pythonhosted.org/packages/d9/5b/88ed39f22e8c6eb4f6192ab9a62adaa115579fcbcadb3f0241ee645eea56/opentelemetry_instrumentation_logging-0.54b1.tar.gz", hash = "sha256:893a3cbfda893b64ff71b81991894e2fd6a9267ba85bb6c251f51c0419fbe8fa", size = 9976, upload-time = "2025-05-16T19:03:49.976Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/a7/c4/0eedcf9ccce07a64baa002fae7001d84f2c032cf5b2ff1a9438bff0479dd/opentelemetry_instrumentation_logging-0.49b0-py3-none-any.whl", hash = "sha256:9f9405d2f8e6fd756d49da979710f7b5ba1b95bd534467f176aae756102eed58", size = 12150, upload-time = "2024-11-05T19:21:07.826Z" }, + { url = "https://files.pythonhosted.org/packages/96/0c/b441fb30d860f25040eaed61e89d68f4d9ee31873159ed18cbc1b92eba56/opentelemetry_instrumentation_logging-0.54b1-py3-none-any.whl", hash = "sha256:01a4cec54348f13941707d857b850b0febf9d49f45d0fcf0673866e079d7357b", size = 12579, upload-time = "2025-05-16T19:02:49.039Z" }, ] [[package]] name = "opentelemetry-instrumentation-marqo" -version = "0.33.12" +version = "0.34.0" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "opentelemetry-api" }, @@ -6461,14 +6462,14 @@ dependencies = [ { name = "opentelemetry-semantic-conventions" }, { name = "opentelemetry-semantic-conventions-ai" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/82/fb/775a1a5f9b9f641b3c7aa7ea8a7d83cbb97a4ddf86e1c5a4dd2a3c42af8d/opentelemetry_instrumentation_marqo-0.33.12.tar.gz", hash = "sha256:802def00b35033055618dc137f81895496bb449ff405580ff9414eeda139b89f", size = 3479, upload-time = "2024-11-13T20:28:04.892Z" } +sdist = { url = "https://files.pythonhosted.org/packages/e7/50/2585b0d337a15b7fe31ec0e7245c09154b7d3d7e7e172c3973d40ad313ee/opentelemetry_instrumentation_marqo-0.34.0.tar.gz", hash = "sha256:7bcc091b89717ac7b04c224dfc1429f200ba2b3e930d7a4de80bf9bc054fc0db", size = 3471, upload-time = "2024-12-12T21:02:17.979Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/5b/b1/153592356d8cc6faab61f7e5c727ab3c382984cede3fbdc228872ba5ac7c/opentelemetry_instrumentation_marqo-0.33.12-py3-none-any.whl", hash = "sha256:6f939532f1f953a22eb2811dfdb439bb8bd813182d59d3444dcc4eca46d0805f", size = 5091, upload-time = "2024-11-13T20:27:31.133Z" }, + { url = "https://files.pythonhosted.org/packages/d4/51/9be4f5df62db6ff6e786933136541e3534656bb49a7d35324d96a5c07818/opentelemetry_instrumentation_marqo-0.34.0-py3-none-any.whl", hash = "sha256:dd342cfd4b70d4f65830708bc253397734d3da51d3773b677693e9007217e3ed", size = 5077, upload-time = "2024-12-12T21:01:35.643Z" }, ] [[package]] name = "opentelemetry-instrumentation-milvus" -version = "0.33.12" +version = "0.34.0" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "opentelemetry-api" }, @@ -6476,14 +6477,14 @@ dependencies = [ { name = "opentelemetry-semantic-conventions" }, { name = "opentelemetry-semantic-conventions-ai" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/58/be/d86538d7b09c6ed77f137229ba418e402659e4aa9268ffff696e059ec8fb/opentelemetry_instrumentation_milvus-0.33.12.tar.gz", hash = "sha256:8720e8fd29ea3009dd0e5b8849b1d24657a5750d81f6c1af605e8aabe827f742", size = 3666, upload-time = "2024-11-13T20:28:06.004Z" } +sdist = { url = "https://files.pythonhosted.org/packages/3b/a8/18725e95cb5cf0d01c001698aa4b01198b5f205da4bb98acc8e0b0da1b6c/opentelemetry_instrumentation_milvus-0.34.0.tar.gz", hash = "sha256:6c19aa93c392f5c736390320b27e035761f36ead904277943d62ac6662d77f83", size = 3657, upload-time = "2024-12-12T21:02:18.849Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/6d/22/0972d94433358624e8f956228763650a2366921d4659035b260aad9775aa/opentelemetry_instrumentation_milvus-0.33.12-py3-none-any.whl", hash = "sha256:608783fa555aded64606cfda64f4aa6ace5c2f9790b2b920e114ca229ab00915", size = 5311, upload-time = "2024-11-13T20:27:32.219Z" }, + { url = "https://files.pythonhosted.org/packages/7b/fb/685282b0e0339d629d4fdb03af7d2904461f20c128ae88fa148847e8664a/opentelemetry_instrumentation_milvus-0.34.0-py3-none-any.whl", hash = "sha256:4c587c6031bc82d78189b31f6acd4f36a62ce3ae1f2b18bc7fabb667912cc2d7", size = 5294, upload-time = "2024-12-12T21:01:38.115Z" }, ] [[package]] name = "opentelemetry-instrumentation-mistralai" -version = "0.33.12" +version = "0.34.0" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "opentelemetry-api" }, @@ -6491,14 +6492,14 @@ dependencies = [ { name = "opentelemetry-semantic-conventions" }, { name = "opentelemetry-semantic-conventions-ai" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/f6/78/5cab3468d3885cc391f67ef0a4a845abb085aa8d786aaa565b9cd9c6612f/opentelemetry_instrumentation_mistralai-0.33.12.tar.gz", hash = "sha256:c22f7006a56180ab6384e47b4e49bde8597833f73955e48ac323cdbe107f06ae", size = 4383, upload-time = "2024-11-13T20:28:09.179Z" } +sdist = { url = "https://files.pythonhosted.org/packages/49/0e/3d86aa6b5a31a20ecadbd4423e83255fd09d9648f431ccc786665e0f98be/opentelemetry_instrumentation_mistralai-0.34.0.tar.gz", hash = "sha256:7c81d8602a16b37d698002a7b06233095fe5c17ddf2f0b9d973b78255cdf7547", size = 4387, upload-time = "2024-12-12T21:02:19.71Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/12/c8/f47d404273e7130f4e4360f33a093d73f8fbbf232ef536a761baeb91027c/opentelemetry_instrumentation_mistralai-0.33.12-py3-none-any.whl", hash = "sha256:66c5961a33492aaf4420ed1ab8c63e533162a06366108c3398c461d05b2a1154", size = 5858, upload-time = "2024-11-13T20:27:33.251Z" }, + { url = "https://files.pythonhosted.org/packages/16/c8/5644b1a821b60a34bebc58f96367571bcdcdf5ab1522137e13ae3a936360/opentelemetry_instrumentation_mistralai-0.34.0-py3-none-any.whl", hash = "sha256:f682b8d4011124fa326308e8fc4ce9e9fdbacfc72fc77431682d2ef950e636d8", size = 5842, upload-time = "2024-12-12T21:01:39.204Z" }, ] [[package]] name = "opentelemetry-instrumentation-ollama" -version = "0.33.12" +version = "0.34.0" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "opentelemetry-api" }, @@ -6506,14 +6507,14 @@ dependencies = [ { name = "opentelemetry-semantic-conventions" }, { name = "opentelemetry-semantic-conventions-ai" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/d9/aa/e9c0f903b8ae750794688f82a00ac0b7ab00a42e57d7c279738b5a80a0ce/opentelemetry_instrumentation_ollama-0.33.12.tar.gz", hash = "sha256:4cd012503f8d692453645353231e216c756fc926bdd3142e8c97fc8e87cbe06f", size = 4512, upload-time = "2024-11-13T20:28:10.969Z" } +sdist = { url = "https://files.pythonhosted.org/packages/ff/f2/4c8e16bb5b13a85d86f6a3c515bee05051dcc08f8023dd201a69c9c0580f/opentelemetry_instrumentation_ollama-0.34.0.tar.gz", hash = "sha256:c9cabfac35945eb9b167f174a9fcafe82ec7c70ae1ba04d462486d2ef4c20f70", size = 4491, upload-time = "2024-12-12T21:02:20.656Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/30/58/5f11976bc5fde11390e709d96c757e1b21ff73faf88d5b0fd97ace56b061/opentelemetry_instrumentation_ollama-0.33.12-py3-none-any.whl", hash = "sha256:ed5313f45f5d46e17d93096eba91ed6338abf535cf2d2eca648d0e2d621e9d6b", size = 5847, upload-time = "2024-11-13T20:27:35.323Z" }, + { url = "https://files.pythonhosted.org/packages/16/3b/1547e92c76b9dd3097a98a67a84b0641ecbba3348f07d5825ecfa3433c7d/opentelemetry_instrumentation_ollama-0.34.0-py3-none-any.whl", hash = "sha256:17beea413c78be8510409aa4b5a5f909ba9e9d14799fd6372b16d84cefb21120", size = 5832, upload-time = "2024-12-12T21:01:40.792Z" }, ] [[package]] name = "opentelemetry-instrumentation-openai" -version = "0.33.12" +version = "0.34.0" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "opentelemetry-api" }, @@ -6522,14 +6523,14 @@ dependencies = [ { name = "opentelemetry-semantic-conventions-ai" }, { name = "tiktoken" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/10/ec/2f9bb0a22ba916c10b2ef63ccde48f17348c49c2a651b8590a94076308e8/opentelemetry_instrumentation_openai-0.33.12.tar.gz", hash = "sha256:2c6dfd74d9d56ca393f9dbfc92883c7397d63408ff18b3d9a774ea1611a48ed9", size = 14631, upload-time = "2024-11-13T20:28:11.767Z" } +sdist = { url = "https://files.pythonhosted.org/packages/b2/9a/04bb865c14d44111ccde056ffa994d5c29ee604c286d6248b2365f53d676/opentelemetry_instrumentation_openai-0.34.0.tar.gz", hash = "sha256:67fabd6b178837c3d115296654a0daaebeeec763789e3f7ffd9a3db6117b354e", size = 14967, upload-time = "2024-12-12T21:02:22.935Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/c5/21/c4a2b70e9f3487ba7123fde8c55090ff4a3f227477261fac1d0b73d6d349/opentelemetry_instrumentation_openai-0.33.12-py3-none-any.whl", hash = "sha256:d5d0c83a469dbf7ab97d1c482ce78f7ba23c00015b01bc8be43cdc0e5d7c497f", size = 22089, upload-time = "2024-11-13T20:27:36.375Z" }, + { url = "https://files.pythonhosted.org/packages/fe/08/6b3c0404d53a2ca913a98fecb4228be9238965cf2f9092acf5c3e960cba0/opentelemetry_instrumentation_openai-0.34.0-py3-none-any.whl", hash = "sha256:22e902b1b830ca53a0a94ec523880a4d39a210e4ec34d0ce76605b726eed1aab", size = 22597, upload-time = "2024-12-12T21:01:43.669Z" }, ] [[package]] name = "opentelemetry-instrumentation-pinecone" -version = "0.33.12" +version = "0.34.0" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "opentelemetry-api" }, @@ -6537,14 +6538,14 @@ dependencies = [ { name = "opentelemetry-semantic-conventions" }, { name = "opentelemetry-semantic-conventions-ai" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/9a/2f/f9e0d1d5eb6f3eee791dbeac13185f4b77bbdce774a01011e03a5d8e2a71/opentelemetry_instrumentation_pinecone-0.33.12.tar.gz", hash = "sha256:92ed3221bddb061ebe7f50cd4804c76c9f5d019e2afb967858c1be62c1a3ebf2", size = 4651, upload-time = "2024-11-13T20:28:14.402Z" } +sdist = { url = "https://files.pythonhosted.org/packages/0e/a0/35791b83b78157f4bfb2f7d9c356a1f42a6ffc30f17f2e96210dabe090f7/opentelemetry_instrumentation_pinecone-0.34.0.tar.gz", hash = "sha256:573483686da9fd2be48c6de870b87515e479d6ee489ebc471d7c90e0de4106e1", size = 4649, upload-time = "2024-12-12T21:02:23.857Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/32/36/8458e916a2cca0378ac28e6d70347d5263160600fcb47b131f029bdd390a/opentelemetry_instrumentation_pinecone-0.33.12-py3-none-any.whl", hash = "sha256:8d6185bd2f5bf34f3983cad48bbaa86fc72925c02a07ab4a79eb805401556b19", size = 6377, upload-time = "2024-11-13T20:27:38.177Z" }, + { url = "https://files.pythonhosted.org/packages/9b/39/0f09e3de4fa72f17438a1ab7f2a8693a9c983a1e1ad6bf6e72c590616e4a/opentelemetry_instrumentation_pinecone-0.34.0-py3-none-any.whl", hash = "sha256:d81387e703bfd59ff03ed46acf0bf6a0c11cf4cdec16a440acbdea18987fda71", size = 6363, upload-time = "2024-12-12T21:01:46.926Z" }, ] [[package]] name = "opentelemetry-instrumentation-qdrant" -version = "0.33.12" +version = "0.34.0" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "opentelemetry-api" }, @@ -6552,14 +6553,14 @@ dependencies = [ { name = "opentelemetry-semantic-conventions" }, { name = "opentelemetry-semantic-conventions-ai" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/1e/0a/216d28c48dc8b9e37094c54cd1e3ef9a43609fc25b2eeb8bd939f0026047/opentelemetry_instrumentation_qdrant-0.33.12.tar.gz", hash = "sha256:ba34c6863c652f27ae28b9922b25d77617ece4a2233ad0c9c1ebd257605853a5", size = 3988, upload-time = "2024-11-13T20:28:18.929Z" } +sdist = { url = "https://files.pythonhosted.org/packages/da/03/bbf02439ba6c6077ac814957695846bc44edc4e631fce8f0cbb792aa5572/opentelemetry_instrumentation_qdrant-0.34.0.tar.gz", hash = "sha256:8d569b2d7ac70bbf7e75abe5f572ff9576fa175660d1ecdc98f60c6ea1d7010b", size = 3977, upload-time = "2024-12-12T21:02:24.795Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/d2/dc/caa6f4951c84ecb11594ab8545a8692c8a166d618a67687e7f07694dcd97/opentelemetry_instrumentation_qdrant-0.33.12-py3-none-any.whl", hash = "sha256:e759fe49c67092197eaa547570d67e15f30675bfdc772839d0456d535297d4c0", size = 6317, upload-time = "2024-11-13T20:27:39.638Z" }, + { url = "https://files.pythonhosted.org/packages/2f/fe/1797190a4a6b81b50a5c83615590929a21775bd4cd8a855102703243fd51/opentelemetry_instrumentation_qdrant-0.34.0-py3-none-any.whl", hash = "sha256:34ef85e62f3039a2b61a68c6decdb9e5d05ab9f7303d08fc010d6d5ec8f144e0", size = 6302, upload-time = "2024-12-12T21:01:48.042Z" }, ] [[package]] name = "opentelemetry-instrumentation-replicate" -version = "0.33.12" +version = "0.34.0" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "opentelemetry-api" }, @@ -6567,14 +6568,14 @@ dependencies = [ { name = "opentelemetry-semantic-conventions" }, { name = "opentelemetry-semantic-conventions-ai" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/89/1e/b4260513c9526b2113772e4bf10bee714555abfb6e7cf69f86326e5607d5/opentelemetry_instrumentation_replicate-0.33.12.tar.gz", hash = "sha256:5dafad1a7a20ba762f689f30c4f76bcb3817b617adb7da3288ac545d15a14565", size = 3767, upload-time = "2024-11-13T20:28:19.766Z" } +sdist = { url = "https://files.pythonhosted.org/packages/0e/98/a36a16df6876396962071d8fbc6f0dc1c81bc1fdcb324e72871683b83e51/opentelemetry_instrumentation_replicate-0.34.0.tar.gz", hash = "sha256:124796ff8593cd211bfa05773f70e8f087a8c0522a544be39bed212a95c8dec3", size = 3767, upload-time = "2024-12-12T21:02:25.678Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/4d/6d/20dab686dce1ce491c97ea4a0feec86b9a2207891a2627b1f4a670d109b5/opentelemetry_instrumentation_replicate-0.33.12-py3-none-any.whl", hash = "sha256:dc527f470080248a57b738b63ed29eae2d82ef68a100fe6e3548f5de7677f2ed", size = 5189, upload-time = "2024-11-13T20:27:40.792Z" }, + { url = "https://files.pythonhosted.org/packages/69/4b/cb70ab819ec045c2e494deea99a542e4819c9d1c5f09ec99d6dffacb00ad/opentelemetry_instrumentation_replicate-0.34.0-py3-none-any.whl", hash = "sha256:c5f3d712702f3cbcfde619d08e83b1c2fd70e4ad36190d68575d576e27370c4d", size = 5175, upload-time = "2024-12-12T21:01:49.409Z" }, ] [[package]] name = "opentelemetry-instrumentation-requests" -version = "0.49b0" +version = "0.54b1" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "opentelemetry-api" }, @@ -6582,14 +6583,14 @@ dependencies = [ { name = "opentelemetry-semantic-conventions" }, { name = "opentelemetry-util-http" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/c1/16/c71196d8f4cac30b6936c77567ae769f44ac97227255627f5277d825277d/opentelemetry_instrumentation_requests-0.49b0.tar.gz", hash = "sha256:b75a282b3641547272dc7d2fdc0dd68269d0c1e685e4d17579b7fbd34c19b6bb", size = 14123, upload-time = "2024-11-05T19:22:14.128Z" } +sdist = { url = "https://files.pythonhosted.org/packages/e7/45/116da84930d3dc2f5cdd876283ca96e9b96547bccee7eaa0bd01ce6bf046/opentelemetry_instrumentation_requests-0.54b1.tar.gz", hash = "sha256:3eca5d697c5564af04c6a1dd23b6a3ffbaf11e64887c6051655cee03998f4654", size = 15148, upload-time = "2025-05-16T19:04:00.488Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/79/33/4b8a4a839401290c44c65a8ca926a60a86c5ee3ecdcf54de4575c288b5ac/opentelemetry_instrumentation_requests-0.49b0-py3-none-any.whl", hash = "sha256:bb39803359e226b8eb0d4c8aaba6fd8a883a7f869fc331ff861743173b33d26d", size = 12368, upload-time = "2024-11-05T19:21:22.387Z" }, + { url = "https://files.pythonhosted.org/packages/2b/b1/6e33d2c3d3cc9e3ae20a9a77625ec81a509a0e5d7fa87e09e7f879468990/opentelemetry_instrumentation_requests-0.54b1-py3-none-any.whl", hash = "sha256:a0c4cd5d946224f336d6bd73cdabdecc6f80d5c39208f84eb96eb15f16cd41a0", size = 12968, upload-time = "2025-05-16T19:03:03.131Z" }, ] [[package]] name = "opentelemetry-instrumentation-sagemaker" -version = "0.33.12" +version = "0.34.0" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "opentelemetry-api" }, @@ -6597,14 +6598,14 @@ dependencies = [ { name = "opentelemetry-semantic-conventions" }, { name = "opentelemetry-semantic-conventions-ai" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/c2/97/2ddbceba0f95f9b28e7ed75ee2d38214ea1e7d071585d9afea35d0e71619/opentelemetry_instrumentation_sagemaker-0.33.12.tar.gz", hash = "sha256:286bb0e7765967212e111274ca523084d8105a3f18d1dfc90873bca60f6ad766", size = 4508, upload-time = "2024-11-13T20:28:21.829Z" } +sdist = { url = "https://files.pythonhosted.org/packages/e6/98/fc4a33b8800a4a774c50a4b3da83a95359b89b3744c259b2ebbafc3b2f3a/opentelemetry_instrumentation_sagemaker-0.34.0.tar.gz", hash = "sha256:b7c2be5ba9ea4f4b9705705cddc1c3474cf4cb4e6db9fdf6968ad97ec8e6f1df", size = 4506, upload-time = "2024-12-12T21:02:26.652Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/c3/f3/21896275fb1b4954082c4b95277d8ce66d6e947f1c153b0f01182e9a852f/opentelemetry_instrumentation_sagemaker-0.33.12-py3-none-any.whl", hash = "sha256:da72e78a094106c3ce48e2410665016f161211976c577e92b4624dfbbc54e47e", size = 6296, upload-time = "2024-11-13T20:27:41.815Z" }, + { url = "https://files.pythonhosted.org/packages/89/c2/b60f211e51b3c8346073dde33e7053ba1027b943da50d34ec6f00afe7d78/opentelemetry_instrumentation_sagemaker-0.34.0-py3-none-any.whl", hash = "sha256:ed7a50a5a863bfc81bc792fd3bc7b33bbf0af9e279b6e527c79e93034deda1a0", size = 6282, upload-time = "2024-12-12T21:01:50.527Z" }, ] [[package]] name = "opentelemetry-instrumentation-sqlalchemy" -version = "0.49b0" +version = "0.54b1" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "opentelemetry-api" }, @@ -6613,28 +6614,28 @@ dependencies = [ { name = "packaging" }, { name = "wrapt" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/a0/a7/24f6cce3808ae1802dd1b60d752fbab877db5655198929cf4ee8ea416923/opentelemetry_instrumentation_sqlalchemy-0.49b0.tar.gz", hash = "sha256:32658e520fc8b35823c722f5d8831d3a410b76dd2724adb2887befc041ddef04", size = 13194, upload-time = "2024-11-05T19:22:14.92Z" } +sdist = { url = "https://files.pythonhosted.org/packages/ac/33/78a25ae4233d42058bb0b363ba4fea7d7210e53c24e5e31f16d5cf6cf957/opentelemetry_instrumentation_sqlalchemy-0.54b1.tar.gz", hash = "sha256:97839acf1c9b96ded857fca57a09b86a56cf8d9eb6d706b7ceaee9352a460e03", size = 14620, upload-time = "2025-05-16T19:04:01.215Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/ec/6b/a1a3685fed593282999cdc374ece15efbd56f8d774bd368bf7ff2cf5923c/opentelemetry_instrumentation_sqlalchemy-0.49b0-py3-none-any.whl", hash = "sha256:d854052d2b02cd0562e5628a514c8153fceada7f585137e173165dfd0a46ef6a", size = 13358, upload-time = "2024-11-05T19:21:23.654Z" }, + { url = "https://files.pythonhosted.org/packages/c7/2b/1c954885815614ef5c1e8c7bbf57a5275e64cd6fb5946b65e17162a34037/opentelemetry_instrumentation_sqlalchemy-0.54b1-py3-none-any.whl", hash = "sha256:d2ca5edb4c7ecef120d51aad6793b7da1cc80207ccfd31c437ee18f098e7c4c4", size = 14169, upload-time = "2025-05-16T19:03:04.119Z" }, ] [[package]] name = "opentelemetry-instrumentation-threading" -version = "0.49b0" +version = "0.54b1" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "opentelemetry-api" }, { name = "opentelemetry-instrumentation" }, { name = "wrapt" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/80/88/b19f064ebf1650a7291cb7fcb623129997a7d8af603ffe7cd1907fe469ba/opentelemetry_instrumentation_threading-0.49b0.tar.gz", hash = "sha256:b65ec668a3ee73fccb1432edf52556f374cb9d9e5b160a6da3a6f67890adf444", size = 8283, upload-time = "2024-11-05T19:22:18.778Z" } +sdist = { url = "https://files.pythonhosted.org/packages/a0/bd/561245292e7cc78ac7a0a75537873aea87440cb9493d41371421b3308c2b/opentelemetry_instrumentation_threading-0.54b1.tar.gz", hash = "sha256:3a081085b59675baf7bd93126a681903e6304a5f283df5eaecdd44bcb66df578", size = 8774, upload-time = "2025-05-16T19:04:04.482Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/e1/77/cf262caae1a8903bbe9c379dc6908fddc9f7bbd5c51866d7c7fbae2edb70/opentelemetry_instrumentation_threading-0.49b0-py3-none-any.whl", hash = "sha256:47a49931a2244c2b17db985c512e6c922328b891ff2b64d37b0cd3bd00fd00a9", size = 9072, upload-time = "2024-11-05T19:21:29.564Z" }, + { url = "https://files.pythonhosted.org/packages/81/10/d87ec07d69546adaad525ba5d40d27324a45cba29097d9854a53d9af5047/opentelemetry_instrumentation_threading-0.54b1-py3-none-any.whl", hash = "sha256:bc229e6cd3f2b29fafe0a8dd3141f452e16fcb4906bca4fbf52609f99fb1eb42", size = 9314, upload-time = "2025-05-16T19:03:09.527Z" }, ] [[package]] name = "opentelemetry-instrumentation-together" -version = "0.33.12" +version = "0.34.0" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "opentelemetry-api" }, @@ -6642,14 +6643,14 @@ dependencies = [ { name = "opentelemetry-semantic-conventions" }, { name = "opentelemetry-semantic-conventions-ai" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/7f/5f/63ee7efc3de97e12eadc927ac4079caef454ee41f8e608bf7a83734024a9/opentelemetry_instrumentation_together-0.33.12.tar.gz", hash = "sha256:4ac8676560e93492bdd0540d67672424e26f1eb9a41a266d5248eb09b00dc4d2", size = 3907, upload-time = "2024-11-13T20:28:22.971Z" } +sdist = { url = "https://files.pythonhosted.org/packages/1c/b5/f306f884fd775aff4195ca1d8c7a1426829cd7060ff50b293be23e1ad869/opentelemetry_instrumentation_together-0.34.0.tar.gz", hash = "sha256:f8968d2aaae123e556e9bd7ce9213f40888a180a8014382bb738cff0bc8de8a1", size = 3907, upload-time = "2024-12-12T21:02:28.939Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/e4/6a/56f3a5abea0a3086d24a32398231e0a578c01b7fad174cd69cdd644fb87d/opentelemetry_instrumentation_together-0.33.12-py3-none-any.whl", hash = "sha256:6a1941e3d02b1505bd79a1ef3540d1fe15bf4f61c72cc162445d25d8715386b3", size = 5284, upload-time = "2024-11-13T20:27:42.798Z" }, + { url = "https://files.pythonhosted.org/packages/78/82/32bc20923c9ecd4495a01bc2dcabd377e4fd82c9cf334ed3ad3a81afaf02/opentelemetry_instrumentation_together-0.34.0-py3-none-any.whl", hash = "sha256:9b5069c3a294c161d8ad638a6d234484a2c600f77260902fb8e15afdd8dfdd33", size = 5267, upload-time = "2024-12-12T21:01:51.576Z" }, ] [[package]] name = "opentelemetry-instrumentation-transformers" -version = "0.33.12" +version = "0.34.0" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "opentelemetry-api" }, @@ -6657,14 +6658,14 @@ dependencies = [ { name = "opentelemetry-semantic-conventions" }, { name = "opentelemetry-semantic-conventions-ai" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/3f/25/f73bfae73a466b0ee206168a0ec2212db694132f0af75ff7bf8d7da74488/opentelemetry_instrumentation_transformers-0.33.12.tar.gz", hash = "sha256:d7b9c0d4bd71b834a79c2522455799feb7e76148e1dd371408e9907e847e8d6a", size = 3714, upload-time = "2024-11-13T20:28:24.399Z" } +sdist = { url = "https://files.pythonhosted.org/packages/02/54/1ab4fb5409cf6c48f7b0c0a48b39cbea70b4083d994719ba0975ba9a9580/opentelemetry_instrumentation_transformers-0.34.0.tar.gz", hash = "sha256:586b146509a90900486039850f5f3d63256c7f1546e1a897912ba454aa14e5af", size = 3714, upload-time = "2024-12-12T21:02:29.913Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/12/ef/4305bdf6af7161c2b38d5341b25bb2a0817f8ba40bf370bb6eeb131a223c/opentelemetry_instrumentation_transformers-0.33.12-py3-none-any.whl", hash = "sha256:14c3f3831a892ae38f8bb85240c195ed95e8fa996f60930e2e4f00bb73073036", size = 5255, upload-time = "2024-11-13T20:27:43.888Z" }, + { url = "https://files.pythonhosted.org/packages/25/e9/081aeb69bf4170a5d88de48db11837cce136649b35304ab0ac7164fcc06a/opentelemetry_instrumentation_transformers-0.34.0-py3-none-any.whl", hash = "sha256:984cf5e0f4ef31662382019e3a18edf821f8ce3c20d53aeea68cee5718aad752", size = 5241, upload-time = "2024-12-12T21:01:53.001Z" }, ] [[package]] name = "opentelemetry-instrumentation-urllib3" -version = "0.49b0" +version = "0.54b1" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "opentelemetry-api" }, @@ -6673,14 +6674,14 @@ dependencies = [ { name = "opentelemetry-util-http" }, { name = "wrapt" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/fb/fd/79fa96e997a9ba9f90dd6fd9bd20c67db8b965dea035e54b864665a2508d/opentelemetry_instrumentation_urllib3-0.49b0.tar.gz", hash = "sha256:33db59eafc80877c225467bf71dfe098874dd7f4463a4f12c61fb7dbcd3b4e31", size = 15432, upload-time = "2024-11-05T19:22:23.261Z" } +sdist = { url = "https://files.pythonhosted.org/packages/ed/6f/76a46806cd21002cac1bfd087f5e4674b195ab31ab44c773ca534b6bb546/opentelemetry_instrumentation_urllib3-0.54b1.tar.gz", hash = "sha256:0d30ba3b230e4100cfadaad29174bf7bceac70e812e4f5204e681e4b55a74cd9", size = 15697, upload-time = "2025-05-16T19:04:07.709Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/82/56/6339b51038142ffacc33821d1cf9a3cf91d9c9166088a5c7d862d40000bb/opentelemetry_instrumentation_urllib3-0.49b0-py3-none-any.whl", hash = "sha256:672855f033e608c857353b6e098551f70088664fbec227f4ea5d90463d602adc", size = 12847, upload-time = "2024-11-05T19:21:34.14Z" }, + { url = "https://files.pythonhosted.org/packages/ff/7a/d75bec41edb6deaf1d2859bab66a84c8ba03e822e7eafdb245da205e53f6/opentelemetry_instrumentation_urllib3-0.54b1-py3-none-any.whl", hash = "sha256:e87958c297ddd36d30e1c9069f34a9690e845e4ccc2662dd80e99ed976d4c03e", size = 13123, upload-time = "2025-05-16T19:03:14.053Z" }, ] [[package]] name = "opentelemetry-instrumentation-vertexai" -version = "0.33.12" +version = "0.34.0" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "opentelemetry-api" }, @@ -6688,14 +6689,14 @@ dependencies = [ { name = "opentelemetry-semantic-conventions" }, { name = "opentelemetry-semantic-conventions-ai" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/70/99/355c73ba6fb1679f32caa5579d9956dd3e0d40fa2205b41932694bd54696/opentelemetry_instrumentation_vertexai-0.33.12.tar.gz", hash = "sha256:a4ff534f24d4e1caecc621bea1ad19905bafc8ebf2fd1506e9eb1ae8f2a7831a", size = 4356, upload-time = "2024-11-13T20:28:25.514Z" } +sdist = { url = "https://files.pythonhosted.org/packages/f0/63/37f55389efdffcb167ec6e5cdb6b01cfeaaab01becac80d300aff547c2f5/opentelemetry_instrumentation_vertexai-0.34.0.tar.gz", hash = "sha256:4db963d487a4c26875c50dfeddfb589d998cc46b3cb89dc9a3f1083352b9e607", size = 4343, upload-time = "2024-12-12T21:02:32.037Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/5c/e6/04cb7853674e4412d929d7c3172b073b402b3ee8f049e57dce042bdd5a2d/opentelemetry_instrumentation_vertexai-0.33.12-py3-none-any.whl", hash = "sha256:cf61cdc08bb6cb4dcbb0b59d1d0432cc1b0b7bee8fa25c69ae4886cf048204c4", size = 5789, upload-time = "2024-11-13T20:27:45.036Z" }, + { url = "https://files.pythonhosted.org/packages/4a/11/6dbf0defdfeeeaf4bb2037732bedbe020e048157642519cd022b33af84e6/opentelemetry_instrumentation_vertexai-0.34.0-py3-none-any.whl", hash = "sha256:d9206a65a416159597676ac60d1331abdc3844e98982126c155e1cacd939d395", size = 5773, upload-time = "2024-12-12T21:01:55.753Z" }, ] [[package]] name = "opentelemetry-instrumentation-watsonx" -version = "0.33.12" +version = "0.34.0" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "opentelemetry-api" }, @@ -6703,14 +6704,14 @@ dependencies = [ { name = "opentelemetry-semantic-conventions" }, { name = "opentelemetry-semantic-conventions-ai" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/5b/f4/13359f1ef849d87010e494f503ce0d90c0b37f25b3cdf1ce58f0eaf0aa0b/opentelemetry_instrumentation_watsonx-0.33.12.tar.gz", hash = "sha256:98d537e3e9a919eab87f1f5f487679dd642d0742032635001b974c2154cedc0b", size = 6552, upload-time = "2024-11-13T20:28:26.346Z" } +sdist = { url = "https://files.pythonhosted.org/packages/2e/ed/78fafee5b64f728c5d8958f0fa558148674fb878b5585873e7d76708fa18/opentelemetry_instrumentation_watsonx-0.34.0.tar.gz", hash = "sha256:149a2ec1c6aa476c6258d7f00fc7951220ea8cc23be9a7a1273009377b9df0a4", size = 6552, upload-time = "2024-12-12T21:02:32.962Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/a3/f6/74c9e14dc3324a9e5fb9a31c1ae29f92d25afe7930d1430dc55923867547/opentelemetry_instrumentation_watsonx-0.33.12-py3-none-any.whl", hash = "sha256:76bde9b15ca9be9fa6124b7e09606203103b77e7fa05227b8c9145fd2a782102", size = 7457, upload-time = "2024-11-13T20:27:46.861Z" }, + { url = "https://files.pythonhosted.org/packages/9c/ef/4b2189eda9ed49f4ea69e6b102351944d7e17ab90bfa6ff451ee20c1c97d/opentelemetry_instrumentation_watsonx-0.34.0-py3-none-any.whl", hash = "sha256:85d352880c8abccba92c728cbea7cab455a4acb454d43ed0037b6afecdb3a90c", size = 7442, upload-time = "2024-12-12T21:01:58.071Z" }, ] [[package]] name = "opentelemetry-instrumentation-weaviate" -version = "0.33.12" +version = "0.34.0" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "opentelemetry-api" }, @@ -6718,48 +6719,48 @@ dependencies = [ { name = "opentelemetry-semantic-conventions" }, { name = "opentelemetry-semantic-conventions-ai" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/1d/50/38b46f295c4f28301d6aea15aeddcbc9550bb51a005da95045310549191f/opentelemetry_instrumentation_weaviate-0.33.12.tar.gz", hash = "sha256:1d14949e2123e5a2bd0eb149d8281713b33623d3f09f7aa587d4fca130d11b70", size = 4635, upload-time = "2024-11-13T20:28:27.189Z" } +sdist = { url = "https://files.pythonhosted.org/packages/68/47/9f0fc2310ef155edd22ae8ee3444e76d91a100a1579b40d034d85d2b0806/opentelemetry_instrumentation_weaviate-0.34.0.tar.gz", hash = "sha256:b69294e0b6b2fc5b90cd389c1a2bc75d18ed09f075ab589a61a0bcbe049ef9db", size = 4654, upload-time = "2024-12-12T21:02:34.344Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/28/e7/16d9a936d716af84546045fa62e593079069e14c667d049602b6e19d6e31/opentelemetry_instrumentation_weaviate-0.33.12-py3-none-any.whl", hash = "sha256:afa500e59bd7059495c6190decb1dd57dc620e17181c9543bd91e26afce74dcd", size = 6428, upload-time = "2024-11-13T20:27:48.71Z" }, + { url = "https://files.pythonhosted.org/packages/6a/9f/f55c020a3619d31dd39d32e376d8d5f8f6322f82b1c27acd5666503d6643/opentelemetry_instrumentation_weaviate-0.34.0-py3-none-any.whl", hash = "sha256:79eaa9be4393702d7b3cc938f3d01d82371d4a236326b01819002bac3f118194", size = 6410, upload-time = "2024-12-12T21:02:00.604Z" }, ] [[package]] name = "opentelemetry-proto" -version = "1.28.0" +version = "1.33.1" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "protobuf" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/c9/63/ac4cef4d30ea0ca1d2153ad2fc62d91d1cf3b89b0e4e5cbd61a8c567885f/opentelemetry_proto-1.28.0.tar.gz", hash = "sha256:4a45728dfefa33f7908b828b9b7c9f2c6de42a05d5ec7b285662ddae71c4c870", size = 34331, upload-time = "2024-11-05T19:14:59.503Z" } +sdist = { url = "https://files.pythonhosted.org/packages/f6/dc/791f3d60a1ad8235930de23eea735ae1084be1c6f96fdadf38710662a7e5/opentelemetry_proto-1.33.1.tar.gz", hash = "sha256:9627b0a5c90753bf3920c398908307063e4458b287bb890e5c1d6fa11ad50b68", size = 34363, upload-time = "2025-05-16T18:52:52.141Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/86/94/c0b43d16e1d96ee1e699373aa59f14a3aa2e7126af3f11d6adc5dcc531cd/opentelemetry_proto-1.28.0-py3-none-any.whl", hash = "sha256:d5ad31b997846543b8e15504657d9a8cf1ad3c71dcbbb6c4799b1ab29e38f7f9", size = 55832, upload-time = "2024-11-05T19:14:40.446Z" }, + { url = "https://files.pythonhosted.org/packages/c4/29/48609f4c875c2b6c80930073c82dd1cafd36b6782244c01394007b528960/opentelemetry_proto-1.33.1-py3-none-any.whl", hash = "sha256:243d285d9f29663fc7ea91a7171fcc1ccbbfff43b48df0774fd64a37d98eda70", size = 55854, upload-time = "2025-05-16T18:52:36.269Z" }, ] [[package]] name = "opentelemetry-sdk" -version = "1.28.0" +version = "1.33.1" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "opentelemetry-api" }, { name = "opentelemetry-semantic-conventions" }, { name = "typing-extensions" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/0c/5b/a509ccab93eacc6044591d5ec437d8266e76f893d0389bbf7e5592c7da32/opentelemetry_sdk-1.28.0.tar.gz", hash = "sha256:41d5420b2e3fb7716ff4981b510d551eff1fc60eb5a95cf7335b31166812a893", size = 156155, upload-time = "2024-11-05T19:15:00.451Z" } +sdist = { url = "https://files.pythonhosted.org/packages/67/12/909b98a7d9b110cce4b28d49b2e311797cffdce180371f35eba13a72dd00/opentelemetry_sdk-1.33.1.tar.gz", hash = "sha256:85b9fcf7c3d23506fbc9692fd210b8b025a1920535feec50bd54ce203d57a531", size = 161885, upload-time = "2025-05-16T18:52:52.832Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/c3/fe/c8decbebb5660529f1d6ba65e50a45b1294022dfcba2968fc9c8697c42b2/opentelemetry_sdk-1.28.0-py3-none-any.whl", hash = "sha256:4b37da81d7fad67f6683c4420288c97f4ed0d988845d5886435f428ec4b8429a", size = 118692, upload-time = "2024-11-05T19:14:41.669Z" }, + { url = "https://files.pythonhosted.org/packages/df/8e/ae2d0742041e0bd7fe0d2dcc5e7cce51dcf7d3961a26072d5b43cc8fa2a7/opentelemetry_sdk-1.33.1-py3-none-any.whl", hash = "sha256:19ea73d9a01be29cacaa5d6c8ce0adc0b7f7b4d58cc52f923e4413609f670112", size = 118950, upload-time = "2025-05-16T18:52:37.297Z" }, ] [[package]] name = "opentelemetry-semantic-conventions" -version = "0.49b0" +version = "0.54b1" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "deprecated" }, { name = "opentelemetry-api" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/ee/c8/433b0e54143f8c9369f5c4a7a83e73eec7eb2ee7d0b7e81a9243e78c8e80/opentelemetry_semantic_conventions-0.49b0.tar.gz", hash = "sha256:dbc7b28339e5390b6b28e022835f9bac4e134a80ebf640848306d3c5192557e8", size = 95227, upload-time = "2024-11-05T19:15:01.443Z" } +sdist = { url = "https://files.pythonhosted.org/packages/5b/2c/d7990fc1ffc82889d466e7cd680788ace44a26789809924813b164344393/opentelemetry_semantic_conventions-0.54b1.tar.gz", hash = "sha256:d1cecedae15d19bdaafca1e56b29a66aa286f50b5d08f036a145c7f3e9ef9cee", size = 118642, upload-time = "2025-05-16T18:52:53.962Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/25/05/20104df4ef07d3bf5c3fd6bcc796ef70ab4ea4309378a9ba57bc4b4d01fa/opentelemetry_semantic_conventions-0.49b0-py3-none-any.whl", hash = "sha256:0458117f6ead0b12e3221813e3e511d85698c31901cac84682052adb9c17c7cd", size = 159214, upload-time = "2024-11-05T19:14:43.047Z" }, + { url = "https://files.pythonhosted.org/packages/0a/80/08b1698c52ff76d96ba440bf15edc2f4bc0a279868778928e947c1004bdd/opentelemetry_semantic_conventions-0.54b1-py3-none-any.whl", hash = "sha256:29dab644a7e435b58d3a3918b58c333c92686236b30f7891d5e51f02933ca60d", size = 194938, upload-time = "2025-05-16T18:52:38.796Z" }, ] [[package]] @@ -6773,11 +6774,11 @@ wheels = [ [[package]] name = "opentelemetry-util-http" -version = "0.49b0" +version = "0.54b1" source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/a3/99/377ef446928808211b127b9ab31c348bc465c8da4514ebeec6e4a3de3d21/opentelemetry_util_http-0.49b0.tar.gz", hash = "sha256:02928496afcffd58a7c15baf99d2cedae9b8325a8ac52b0d0877b2e8f936dd1b", size = 7863, upload-time = "2024-11-05T19:22:26.973Z" } +sdist = { url = "https://files.pythonhosted.org/packages/a8/9f/1d8a1d1f34b9f62f2b940b388bf07b8167a8067e70870055bd05db354e5c/opentelemetry_util_http-0.54b1.tar.gz", hash = "sha256:f0b66868c19fbaf9c9d4e11f4a7599fa15d5ea50b884967a26ccd9d72c7c9d15", size = 8044, upload-time = "2025-05-16T19:04:10.79Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/66/0e/ab0a89b315d0bacdd355a345bb69b20c50fc1f0804b52b56fe1c35a60e68/opentelemetry_util_http-0.49b0-py3-none-any.whl", hash = "sha256:8661bbd6aea1839badc44de067ec9c15c05eab05f729f496c856c50a1203caf1", size = 6945, upload-time = "2024-11-05T19:21:37.81Z" }, + { url = "https://files.pythonhosted.org/packages/a4/ef/c5aa08abca6894792beed4c0405e85205b35b8e73d653571c9ff13a8e34e/opentelemetry_util_http-0.54b1-py3-none-any.whl", hash = "sha256:b1c91883f980344a1c3c486cffd47ae5c9c1dd7323f9cbe9fdb7cadb401c87c9", size = 7301, upload-time = "2025-05-16T19:03:18.18Z" }, ] [[package]] @@ -9860,7 +9861,7 @@ wheels = [ [[package]] name = "traceloop-sdk" -version = "0.33.12" +version = "0.34.0" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "aiohttp" }, @@ -9906,9 +9907,9 @@ dependencies = [ { name = "pydantic" }, { name = "tenacity" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/7e/0d/d7d413e9fe907a8abc33e6f93044484d158722b5ca0bfe22e1ef9ad4e729/traceloop_sdk-0.33.12.tar.gz", hash = "sha256:999ae50b1e5773b2802a8b3e8585c3826b7867bba032a88b6f30ec2727225dda", size = 19768, upload-time = "2024-11-13T20:29:26.67Z" } +sdist = { url = "https://files.pythonhosted.org/packages/0a/b1/fd7360d97c651098da505e95600e067a7eedb1b78635b2f1d23545ee4a46/traceloop_sdk-0.34.0.tar.gz", hash = "sha256:4aa26003dfa2e417f73728bd847284a12d6da43a946dd588603a0966e753b3e6", size = 19808, upload-time = "2024-12-12T21:03:41.647Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/ce/13/53c2ab6ac27804769314554a062e0651a44db2360be47e21cf0a29d202ee/traceloop_sdk-0.33.12-py3-none-any.whl", hash = "sha256:d47a474afbf4a68ff38a702dbaca7b17d2d4f0b0e14dc2f1560b6bdd3859ac75", size = 25932, upload-time = "2024-11-13T20:29:25.174Z" }, + { url = "https://files.pythonhosted.org/packages/c5/e8/c89cc77c272312930cc263c45fbd2a648536e93358611bf03dba6f176a0b/traceloop_sdk-0.34.0-py3-none-any.whl", hash = "sha256:1cc3e5be9dd2765212feaa5655e1f43ddc66739585d78d9c81134428a2a7d927", size = 25944, upload-time = "2024-12-12T21:03:39.565Z" }, ] [[package]]