mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-29 06:33:28 +00:00
0657ed46cf
* fix(langfuse_otel): derive trace-level name, input, output and tags on the generation span Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(langfuse_otel): keep every reasoning summary in the derived observation output Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(langfuse_otel): leave joined traces untouched by derived trace fields Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(langfuse_otel): restore preset cache key fallback and type the derived-field helpers Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(langfuse_otel): keep trace tags off joined traces entirely Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(langfuse_otel): tolerate out-of-band upstream requests and add sink outage chaos cell Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): build the langfuse rig config without mutating the loaded yaml Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(langfuse_otel): flatten output items with itertools.chain to satisfy LIT014 budget Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(langfuse_otel): type the _RecordingSpan and _emitted unit helpers Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(langfuse_otel): keep joined-trace tags, mask trace io, leave v2 trace io to the root Joined traces (existing_trace_id) keep the caller tags they carried before, and only the derived trace fields are skipped for them. mask_input and mask_output now redact the derived trace input and output, matching the vanilla Langfuse callback. The v2 mapper derives the trace name only, because the root observation already carries the trace input and output there. Uses the public preset cache key helper, moves imports to module top, restores the original formatting of the unit test file, and adds integration cells for the v2 llm_only scope and SDK sync/async calls. * fix(langfuse_otel): split string caller tags the way Langfuse reads them A caller tags value given as a comma list or a JSON array string (body metadata.tags or the langfuse_tags header) used to reach Langfuse raw, where ingestion split it into separate tags. Merging it with the derived tags kept it as one tag, so parse it the same way before the merge * test(otel): expect langfuse.trace.name on the v2 langfuse_otel generation span * fix(langfuse_otel): read string mask flags the way the classic callback does The langfuse_mask_input / langfuse_mask_output headers arrive as strings, so "false" redacted the derived trace io. Parse them with str_to_bool and fall back to truthiness, as the classic langfuse callback does * fix(langfuse_otel): derive trace fields only on traces litellm starts A litellm span that joins a trace an app or caller started (inbound traceparent, the app's active span, a caller-provided parent span) no longer gets a derived trace name, input, output or tags, like a request joining an existing_trace_id. Langfuse applies the later-starting span's trace update, so the derived values overwrote the app's own trace name and io. OTel v2 derives the name on the request root only when it has no parent, and the generation span keeps main's caller-only trace name. * test(langfuse_otel): keep main's OTel v2 generation key set in the arize family cell The OTel v2 generation span no longer carries a derived langfuse.trace.name, so the pinned key set goes back to main's. * fix(langfuse_otel): name the OTel v2 request root once, from the request's own call The v2 Langfuse logger re-derived langfuse.trace.name on the request root for every LLM call in the request. A post_call llm_as_a_judge guardrail's judge call replaced the caller's trace_name on the root with litellm-acompletion, and with another v2 callback (otel, arize_phoenix) listed before langfuse_otel that vendor's root span gained langfuse.trace.name on every request. The derived name now fills only an unnamed root this logger's provider started, and never comes from a call litellm makes for its own features. A caller trace_name still names the root on every call, as on main. * fix(langfuse_otel): keep the derived trace fields from pushing other attributes off a full span The derived trace name, tags, input and output were written after the OpenInference attributes with no room check, so a span near the SDK's attribute limit (about 34 tools at the default 128) evicted keys main exported, such as openinference.span.kind, metadata and llm.model_name, and raised the dropped count. They are now written last and only into the room the limit leaves next to the error attributes a failed call adds, so a full span exports what it would without them. * test(langfuse_otel): give the full-span cells' derived request a caller trace id, as the joined one's traceparent does * chore(langfuse_otel): drop the docstrings and comments the house rule disallows and type the new tests' parameters * test(langfuse_otel): set the span attribute limit the full-span tests run at and keep Final locals out of the app-trace loop --------- Co-authored-by: yucheng <yucheng@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: yucheng-berri <293926326+yucheng-berri@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| _support | ||
| audio_tests | ||
| base_sdk_tests | ||
| benchmarks | ||
| code_coverage_tests | ||
| documentation_tests | ||
| e2e | ||
| e2e_harness | ||
| fixtures | ||
| harness_e2e | ||
| integration | ||
| rust-python-harness | ||
| test_litellm_rust | ||
| typing | ||
| unit | ||
| windows_tests | ||
| __init__.py | ||
| _fake_openai_endpoint_server.py | ||
| _flush_vcr_cache.py | ||
| _live_test_helpers.py | ||
| _master_key.py | ||
| _openai_record_replay_proxy.py | ||
| _process_helpers.py | ||
| _vcr_conftest_common.py | ||
| _vcr_redis_persister.py | ||
| _wait_helpers.py | ||
| _ws_vcr.py | ||
| AGENTS.md | ||
| capturing_transport.py | ||
| eval_swe_bench.py | ||
| fake_openai_endpoint.py | ||
| gettysburg.wav | ||
| large_text.py | ||
| openai_batch_completions.jsonl | ||
| pyrightconfig.json | ||
| README.MD | ||
| test_logging.conf | ||
| test_rust_python_harness.py | ||
| VCR_TESTING.md | ||
| white_100x100.png | ||
In total litellm runs 1000+ tests
[02/20/2025] Update:
To make it easier to contribute and map what behavior is tested,
we've started mapping the litellm directory in tests/unit
This folder can only run mock tests.