litellm/tests/unit/integrations
Ishaan 16287b7133 fix(ci): unblock dependency checks (#48055)
* fix(deps): upgrade Python dependencies

* fix(deps): lock patched Python dependency versions

* fix(otel): preserve JSON export with the updated SDK

* test(otel): cover JSON export with the updated SDK

* test(otel): update SDK log records and worker discovery

* fix(ui): align Lens peer dependency with patched Next.js

* fix(otel): hand the AgentOps exporter a session it owns for the JWT header

OpenTelemetry 1.45 moved the OTLP HTTP exporter's requests.Session behind a
transport, so writing the minted JWT to the exporter's private _session raised,
was swallowed, and every AgentOps batch went out unauthenticated. The exporter
now creates the session, passes it through the public session= argument and
sets the Authorization header on it

The tests now drive a real export, serving the token endpoint with respx and
capturing the OTLP POST with requests_mock, instead of reading SDK internals

* fix(otel): skip FastAPI instrumentation once the app has started

opentelemetry-instrumentation-fastapi 0.66b1 wraps build_middleware_stack instead
of calling add_middleware, so instrumenting an app whose middleware stack is
already built no longer raises. The call silently did nothing while we still
registered the global response propagator. Check the built stack up front and
skip both, which is what the existing test expects

* test(otel): assert exported requests and span flags instead of SDK internals

OpenTelemetry 1.45 dropped the exporter's private _headers and sets the W3C
random-trace-id flag on new traces, so traceresponse ends in -03. The Langtrace
and Langfuse tests now export through requests_mock and check the URL and headers
the collector receives, and the traceresponse tests expect the server span's own
trace flags

* fix(deps): bump dashboard zod to 4.6.6 and make the vendored Lens use it

zod 4.6.5 is flagged by the dependency scan and blocks osv-scan on every PR. A
root bump alone leaves the vendored Lens tarball, which pins zod 4.6.5 exactly,
on its own nested copy, so a top-level zod override points every consumer at the
root version and the lock carries a single zod 4.6.6

* test(ui): stub the tag summary fetch the client panel retry refreshes

Retrying the model breakdown refetches /tag/summary, which this integration test
left on the real network. When that request failed before the assertion, the
panel switched to its summary error and unmounted the model row, so the test
passed or failed by timing

* fix(deps): drop the duplicated Lens override from the dashboard package.json

* style(ui): format the client panel fetch stub

Backport notes for stable 1.104.x. The dependency files (pyproject.toml, uv.lock, the dashboard package.json and package-lock.json) were left out of this pick and are re-derived on the line in the following commits

ClientUsagePanel.integration.test.tsx does not exist on this line, so its stub hunk is dropped
test_otel_v2_mount.py keeps the line's version because the traceresponse tests the squash edits are not on this line
test_litellm_logging.py keeps the line's version because the Langtrace export test the squash edits is not on this line
test_opentelemetry.py keeps the line's trace import without SpanLimits, which only the newer tests on main use

---------

Co-authored-by: Yuneng Jiang <yuneng@berri.ai>
(cherry picked from commit 8f9894fbd6)
2026-10-23 15:17:20 -07:00
..
arize chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
azure_storage chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
bitbucket chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
cloudzero chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
code_interpreter_interception chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
compression_interception chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
datadog chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
dotprompt chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
focus chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
gcs_bucket test: add __init__.py to migrated tests/unit packages 2026-09-20 11:53:49 +00:00
gcs_pubsub test: add __init__.py to migrated tests/unit packages 2026-09-20 11:53:49 +00:00
gitlab chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
helicone test: add __init__.py to migrated tests/unit packages 2026-09-20 11:53:49 +00:00
langfuse chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
levo test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
litellm_agent test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
mavvrik_focus test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
newrelic chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
open_telemetry chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
opik test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
otel fix(ci): unblock dependency checks (#48055) 2026-10-23 15:17:20 -07:00
pointfive test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
SlackAlerting chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
vector_store_integrations test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
websearch_interception chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
__init__.py test: add __init__.py to migrated tests/unit packages 2026-09-20 11:53:49 +00:00
conftest.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
rubrik_test_helpers.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_adaptive_concurrency.py fix(s3_v2): upload fresh events first, drop terminal failures and hour-old retries by default, opt-in adaptive concurrency (#43022) (#43354) 2026-09-26 15:46:08 -07:00
test_agentops.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_anthropic_cache_control_hook.py fix(caching): stand default cache points down when extra_body hides a direct client mark (#43341) 2026-10-19 22:25:41 -07:00
test_athina.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_azure_sentinel.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_braintrust_logging.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_braintrust_span_name.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_custom_guardrail.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_custom_guardrail_recursion.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_custom_prompt_management.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_deepeval.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_galileo.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_guardrail_logging_sync.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_helicone.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_langfuse.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_langfuse_otel.py fix(ci): unblock dependency checks (#48055) 2026-10-23 15:17:20 -07:00
test_langsmith_init.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_lunary.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_mlflow.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_openmeter.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_opentelemetry.py fix(ci): unblock dependency checks (#48055) 2026-10-23 15:17:20 -07:00
test_opentelemetry_dynamic_imports.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_opik_utils.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_otel_guardrail_violation_spans.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_otel_team_attributes_matrix.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_prometheus_api_promql_escape.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_prometheus_budget_metric_guard.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_prometheus_budget_metrics_db_lookups.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_prometheus_budget_metrics_timeout.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_prometheus_cache_metrics.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_prometheus_caller_identity.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_prometheus_carried_budget_state.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_prometheus_client_ip_user_agent.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_prometheus_custom_metadata_label_counts.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_prometheus_deployment_state_proxy_rejects.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_prometheus_end_user_cardinality.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_prometheus_input_sequence_length_label.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_prometheus_invalid_key_filtering.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_prometheus_labels.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_prometheus_mcp_tool_metrics.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_prometheus_media_generation_metrics.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_prometheus_metric_name_consistency.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_prometheus_metrics_endpoint.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_prometheus_missing_metrics.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_prometheus_none_metadata.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_prometheus_overhead_with_guardrails.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_prometheus_queue_guardrail_metrics.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_prometheus_rate_limit_labels.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_prometheus_remaining_tokens_router_fallback.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_prometheus_requested_model_cardinality.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_prometheus_service_tier_label.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_prometheus_services.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_prometheus_spend_capture_rate.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_prometheus_spend_logs_metadata.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_prometheus_stream_label.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_prometheus_token_detail_metrics.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_prometheus_user_team_metrics.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_prometheus_zero_cost_metric.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_prompt_manager_ssti.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_responses_background_cost.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_rubrik.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_s3.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_s3_v2.py fix(s3_v2): upload fresh events first, drop terminal failures and hour-old retries by default, opt-in adaptive concurrency (#43022) (#43354) 2026-09-26 15:46:08 -07:00
test_shadow_eval_logger.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_weave_otel.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00