litellm/tests/unit/integrations
devin-ai-integration[bot] 50f5cc9bbb
feat(otel v2): excluded_services opt-out for datastore spans on tenant destinations (#43278)
* feat(otel): excluded_services opt-out for datastore spans on tenant destinations

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): keep upstream support unchanged

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(otel): excluded_services resolves from the otel callback config only

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(otel): name the otel callback logger so excluded_services owner lookup matches

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): assert no aux datastore traces reach the tenant sink

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): read bogus-start proxy log from the results dir

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): read only this invocation's bogus-start proxy log

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): assert operator kept db spans over the whole recorded window

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): split operator db-span asserts by trace scope

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(otel): build the otel logger after preset callbacks and validate the exclusion env at boot

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(otel): tolerate a bogus exclusion env when callback config wins

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(otel): keep bogus exclusion env fatal when a preset parses it

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(otel): hoist the preset check out of the callback loop

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(otel): use a rule-scoped pyright suppression

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(otel): log and drop unknown excluded_services instead of failing boot

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): wait for the operator spend-writer span before checking the tenant for postgres

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(otel v2): leave callback init and boot untouched when excluded_services is unset

Read callback_settings.otel.excluded_services directly instead of making the otel callback build its own logger, and drop the new boot-time parse of callback_settings.otel, so a proxy without the setting behaves exactly as on main

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(otel v2): normalize callback_settings excluded_services without rereading OTel env vars

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(otel v2): log and ignore malformed excluded_services instead of failing startup

Lowercase and trim names, drop non-string items, and add an integration matrix over endpoints, clients, cache hits, destination outages and setting shapes

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(otel v2): cover failed upstream calls in the excluded_services matrix

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(otel v2): pin operator Langfuse credentials in preset-only excluded_services tests

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mrinal <mrinal@berri.ai>
2026-09-30 12:20:27 -07:00
..
arize test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
azure_storage test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
bitbucket test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
cloudzero test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
code_interpreter_interception test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
compression_interception test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
datadog test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
dotprompt test: finish the non-proxy half of tests/test_litellm (#43281) 2026-09-25 22:43:41 -07:00
focus test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
gcs_bucket test: add __init__.py to migrated tests/unit packages 2026-09-20 11:53:49 +00:00
gcs_pubsub test: add __init__.py to migrated tests/unit packages 2026-09-20 11:53:49 +00:00
gitlab test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
helicone test: add __init__.py to migrated tests/unit packages 2026-09-20 11:53:49 +00:00
langfuse test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
levo test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
litellm_agent test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
mavvrik_focus test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
newrelic test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
open_telemetry test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
opik test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
otel feat(otel v2): excluded_services opt-out for datastore spans on tenant destinations (#43278) 2026-09-30 12:20:27 -07:00
pointfive test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
SlackAlerting feat(proxy): email alerts at configured percentages of a team member budget (#42665) 2026-09-26 02:42:45 +00:00
vector_store_integrations feat(guardrails): scan retrieved vector store chunks with the request's pre-call guardrails (#43271) 2026-09-26 14:18:38 -07:00
websearch_interception refactor(anthropic): rename experimental_pass_through to pass_through (#43329) 2026-09-26 13:00:50 -07:00
zerobus test(zerobus): move tests into active CI selection (#43235) 2026-09-25 23:35:15 +00:00
__init__.py test: add __init__.py to migrated tests/unit packages 2026-09-20 11:53:49 +00:00
conftest.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
rubrik_test_helpers.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_adaptive_concurrency.py fix(s3_v2): upload fresh events first, drop terminal failures and hour-old retries by default, opt-in adaptive concurrency (#43022) 2026-09-26 14:58:28 -07:00
test_agentops.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_anthropic_cache_control_hook.py fix(caching): stand default cache points down when extra_body hides a direct client mark (#43341) 2026-09-26 16:44:30 -07:00
test_athina.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_azure_sentinel.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_braintrust_logging.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_braintrust_span_name.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_custom_guardrail.py feat(guardrails): scan retrieved vector store chunks with the request's pre-call guardrails (#43271) 2026-09-26 14:18:38 -07:00
test_custom_guardrail_recursion.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_custom_prompt_management.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_deepeval.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_galileo.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_guardrail_logging_sync.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_helicone.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_langfuse.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_langfuse_otel.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_langsmith_init.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_lunary.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_mlflow.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_openmeter.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_opentelemetry.py fix(langtrace): deliver spans to app.langtrace.ai/api/trace with x-api-key (#43322) 2026-09-26 16:31:52 -07:00
test_opentelemetry_dynamic_imports.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_opik_utils.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_otel_guardrail_violation_spans.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_otel_team_attributes_matrix.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_api_promql_escape.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_budget_metric_guard.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_budget_metrics_db_lookups.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_budget_metrics_timeout.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_cache_metrics.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_caller_identity.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_carried_budget_state.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_client_ip_user_agent.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_custom_metadata_label_counts.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_deployment_state_proxy_rejects.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_end_user_cardinality.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_input_sequence_length_label.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_invalid_key_filtering.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_labels.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_mcp_tool_metrics.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_media_generation_metrics.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_metric_name_consistency.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_metrics_endpoint.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_missing_metrics.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_none_metadata.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_overhead_with_guardrails.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_queue_guardrail_metrics.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_rate_limit_labels.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_remaining_tokens_router_fallback.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_requested_model_cardinality.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_service_tier_label.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_services.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_spend_capture_rate.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_spend_logs_metadata.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_stream_label.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_token_detail_metrics.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_user_team_metrics.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_zero_cost_metric.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prompt_manager_ssti.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_responses_background_cost.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_rubrik.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_s3.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_s3_v2.py fix(s3_v2): upload fresh events first, drop terminal failures and hour-old retries by default, opt-in adaptive concurrency (#43022) 2026-09-26 14:58:28 -07:00
test_shadow_eval_logger.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_weave_otel.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00