scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-07 08:26:10 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 36
litellm/tests/test_litellm/integrations
History
Exact
ryan-crabbe-berri b76def0e5d
test: require a match= on broad pytest.raises, and drop duplicate parametrize cases (#37769) 
`pytest.raises(Exception)` with no `match=` passes on any error that broad. A
TypeError from a refactor, a botched fixture, an import that moved: all of them
read as the rejection the test claims to police, so the test goes green for the
wrong reason and stays green after the behaviour it guards is gone.

PT011 closes that gap for the 317 sites B017 could not reach, because B017 only
fires on a single-statement body with no `as e` binding. Each pattern here is the
message the code actually raised, recorded by running the sites under a plugin
that logged the concrete type and text per call site, so the assertions describe
observed behaviour rather than a guess. Where a site raises more than one message
across its parametrize cases, the pattern is an alternation of what was seen;
where the exception carries an empty `str()` and puts the text on `.message`, the
site keeps a narrow `noqa` with the reason.

PT014 removes four parametrize cases that were listed twice. The duplicate re-runs
an assertion that already passed, and it usually marks a case someone meant to
vary and forgot to edit.
2026-08-20 20:24:49 -07:00
..
arize fix(arize): trace MCP tool calls instead of crashing on CallToolResult (#36453) 2026-08-10 16:37:12 -07:00
azure_storage fix(azure_storage): honor AZURE_STORAGE_ENDPOINT_SUFFIX for sovereign clouds (#35806) 2026-08-04 16:06:41 -07:00
bitbucket test: enforce PT012 so a pytest.raises block cannot hide dead assertions (#37748) 2026-08-20 19:36:26 -07:00
cloudzero test: require a match= on broad pytest.raises, and drop duplicate parametrize cases (#37769) 2026-08-20 20:24:49 -07:00
code_interpreter_interception feat(sandbox): reuse e2b container across requests when metadata.session_id is set (#31688) 2026-06-30 18:58:09 -07:00
compression_interception feat(spend): track prompt compression saved tokens in daily spend aggregates (#33810) 2026-07-18 17:47:54 -07:00
datadog fix(datadog): normalize alias and request tag values before submission (#37682) 2026-08-20 16:03:47 -07:00
dotprompt style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
focus test: require a match= on broad pytest.raises, and drop duplicate parametrize cases (#37769) 2026-08-20 20:24:49 -07:00
gcs_bucket fix cloud storage file guards 2026-05-01 15:34:11 -07:00
gcs_pubsub Litellm fix GitHub action testing (#11163) 2025-05-26 14:41:42 -07:00
gitlab test: require a match= on broad pytest.raises, and drop duplicate parametrize cases (#37769) 2026-08-20 20:24:49 -07:00
helicone test: run the 30 test files stranded in the second mirror (#37595) 2026-08-20 10:59:43 -07:00
langfuse Litellm oss staging 050626 (#29774) 2026-06-05 13:51:51 -07:00
levo test: require a match= on broad pytest.raises, and drop duplicate parametrize cases (#37769) 2026-08-20 20:24:49 -07:00
litellm_agent Agent Builder - support new experimental agent builder, to ensure agents pass compliance checks (#21817) 2026-02-21 15:32:47 -08:00
mavvrik_focus chore: litellm oss staging (#31185) 2026-06-26 09:17:44 -07:00
newrelic perf: build log messages lazily so filtered-out log records cost nothing (#35703) 2026-08-04 04:34:52 +00:00
open_telemetry fix(management): cover the new control plane route in CI's two guards 2026-07-27 09:28:32 -07:00
opik Litellm oss staging 030626 (#29578) 2026-06-03 11:01:51 -07:00
otel test: require a match= on broad pytest.raises, and drop duplicate parametrize cases (#37769) 2026-08-20 20:24:49 -07:00
SlackAlerting fix(alerting): back off a day after a deprecation pass raises and label the alert in the UI 2026-08-17 17:57:37 -07:00
websearch_interception Merge remote-tracking branch 'origin/litellm_internal_staging' into devin_ai_lit_5033_websearch_interception_spend 2026-08-08 12:00:06 -07:00
rubrik_test_helpers.py Litellm oss staging 04 21 2026 2 (#26569) 2026-05-20 21:25:19 -07:00
test_agentops.py Litellm fix GitHub action testing (#11163) 2025-05-26 14:41:42 -07:00
test_anthropic_cache_control_hook.py fix(router): make prompt caching affinity aware of auto-injected cache_control (#37689) 2026-08-20 16:10:27 -07:00
test_athina.py Litellm fix GitHub action testing (#11163) 2025-05-26 14:41:42 -07:00
test_azure_sentinel.py test: remove tests that never execute 2026-08-12 10:45:38 -07:00
test_braintrust_logging.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_braintrust_span_name.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_custom_guardrail.py fix(guardrails): classify all 4xx HTTPException guardrail blocks as intervened (#33821) 2026-07-27 16:50:13 -07:00
test_custom_guardrail_recursion.py fix: recursive pydantic issue (#19531) 2026-01-22 19:56:41 -08:00
test_custom_prompt_management.py [Fix] CI/CD - Fix failing proxy and core integration tests (#17926) 2025-12-13 10:11:01 -08:00
test_deepeval.py fix: use fastuuid helper (#14903) 2025-09-25 15:47:01 -07:00
test_galileo.py feat(galileo): add health check support for UI callback test (#29908) 2026-06-08 13:57:03 -07:00
test_guardrail_logging_sync.py fix(guardrails): keep guardrail information in spend logs when the caller sends its own metadata 2026-07-24 16:20:44 -07:00
test_helicone.py test(helicone): stub the anthropic module unconditionally 2026-08-01 11:26:09 -07:00
test_langfuse.py test: require a match= on broad pytest.raises, and drop duplicate parametrize cases (#37769) 2026-08-20 20:24:49 -07:00
test_langfuse_otel.py fix(otel): bound and shut down credential-scoped tracer providers (#36591) 2026-08-18 16:21:58 -07:00
test_langsmith_init.py fix(proxy): sanitize per-key callback config out of logged metadata 2026-07-24 22:16:50 -07:00
test_lunary.py fix(litellm): honor dict-form reasoning_effort in the bridge escape hatch and serialize custom tool calls in helicone and lunary logs 2026-08-01 11:26:09 -07:00
test_mlflow.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_openmeter.py test: remove tests that never execute 2026-08-12 10:45:38 -07:00
test_opentelemetry.py perf(otel): build the credential-scoped tracer Resource once per logger (#37542) 2026-08-19 16:40:40 -07:00
test_opentelemetry_dynamic_imports.py Make grpc dependency optional (#19447) 2026-01-20 19:03:52 -08:00
test_opik_utils.py chore(ci): merge oss branch (#33784) 2026-07-17 23:22:13 +00:00
test_otel_guardrail_violation_spans.py fix(guardrails): keep guardrail information in spend logs when the caller sends its own metadata 2026-07-24 16:20:44 -07:00
test_otel_team_attributes_matrix.py feat: propagate team_id and team_alias to all child OTEL spans (#28273) 2026-05-19 15:31:25 -07:00
test_prometheus_api_promql_escape.py fix(prometheus): quote api_key for PromQL string literal in spend lookup 2026-05-01 21:38:19 +00:00
test_prometheus_budget_metric_guard.py fix(prometheus): skip budget metric DB lookups when gauges are NoOpMetric (#32834) 2026-07-10 20:25:47 -07:00
test_prometheus_budget_metrics_db_lookups.py perf(auth): negative-cache missing user/key lookups on the request hot path (#32368) 2026-07-08 09:59:57 +03:00
test_prometheus_budget_metrics_timeout.py feat: litellm oss staging (#31935) 2026-07-03 09:27:31 +05:30
test_prometheus_cache_metrics.py fix(prometheus): populate cache write token metrics for OpenAI-style usage (#34803) 2026-07-27 12:28:19 -07:00
test_prometheus_client_ip_user_agent.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_prometheus_custom_metadata_label_counts.py Fix Prometheus remaining metric zero values (#27348) 2026-05-06 17:22:20 -07:00
test_prometheus_end_user_cardinality.py perf: cap Prometheus end-user metric cardinality with TTL + LRU eviction (#27272) 2026-05-06 13:35:13 -07:00
test_prometheus_invalid_key_filtering.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_prometheus_labels.py test: rename tests that a later definition shadowed 2026-08-12 11:15:54 -07:00
test_prometheus_mcp_tool_metrics.py feat(prometheus): expose MCP tool metadata in Prometheus metrics (#31899) 2026-07-02 10:56:35 +03:00
test_prometheus_media_generation_metrics.py feat(prometheus): expose video duration and image count consumption metrics (#33138) 2026-07-13 18:51:13 -07:00
test_prometheus_metric_name_consistency.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_prometheus_metrics_endpoint.py perf(prometheus): render /metrics off the event loop and coalesce concurrent scrapes (#37702) 2026-08-20 16:08:22 -07:00
test_prometheus_missing_metrics.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_prometheus_none_metadata.py fix: guard against None metadata in prometheus metrics (#21489) 2026-02-18 12:40:45 -08:00
test_prometheus_overhead_with_guardrails.py feat(prometheus): add litellm_total_overhead_latency_metric (SDK overhead + guardrails) (#31593) 2026-06-30 17:34:17 +08:00
test_prometheus_queue_guardrail_metrics.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_prometheus_rate_limit_labels.py fix(prometheus): read v3 rate limiter remaining values for per-key model gauges (#33119) 2026-07-13 14:27:56 -07:00
test_prometheus_remaining_tokens_router_fallback.py fix(prometheus): emit litellm_remaining_tokens_metric for Bedrock and Vertex (#27705) 2026-05-13 17:40:59 -07:00
test_prometheus_service_tier_label.py feat(otel): stamp service tier attributes on inference spans (#35679) 2026-08-03 23:10:01 -07:00
test_prometheus_services.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_prometheus_spend_logs_metadata.py fix(prometheus): expose project_alias in custom metadata labels (LIT-3741) (#31784) 2026-07-01 10:44:02 +08:00
test_prometheus_stream_label.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_prometheus_token_detail_metrics.py fix(prometheus): populate cache write token metrics for OpenAI-style usage (#34803) 2026-07-27 12:28:19 -07:00
test_prometheus_user_team_metrics.py fix(proxy): count only active users toward license seat limit (#31227) 2026-06-29 18:01:02 -07:00
test_prompt_manager_ssti.py fix(security): sandbox jinja2 in gitlab/arize/bitbucket prompt managers 2026-05-02 09:14:02 +00:00
test_responses_background_cost.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_rubrik.py fix(guardrails/rubrik): attribute blocked requests to the caller that made them (#35734) 2026-08-03 19:56:24 -07:00
test_s3.py feat(s3): support SSE-KMS encryption params on both S3 logging paths (#35291) 2026-07-30 21:18:33 -07:00
test_s3_v2.py fix(s3_v2): sign S3 object URLs with S3SigV4Auth so encoded paths verify (#35726) 2026-08-05 01:14:48 +00:00
test_shadow_eval_logger.py feat(shadow_eval)!: gate the per-key budget on dollar spend instead of turns (#37555) 2026-08-20 14:55:21 -07:00
test_weave_otel.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00